Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DPO training from locale file only works for jsonl files #1282

Open
6 of 8 tasks
PhilipMay opened this issue Feb 9, 2024 · 0 comments · May be fixed by #1283
Open
6 of 8 tasks

DPO training from locale file only works for jsonl files #1282

PhilipMay opened this issue Feb 9, 2024 · 0 comments · May be fixed by #1283
Labels
bug Something isn't working

Comments

@PhilipMay
Copy link
Contributor

PhilipMay commented Feb 9, 2024 •
edited

Please check that this issue hasn't been reported before.

  • I searched previous Bug Reports didn't find any similar reports.

Expected Behavior

When I do sft and start training with a locale parquet.gz file it works. It should also work for DPO with locale files.

Current behaviour

It does not work with an other locale file format than jsonl. It does not work with a locale parquet.gz

See below:

Traceback (most recent call last):
  File "/root/miniconda3/envs/py3.10/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/root/miniconda3/envs/py3.10/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/workspace/axolotl/src/axolotl/cli/train.py", line 59, in <module>
    fire.Fire(do_cli)
  File "/root/miniconda3/envs/py3.10/lib/python3.10/site-packages/fire/core.py", line 141, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
  File "/root/miniconda3/envs/py3.10/lib/python3.10/site-packages/fire/core.py", line 475, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
  File "/root/miniconda3/envs/py3.10/lib/python3.10/site-packages/fire/core.py", line 691, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
  File "/workspace/axolotl/src/axolotl/cli/train.py", line 35, in do_cli
    return do_train(parsed_cfg, parsed_cli_args)
  File "/workspace/axolotl/src/axolotl/cli/train.py", line 51, in do_train
    dataset_meta = load_rl_datasets(cfg=cfg, cli_args=cli_args)
  File "/workspace/axolotl/src/axolotl/cli/__init__.py", line 400, in load_rl_datasets
    train_dataset, eval_dataset = load_prepare_dpo_datasets(cfg)
  File "/workspace/axolotl/src/axolotl/utils/data.py", line 958, in load_prepare_dpo_datasets
    train_dataset = load_split(cfg.datasets, cfg)
  File "/workspace/axolotl/src/axolotl/utils/data.py", line 931, in load_split
    ds = load_dataset(  # pylint: disable=invalid-name
  File "/root/miniconda3/envs/py3.10/lib/python3.10/site-packages/datasets/load.py", line 2523, in load_dataset
    builder_instance = load_dataset_builder(
  File "/root/miniconda3/envs/py3.10/lib/python3.10/site-packages/datasets/load.py", line 2195, in load_dataset_builder
    dataset_module = dataset_module_factory(
  File "/root/miniconda3/envs/py3.10/lib/python3.10/site-packages/datasets/load.py", line 1848, in dataset_module_factory
    raise FileNotFoundError(
FileNotFoundError: Couldn't find a dataset script at /workspace/axolotl/persist/data/rag_tiny.parquet.gz/rag_tiny.parquet.gz.py or any data file in the same directory.

Steps to reproduce

See example config:

base_model: TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T
model_type: LlamaForCausalLM
tokenizer_type: LlamaTokenizer
is_llama_derived_model: true
special_tokens:

load_in_8bit: false
load_in_4bit: true
strict: false
adapter: qlora
lora_model_dir:
lora_r: 16
lora_alpha: 16
lora_dropout: 0.05
lora_target_linear: true
lora_fan_in_fan_out:

dpo_beta: 0.1
rl: dpo

datasets:
  - path: persist/data/rag_tiny.parquet.gz
    split: train
    type: chatml.intel

dataset_prepared_path: ./persist/prepared_datasets
output_dir: ./persist/test
save_steps: 50
save_total_limit: 4

sequence_len: 1024
sample_packing: false
pad_to_sequence_len: true
gradient_accumulation_steps: 2
micro_batch_size: 2
num_epochs: 1
optimizer: adamw_bnb_8bit
weight_decay: 0.0
learning_rate: 0.0000002
lr_scheduler: cosine
train_on_inputs: false
group_by_length: false
bf16: false
fp16: true
tf32: false
gradient_checkpointing: true
flash_attention: false
warmup_steps: 100

eval_steps:
val_set_size: 0
early_stopping_patience:
logging_steps: 1

Config yaml

see above

Possible solution

will provide a PR in a min.

Which Operating Systems are you using?

  • Linux
  • macOS
  • Windows

Python Version

3.10

axolotl branch-commit

main

Acknowledgements

  • My issue title is concise, descriptive, and in title casing.
  • I have searched the existing issues to make sure this bug has not been reported yet.
  • I am using the latest version of axolotl.
  • I have provided enough information for the maintainers to reproduce and diagnose the issue.
@PhilipMay PhilipMay added the bug Something isn't working label Feb 9, 2024
@PhilipMay PhilipMay linked a pull request Feb 9, 2024 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix data loading for DPO and other formats like parquet. PhilipMay/axolotl
1 participant
@PhilipMay