8000 Use DataCollatorForCompletionOnlyLM in order to train LLM to follow instructions · Issue #845 · huggingface/course · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Use DataCollatorForCompletionOnlyLM in order to train LLM to follow instructions #845

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
xavier-owkin opened this issue Mar 20, 2025 · 0 comments
Open

Use DataCollatorForCompletionOnlyLM in order to train LLM to follow instructions #845

xavier-owkin opened this issue Mar 20, 2025 · 0 comments

Comments

@xavier-owkin
Copy link
xavier-owkin commented Mar 20, 2025 •
edited
Loading

In Chapter 11 of the course, in the file FineTuning with SFTTrainer (3.mdx), you explain how to fine-tune a DeepSeek model with SFTTrainer on an instruction dataset.

Why don't you use the DataCollatorForCompletionOnlyLM data collator in the SFTTrainer to avoid computing gradients and back-propagating on the user question tokens? The default data collator in the SFTTrainer is DataCollatorForLanguageModeling, which means that in your example the LLM will also learn to complete the user query, which is not intended as far as I understand.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@xavier-owkin
0