8000 Fix nemo_llama_to_hf conversion by tdene · Pull Request #8000 · NVIDIA/NeMo · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Fix nemo_llama_to_hf conversion #8000

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 10 commits into from
Feb 29, 2024
Merged

Fix nemo_llama_to_hf conversion #8000

merged 10 commits into from
Feb 29, 2024

Conversation

tdene
Copy link
Contributor
@tdene tdene commented Dec 8, 2023 •
edited
Loading

What does this PR do ?

Fix convert_nemo_llama_to_hf.py:

  • Correctly account for megatron_amp_O2 flag
  • Save HF model with correct precision
  • Transfer over the NeMo tokenizer, instead of using the possibly-incompatible default tokenizer
  • Make sure to save fast version of tokenizer
  • Resize the model's embedding tensor to match the new tokenizer's vocab
  • Correct a typo in how-to-use example

PR Type:

  • New Feature
  • Bugfix
  • Documentation

Who can review?

@ericharper

Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.

Additional Information

  • Related to # (issue)

tdene added 5 commits December 8, 2023 01:44
@tdene
Account for amp_O2 in nemo_llama_to_hf conversion
b081883 
Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>
@tdene
Package converted model with new tokenizer not old
e2e037f 
Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>
@tdene
Account for variations in megatron_amp_O2 behavior
2827cee 
Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>
@tdene
Resize the embeddings matrix
f87601e 
Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>
@tdene
Correct precision when saving to HF folder
bc31a5c 
Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>
@tdene tdene force-pushed the llama_conversion_script_amp_fix branch 3 times, most recently from d268d49 to 4cd8d33 Compare December 8, 2023 10:50
@tdene
Fix typo in sample script
8479afb 
Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>
@tdene tdene force-pushed the llama_conversion_script_amp_fix branch from 8e4a675 to 8479afb Compare December 8, 2023 12:16
@tdene
Fix typo in logging
5c117d7 
Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>
@tdene tdene force-pushed the llama_conversion_script_amp_fix branch from 89e4620 to 5c117d7 Compare December 15, 2023 22:07
@github-actions GitHub Actions
Copy link
Contributor

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

@github-actions github-actions bot added the stale label Dec 30, 2023
@github-actions GitHub Actions
Copy link
Contributor
github-actions bot commented Jan 6, 2024

This PR was closed because it has been inactive for 7 days since being marked as stale.

@github-actions github-actions bot closed this Jan 6, 2024
@cuichenx cuichenx reopened this Feb 9, 2024
@github-actions github-actions bot removed the stale label Feb 10, 2024
@github-actions GitHub Actions
Copy link
Contributor

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

@github-actions github-actions bot added the stale label Feb 25, 2024
@cuichenx cuichenx removed the stale label Feb 25, 2024
@cuichenx
Merge branch 'main' into llama_conversion_script_amp_fix
7167754 
Signed-off-by: Chen Cui <chcui@nvidia.com>
@cuichenx
fix O2 issue properly in peft mixin
b961e69 
Signed-off-by: Chen Cui <chcui@nvidia.com>
@github-actions github-actions bot added the NLP label Feb 29, 2024
@cuichenx
Copy link
Collaborator

jenkins

github-advanced-security[bot]
github-advanced-security bot found potential problems Feb 29, 2024
View reviewed changes
scripts/nlp_language_modeling/convert_nemo_llama_to_hf.py
@@ -17,8 +17,9 @@
from collections import OrderedDict

import torch
from omegaconf import open_dict

Check notice

Code scanning / CodeQL

Unused import

Import of 'open_dict' is not used.
@cuichenx cuichenx merged commit 974b74c into NVIDIA:main Feb 29, 2024
@cuichenx cuichenx mentioned this pull request Feb 29, 2024
Closed
8 tasks
arendu pushed a commit that referenced this pull request Feb 29, 2024
@tdene @arendu
cherrypicked OOM fix from #8000
3617018 
Signed-off-by: arendu <adithya.r@gmail.com>
zpx01 pushed a commit to zpx01/NeMo that referenced this pull request Mar 8, 2024
@tdene @pre-commit-ci
@cuichenx @zpx01
Fix nemo_llama_to_hf conversion (NVIDIA#8000)
ad026d9 
* Account for amp_O2 in nemo_llama_to_hf conversion

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* Package converted model with new tokenizer not old

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* Account for variations in megatron_amp_O2 behavior

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* Resize the embeddings matrix

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* Correct precision when saving to HF folder

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* Fix typo in sample script

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* Fix typo in logging

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix O2 issue properly in peft mixin

Signed-off-by: Chen Cui <chcui@nvidia.com>

---------

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>
Signed-off-by: Chen Cui <chcui@nvidia.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Chen Cui <chcui@nvidia.com>
Signed-off-by: Zeeshan Patel <zeeshanp@berkeley.edu>
Agoniii pushed a commit to Agoniii/NeMo that referenced this pull request Mar 15, 2024
@tdene @pre-commit-ci
@cuichenx @Agoniii
Fix nemo_llama_to_hf conversion (NVIDIA#8000)
7a8fef3 
* Account for amp_O2 in nemo_llama_to_hf conversion

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* Package converted model with new tokenizer not old

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* Account for variations in megatron_amp_O2 behavior

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* Resize the embeddings matrix

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* Correct precision when saving to HF folder

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* Fix typo in sample script

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* Fix typo in logging

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix O2 issue properly in peft mixin

Signed-off-by: Chen Cui <chcui@nvidia.com>

---------

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>
Signed-off-by: Chen Cui <chcui@nvidia.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Chen Cui <chcui@nvidia.com>
Signed-off-by: Agoniii <815244047@qq.com>
JRD971000 pushed a commit that referenced this pull request Mar 15, 2024
@tdene @pre-commit-ci
@cuichenx @JRD971000
Fix nemo_llama_to_hf conversion (#8000)
f0d5178 
* Account for amp_O2 in nemo_llama_to_hf conversion

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* Package converted model with new tokenizer not old

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* Account for variations in megatron_amp_O2 behavior

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* Resize the embeddings matrix

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* Correct precision when saving to HF folder

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* Fix typo in sample script

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* Fix typo in logging

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix O2 issue properly in peft mixin

Signed-off-by: Chen Cui <chcui@nvidia.com>

---------

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>
Signed-off-by: Chen Cui <chcui@nvidia.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Chen Cui <chcui@nvidia.com>
Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com>
pablo-garay pushed a commit that referenced this pull request Mar 19, 2024
@tdene @pre-commit-ci
@cuichenx @pablo-garay
Fix nemo_llama_to_hf conversion (#8000)
f86a198 
* Account for amp_O2 in nemo_llama_to_hf conversion

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* Package converted model with new tokenizer not old

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* Account for variations in megatron_amp_O2 behavior

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* Resize the embeddings matrix

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* Correct precision when saving to HF folder

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* Fix typo in sample script

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* Fix typo in logging

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix O2 issue properly in peft mixin

Signed-off-by: Chen Cui <chcui@nvidia.com>

---------

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>
Signed-off-by: Chen Cui <chcui@nvidia.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Chen Cui <chcui@nvidia.com>
Signed-off-by: Pablo Garay <pagaray@nvidia.com>
@tdene tdene deleted the llama_conversion_script_amp_fix branch March 21, 2024 18:26
@tdene tdene mentioned this pull request Apr 24, 2024
Closed
3 tasks
rohitrango pushed a commit to rohitrango/NeMo that referenced this pull request Jun 25, 2024
@tdene @pre-commit-ci @cuichenx
Fix nemo_llama_to_hf conversion (NVIDIA#8000)
43096e3 
* Account for amp_O2 in nemo_llama_to_hf conversion

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* Package converted model with new tokenizer not old

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* Account for variations in megatron_amp_O2 behavior

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* Resize the embeddings matrix

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* Correct precision when saving to HF folder

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* Fix typo in sample script

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* Fix typo in logging

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix O2 issue properly in peft mixin

Signed-off-by: Chen Cui <chcui@nvidia.com>

---------

Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>
Signed-off-by: Chen Cui <chcui@nvidia.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Chen Cui <chcui@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cuichenx cuichenx cuichenx approved these changes

Assignees
No one assigned
Labels
NLP
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@tdene @cuichenx
0