8000 sequence parallel optimization for latest transformer · Issue #312 · volcengine/verl · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

sequence parallel optimization for latest transformer #312

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
Tracked by #354
eric-haibin-lin opened this issue Feb 19, 2025 · 1 comment · Fixed by #357
Closed
Tracked by #354

sequence parallel optimization for latest transformer #312

eric-haibin-lin opened this issue Feb 19, 2025 · 1 comment · Fixed by #357
Labels
call for contribution good first issue Good for newcomers

Comments

@eric-haibin-lin
Copy link
Collaborator

verl does not yet support sequence parlallelism with the latest version of transformer > 0.48. The implementation is at verl/models/transformers/monkey_patch.py
@eric-haibin-lin eric-haibin-lin mentioned this issue Feb 23, 2025
Closed
17 tasks
@xffxff
Copy link
Contributor
xffxff commented Feb 24, 2025

I'm working on this

@xffxff xffxff mentioned this issue Feb 24, 2025
Merged
vermouth1992 pushed a commit that referenced this issue Feb 24, 2025
@xffxff
feat: add support for ulysses sequence parallel for transformers >= 0…
d36422b 
….48 (#357)

close #312 

Add support for ulysses sp for transformers >= 0.48

I've tested transformers 0.45.0, 0.46.0, 0.47.0, 0.48.0 and 0.49.0,
using sp=2 with the following script in my local env
```bash
#!/bin/bash

set -ex
VERSIONS=("4.45.0" "4.46.0" "4.47.0" "4.48.0" "4.49.0")

for version in "${VERSIONS[@]}"; do
    echo "Testing with Transformers version ${version}"
    echo "----------------------------------------"
    
    pip install "transformers==${version}"
    
    PYTHONPATH=./ torchrun --nproc_per_node=2 tests/model/test_transformers_ulysses.py
    
    echo "----------------------------------------"
    echo "Completed testing for version ${version}"
    echo ""
done
```
wangyuchen333 pushed a commit to wangyuchen333/verl that referenced this issue Apr 25, 2025
@xffxff
feat: add support for ulysses sequence parallel for transformers >= 0…
1714c81 
….48 (volcengine#357)

close volcengine#312 

Add support for ulysses sp for transformers >= 0.48

I've tested transformers 0.45.0, 0.46.0, 0.47.0, 0.48.0 and 0.49.0,
using sp=2 with the following script in my local env
```bash
#!/bin/bash

set -ex
VERSIONS=("4.45.0" "4.46.0" "4.47.0" "4.48.0" "4.49.0")

for version in "${VERSIONS[@]}"; do
    echo "Testing with Transformers version ${version}"
    echo "----------------------------------------"
    
    pip install "transformers==${version}"
    
    PYTHONPATH=./ torchrun --nproc_per_node=2 tests/model/test_transformers_ulysses.py
    
    echo "----------------------------------------"
    echo "Completed testing for version ${version}"
    echo ""
done
```
histmeisah pushed a commit to SJTU-IAAR/verl that referenced this issue Apr 27, 2025
@xffxff
feat: add support for ulysses sequence parallel for transformers >= 0…
4364549 
….48 (volcengine#357)

close volcengine#312 

Add support for ulysses sp for transformers >= 0.48

I've tested transformers 0.45.0, 0.46.0, 0.47.0, 0.48.0 and 0.49.0,
using sp=2 with the following script in my local env
```bash
#!/bin/bash

set -ex
VERSIONS=("4.45.0" "4.46.0" "4.47.0" "4.48.0" "4.49.0")

for version in "${VERSIONS[@]}"; do
    echo "Testing with Transformers version ${version}"
    echo "----------------------------------------"
    
    pip install "transformers==${version}"
    
    PYTHONPATH=./ torchrun --nproc_per_node=2 tests/model/test_transformers_ulysses.py
    
    echo "----------------------------------------"
    echo "Completed testing for version ${version}"
    echo ""
done
```
yumc-afk pushed a commit to yumc-afk/verl that referenced this issue May 18, 2025
@xffxff
feat: add support for ulysses sequence parallel for transformers >= 0…
1e17a1e 
….48 (volcengine#357)

close volcengine#312 

Add support for ulysses sp for transformers >= 0.48

I've tested transformers 0.45.0, 0.46.0, 0.47.0, 0.48.0 and 0.49.0,
using sp=2 with the following script in my local env
```bash
#!/bin/bash

set -ex
VERSIONS=("4.45.0" "4.46.0" "4.47.0" "4.48.0" "4.49.0")

for version in "${VERSIONS[@]}"; do
    echo "Testing with Transformers version ${version}"
    echo "----------------------------------------"
    
    pip install "transformers==${version}"
    
    PYTHONPATH=./ torchrun --nproc_per_node=2 tests/model/test_transformers_ulysses.py
    
    echo "----------------------------------------"
    echo "Completed testing for version ${version}"
    echo ""
done
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
call for contribution good first issue Good for newcomers
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: add support for ulysses sequence parallel for transformers >= 0.48 xffxff/verl
2 participants
@eric-haibin-lin @xffxff
0