8000 feat(policy): Add DexVLA policy [Community Contributed] by JayceWen · Pull Request #767 · huggingface/lerobot · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

feat(policy): Add DexVLA policy [Community Contributed] #767

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open

feat(policy): Add DexVLA policy [Community Contributed] #767

wants to merge 8 commits into from

Conversation

JayceWen
Copy link
@JayceWen JayceWen commented Feb 24, 2025 •
edited
Loading

What this does

Add new VLA policy: DexVLA.

@JayceWen JayceWen changed the title 2025 02 20 add dexvla 2025 02 24 add dexvla Feb 24, 2025
@imstevenpmwork imstevenpmwork added enhancement Suggestions for new features or improvements policies Items related to robot policies labels Mar 4, 2025
@JayceWen
Copy link
Author

@imstevenpmwork Hi, is this "Review required At least 1 approving review is required by reviewers with write access." means you need review my code or someone else?

@imstevenpmwork
Copy link
Collaborator

Hello @JayceWen ! Thanks for your contribution. Indeed, someone from the team needs to review this PR before it can get merged to upstream. I will ping the policy experts 😄

@imstevenpmwork imstevenpmwork requested a review from Cadene March 14, 2025 16:00
@imstevenpmwork
Copy link
Collaborator

Hi @JayceWen,

After discussing with the team, we've decided that we'd like to merge this PR. But before doing so, we'd like you to provide a bit more context about it. Specifically, it would be helpful to include things like a README for the policy, an example, or some supporting visuals (e.g., videos, images, or graphs), maybe also an explanation on how you tested and which set-up you used.

Since we haven’t had many PRs introducing new policies yet, we're still shaping the guidelines for contributors. Your PR could serve as a great reference point 😊

Thanks for your work!

@JayceWen
Copy link
Author

Hi @imstevenpmwork,

Thanks for replying and it's very exciting to contribute to such an open-sourced project designed exactly for robotics. I have added more commits to improve the code and a README file for quickly train the policy.

Thanks for your help.

@imstevenpmwork
Copy link
Collaborator
imstevenpmwork commented Mar 18, 2025 •
edited
Loading

Hello @JayceWen , happy to see this taking shape 😄 some comments:

  • Can you add some details about training time, inference speed, and what data it has been pretrained on ?
  • Can you mention explicitly at the top of the readme that this policy is "Community Contributed" ?

Once you finish polishing the rough edges, I will ask the policy guys to review the code and merge this

@imstevenpmwork imstevenpmwork changed the title 2025 02 24 add dexvla feat(policy): Add DexVLA policy [Community Contributed] Mar 18, 2025
@JayceWen
Copy link
Author

Hi, @imstevenpmwork ,
I have already added descriptions on training time, inference speed, and pretraining data. Besides, I have added "Community Contributed" at the beginning of the README.

@imstevenpmwork imstevenpmwork requested review from imstevenpmwork and removed request for Cadene March 19, 2025 16:41
@JayceWen
Copy link
Author

Hi @imstevenpmwork,

I have checked the requirements for dexvla and corrected the required version of qwen_vl_utils.

@imstevenpmwork
Copy link
Collaborator

Hi @JayceWen

Thanks for putting in the effort on this—great work so far! I just have a few follow-up questions and requests to ensure everything aligns with the project’s standards:

  1. License Headers: Could you please add the proper license headers to all the newly added files?

  2. Module Structure: I noticed there are two separate modules (policy_heads and qwe2_vla). Was there a specific reason for splitting them into subfolders? For consistency, I’d suggest keeping all the register() calls in configuration_dexvla.py and placing the scripts directly under dexvla/. That said, I’m open to hearing your thoughts—let me know what you think! 😊

  3. Commit History: Since this is a big PR, I’d prefer to Rebase & Merge it. To make the commit history cleaner and more meaningful, could you please rebase/squash/rename the commits as you see fit?

Let me know if you have any questions or need help with any of this!

@JayceWen JayceWen force-pushed the 2025_02_20_add_dexvla branch from e2a8fc2 to ea81279 Compare March 31, 2025 13:46
@imstevenpmwork
Copy link
Collaborator

Thanks for your contributions @JayceWen

I noticed there are some comments in Chinese in the code. To maintain consistency across the codebase, we prefer keeping comments in English only.

Would you mind updating these to English when you have a chance? We really appreciate your help in keeping the codebase clean and accessible for everyone!

@JayceWen
Copy link
Author
JayceWen commented Apr 2, 2025 •
edited
Loading

Hi @imstevenpmwork,

Thanks for the feedback.

  1. I have already added the license header and cleared the commit history.

  2. Regarding the model structure, since DexVLA combines a VLM backbone and a policy head, I suggest separating these two components to help users better understand our framework. Additionally, this separation makes it easier to replace DexVLA with a new policy head or VLM backbone. What's more, current VLAs tend to follow a dual-system architecture. Maybe a hierarchical structure would better organize the code?

  3. Thanks for reminder, I will change them.

Thank you again for your support, I will continue working on this PR.

@JayceWen
Copy link
Author

Hi @imstevenpmwork,

I noticed that the process has been stuck at the "Review required" stage for a long time. I want to check if there are any questions or concerns regarding the code, or if there's anything further I can assist with to help move things forward. I'm more than happy to continue working on improving the code as needed.

Please feel free to let me know if there is anything that requires further attention or improvement.

I really appreciate for your support and look forward to your reply. 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@imstevenpmwork imstevenpmwork Awaiting requested review from imstevenpmwork

@danaaubakirova danaaubakirova Awaiting requested review from danaaubakirova

At least 1 approving review is required to merge this pull request.

Assignees

@JayceWen JayceWen

@imstevenpmwork imstevenpmwork

Labels
enhancement Suggestions for new features or improvements policies Items related to robot policies
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@JayceWen @imstevenpmwork
0