8000 Extend reward classifier for multiple camera views by michel-aractingi · Pull Request #626 · huggingface/lerobot · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Extend reward classifier for multiple camera views #626

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Jan 13, 2025
Merged

Extend reward classifier for multiple camera views #626

merged 2 commits into from
Jan 13, 2025

Conversation

michel-aractingi
Copy link
Contributor

What this does

(A) This PR adds the possibility to train the reward classifier with multiple camera images. The architecture includes:
1- one pretrained resnet encoder for all images (with frozen parameters)
2- each image is passed through the encoder to get a compressed representation.
3- The representations are concatenated and passed through an MLP that will be trained to predict the reward.

(B) This PR also extends the eval_on_robot.py script to load the reward classifier and the label each timestep with the predicted reward in real-time.

(C) in lerobot/common/robot_devices/control_utils.py a new function to reset the follower arm to the initial position it was in at the start of the control. This is important for collecting a dataset with rewards so we don't get redundant frames in the trajectory (related to manually reseting the robot) after the task has terminated.

How to test

  • Collect a dataset with your robot Reward classifier and training #528 .
  • Train a reward classifier with lerobot/scripts/train_hilserl_classifier.py
  • Test the reward classifier with eval_on_robot.py, example:
python lerobot/scripts/eval_on_robot.py \
    --robot-path lerobot/configs/robot/so100.yaml \
    --reward-classifier-pretrained-path outputs/classifier/checkpoints/best/pretrained_model \
    --reward-classifier-config-file lerobot/configs/policy/hilserl_classifier.yaml \
    --display-cameras 1

NOTE you can use a dataset I collected with so100 aractingi/pick_place_lego_cube_1

@michel-aractingi
1- Extended reward classifier to take multiple input images
c170105 
2- Added classifier to the `eval_on_robot.py` script to be able to predict the reward while during the rollout on the robot
3- general fixes and optimizations to the code
@michel-aractingi
Copy link
Contributor Author
michel-aractingi commented Jan 10, 2025 •
edited
Loading

@ChorntonYoel could you give it a quick review?

helper2424
helper2424 approved these changes Jan 10, 2025
View reviewed changes
Copy link
Contributor
@helper2424 helper2424 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The part related to the reward classifier looks good, I have checked locally - it converges, the only one thing is failling test

pytest tests/test_train_hilserl_classifier.py I think that need to fix it before merging

@michel-aractingi
Copy link
Contributor Author

Thanks @helper2424 ! The test issue was due to the addition of multi-camera training. I updated the test files and added additional testing function for training with two camera sources.

@helper2424
Copy link
Contributor

Looks great.

@michel-aractingi michel-aractingi merged commit 3bb5ed5 into user/michel-aractingi/2024-11-27-port-hil-serl Jan 13, 2025
1 check passed
@michel-aractingi michel-aractingi deleted the user/michel-aractingi/2025-01-10-extend-reward-classifier branch January 13, 2025 12:57
michel-aractingi added a commit that referenced this pull request Jan 22, 2025
@michel-aractingi
Extend reward classifier for multiple camera views (#626)
bbb5ba0
AdilZouitine pushed a commit that referenced this pull request Mar 24, 2025
@michel-aractingi @AdilZouitine
Extend reward classifier for multiple camera views (#626)
5ac79d5
AdilZouitine pushed a commit that referenced this pull request Mar 24, 2025
@michel-aractingi @AdilZouitine
Extend reward classifier for multiple camera views (#626)
9e15745
AdilZouitine pushed a commit that referenced this pull request Mar 28, 2025
@michel-aractingi @AdilZouitine
Extend reward classifier for multiple camera views (#626)
6139df5
michel-aractingi added a commit that referenced this pull request Apr 17, 2025
@michel-aractingi
Extend reward classifier for multiple camera views (#626)
6213982
michel-aractingi added a commit that referenced this pull request Apr 18, 2025
@michel-aractingi
Extend reward classifier for multiple camera views (#626)
181727c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@helper2424 helper2424 helper2424 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@michel-aractingi @helper2424
0