8000 Handle low fps cameras by alexcbb · Pull Request #1356 · huggingface/lerobot · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Handle low fps cameras #1356

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Handle low fps cameras #1356

wants to merge 2 commits into from
+13 −1

Conversation

alexcbb
Copy link
Contributor
@alexcbb alexcbb commented Jun 19, 2025

What this does

When recording data with a system and low-fps cameras (<30 fps) the control is impacted by the read function (blocking call) of opencvcamera.
This PR permits to keep smooth control when low fps cameras are used by returning last frames when the delta is higher that its number of fps.

Related to the following discussion on the discord:
https://discord.com/channels/1216765309076115607/1245029082266206251/1385195636982550579

How it was tested

This was not yet tested on the real robot but might need to be checked by someone with the issue.

@CarolinePascal CarolinePascal requested review from CarolinePascal and removed request for CarolinePascal June 23, 2025 10:56
@CarolinePascal CarolinePascal self-assigned this Jun 23, 2025
@CarolinePascal CarolinePascal added enhancement Suggestions for new features or improvements dataset Issues regarding data inputs, processing, or datasets robots Issues concerning robots HW interfaces labels Jun 23, 2025
@CarolinePascal
Copy link
Collaborator

Hi @alexcbb !

I finally got time to review your PR :) With a bit of reflexion, I'm not sure the path we took is the right one.

_read_loop() already does what it is meant to do (i.e. waiting for frames and signaling when a frame is received). Adding a timer here will make it wait "for nothing" while it should wait for frames !

I feel like it would be best to modify async_read() instead, and setting the timeout to 1/dataset_fps. If the timeout is reached but the thread still alive, we can return the last frame. If the thread is dead we can raise an error.

Does that sounds sensible to you ?

Best,

Caroline.

@CarolinePascal
Copy link
Collaborator

P.S. We can even count the number of time and old frame was sent, and raise an error if the counter exceeds the ratio between the dataset fps and the camera fps !

For instance, with a dataset at 30 fps and a camera ar 25 fps, we should not send more that 1 old frame.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@CarolinePascal CarolinePascal

Labels
dataset Issues regarding data inputs, processing, or datasets enhancement Suggestions for new features or improvements robots Issues concerning robots HW interfaces
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@alexcbb @CarolinePascal
0