8000 [Feature Request] key-level granularity in `skip_existing` · Issue #352 · pytorch/tensordict · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

[Feature Request] key-level granularity in skip_existing #352

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
matteobettini opened this issue Apr 20, 2023 · 0 comments
Open

[Feature Request] key-level granularity in skip_existing #352

matteobettini opened this issue Apr 20, 2023 · 0 comments
Assignees
@vmoens
Labels
enhancement New feature or request

Comments

@matteobettini
Copy link
Contributor
matteobettini commented Apr 20, 2023 •
edited
Loading

Currently skip_existing operates on all keys without any granularity.

This is a problem in RL when in a loss module for example you may want to skip existing "values" but you definitely never want to skip existing "memory" in a memory based model (RNN). AKA if you use skip_existing on memory keys you will never update your memory.

This is needed to support rnns in torch rl (issue pytorch/rl#1060)

We need a solution to make skip_existing more granular.

This is really simple and consists in feeding to the set_skip_existing funtion the keys we actually want to skip.

with set_skip_existing(["value", "value_target]):
     loss(td) # Will use existing values but not existing hidden memory

by default, if no keys are passed, the beahvior remains the same as the current set_skip_existing=True
@matteobettini matteobettini added the enhancement New feature or request label Apr 20, 2023
@matteobettini matteobettini changed the title [Feature Request] selecitve skip_existing [Feature Request] granular skip_existing Apr 20, 2023
@matteobettini matteobettini changed the title [Feature Request] granular skip_existing [Feature Request] key-level granularity in skip_existing Apr 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@vmoens vmoens

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@vmoens @matteobettini
0