Rewarped 🌀 is a platform for reinforcement learning in parallel differentiable multiphysics simulation, built on NVIDIA/warp. Rewarped supports:
- parallel environments: to train RL agents at scale.
- differentiable simulation: to compute batched analytic gradients for optimization.
- multiphysics: (CRBA, FEM, MPM, XPBD) physics solvers and coupling to support interaction between rigid bodies, articulations, and various deformables.
We use Rewarped to re-implement a variety of RL tasks from prior works, and demonstrate that first-order RL algorithms (which use differentiable simulation to compute first-order analytic gradients) can be scaled to a range of challenging manipulation and locomotion tasks that involve interaction between rigid bodies, articulations, and deformables.
For control and reinforcement learning algorithms, see
etaoxing/mineral.
We have tested on the following environment: RTX 4090, Ubuntu 22.04, CUDA 12.5, Python 3.10, PyTorch 2.
conda create -n rewarped python=3.10
conda activate rewarped
pip install torch torchvision
pip install gym==0.23.1
pip install rewarped
# --- Example: trajectory optimization
python -m rewarped.envs.warp_examples.bounce --num_envs 4
# will create a `.usd` file in `outputs/`
# use MacOS Preview or alternatives to view
# --- Example: (first-order) reinforcement learning
pip install git+https://github.com/etaoxing/mineral
python -m mineral.scripts.run \
task=Rewarped agent=DFlexAntSAPO task.env.env_name=Ant task.env.env_suite=dflex \
logdir="workdir/RewarpedAnt4M-SAPO/$(date +%Y%m%d-%H%M%S.%2N)" \
agent.shac.max_epochs=2000 agent.shac.max_agent_steps=4.1e6 \
agent.network.actor_kwargs.mlp_kwargs.units=\[128,64,32\] \
agent.network.critic_kwargs.mlp_kwargs.units=\[64,64\] \
run=train_eval seed=1000See
mineral/docs/rewarped.mdfor scripts to reproduce the original paper experiments onrewarped==1.3.0.
|
|
|
|
| dflex • AntRun | isaacgymenvs • HandReorient | plasticinelab • RollingFlat |
|
|
|
|
| gradsim • SoftJumper | dexdeform • HandFlip | softgym • FluidMove |
Copy and modify rewarped/envs/template/task.py into rewarped/envs/<suite>/<task>.py. Put assets in assets/<suite>/.
Contributions are welcome! Please refer to CONTRIBUTING.md.
Versions take the format X.Y.Z, where:
- X.Y matches to version X.Y.* of
NVIDIA/warp. - Z is incremented for bug fixes or new features.
Compatibility with newer or older versions of NVIDIA/warp may work, but is not guaranteed or supported. There can be some breaking changes between minor versions.
Differentiable Simulation
Tasks (alphabetical)
sizhe-li/DexDeformNVlabs/DiffRLhzaskywalker/PlasticineLabgradsim/gradsimisaac-sim/IsaacGymEnvsleggedrobotics/legged_gymXingyu-Lin/softgym- ...
@inproceedings{xing2025stabilizing,
title={Stabilizing Reinforcement Learning in Differentiable Multiphysics Simulation},
author={Xing, Eliot and Luk, Vernon and Oh, Jean},
booktitle={International Conference on Learning Representations},
year={2025},
url={https://openreview.net/forum?id=DRiLWb8bJg}
}