This repository is the official implementation of the paper "VolleyBots: A Testbed for Multi-Drone Volleyball Game Combining Motion Control and Strategic Play".
[ Paper ] [ Project Website ]
Zelai Xu*, Ruize Zhang*, Chao Yu+, Huining Yuan, Xiangmin Yi, Shilong Ji, Chuqi Wang, Wenhao Tang, Feng Gao, Wenbo Ding, Xinlei Chen, Yu Wang+
News | Overview | Installation | Usage | Checkpoints | Experimental Setup | Citation | Acknowledgement
🔥 [2025.06.03] (v0.1.0) Add a full 6 vs 6 volleyball task, new RL & MARL baselines, a hierarchical policy for the 3 vs 3 task, and extra reproducible training/evaluation shells and checkpoints.
[2025.02.09] (v0.0.0) Initial release with 8 tasks, 8 baselines, and reproducible training/evaluation shells & checkpoints.
Overview of the VolleyBots Testbed. VolleyBots comprises three key components: (1) Environment, supported by Isaac Sim and PyTorch, which defines entities, observations, actions, and reward functions; (2) Tasks, including 3 single-agent tasks, 3 multi-agent cooperative tasks and 3 multi-agent competitive tasks; and (3) Algorithms, encompassing RL, MARL, game-theoretic algorithms.
We use Isaac Sim as the simulation environment. You can choose to install the local version or the container version.
Download the Omniverse Isaac Sim and install the desired Isaac Sim release following the official document.
For this repository, we use the Isaac Sim version 2023.1.0-hotfix.1. Since this version is not available on the official website, you can download it directly from here. After downloading, extract the file and move the folder to the following directory:
mv isaac_sim-2023.1.0-hotfix.1 ~/.local/share/ov/pkg/Set the following environment variables to your ~/.bashrc or ~/.zshrc files :
# Isaac Sim root directory
export ISAACSIM_PATH="${HOME}/.local/share/ov/pkg/isaac_sim-2023.1.0-hotfix.1"After adding the environment variable, apply the changes by running:
source ~/.bashrcAlthough Isaac Sim comes with a built-in Python environment, we recommend using a seperate conda environment which is more flexible. We provide scripts to automate environment setup when activating/deactivating a conda environment at VolleyBots/conda_setup.
conda create -n volley python=3.10
conda activate volley
# at VolleyBots/
cp -r conda_setup/etc $CONDA_PREFIX
# re-activate the environment
conda activate volley
# install VolleyBots
pip install -e .
# verification
python -c "from omni.isaac.kit import SimulationApp"
# which torch is being used
python -c "import torch; print(torch.__path__)"VolleyBots requires specific versions of the tensordict, torchrl and orbit packages. For this repository, we manage these three packages using Git submodules to ensure that the correct versions are used. To initialize and update the submodules, follow these steps:
Get the submodules:
# at VolleyBots/
git submodule update --init --recursiveInstall tensordict:
# at VolleyBots/
cd third-party/tensordict
python setup.py developBefore install torchrl, first check and update gcc and g++:
# check gcc version, should be like: gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0 ...
gcc --version
# if not gcc 9.x, check available gcc
ls /usr/bin/gcc*
# if gcc-9 is not installed
sudo apt update && sudo apt install gcc-9
# if gcc-9 is installed
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-9 100
sudo update-alternatives --config gcc
# then follow instructions to select gcc-9.
# check gcc version again
gcc --version
# apply same update and config to g++Then install torchrl:
# at VolleyBots/
cd third-party/rl
python setup.py developAlso we need to install orbit package of Isaac Sim. Note that currently orbit package has been integrated into Isaac Lab package, but this branch still uses the standalone orbit package (this can be updated in the future). So we manage the older version of orbit package using Git submodule. To install the orbit package, follow these steps:
# at VolleyBots/
cd third-party/orbit
# create a symbolic link
ln -s ${ISAACSIM_PATH} _isaac_sim
# create environment variable
echo -e "alias orbit=$(pwd)/orbit.sh" >> ${HOME}/.bashrc
source ${HOME}/.bashrc
# building extentions
sudo apt install cmake build-essential
./orbit.sh --install # or "./orbit.sh -i"# at VolleyBots/
cd scripts
python train.py headless=true wandb.mode=disabled total_frames=50000 task=SingleJuggleVolleyball task.env.num_envs=16The Container version is easier to set up compared to the Local version. However, it's important to note that the Container version does not support real-time rendering. Therefore, it only supports the command with headless=true. You can save videos during training and upload them to Wandb.
First, make sure your computer has installed Docker, NVIDIA Driver and NVIDIA Container Toolkit. Then, you should successfully run:
# Verification
docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smiDownload the image from Docker Hub:
docker pull jimmyzhangruize/isaac-sim:2023.1.0-hotfix.1This image already includes Isaac sim, so you don't neet to download Isaac sim. However, you need to clone the VolleyBots repository to your local computer because we use VolleyBots mounted in the container.
Then, run the image:
docker run --name *** --entrypoint bash -dit --gpus all -e "ACCEPT_EULA=Y" --rm --network=host \
-e "PRIVACY_CONSENT=Y" \
-v ~/docker/isaac-sim/cache/kit:/isaac-sim/kit/cache:rw \
-v ~/docker/isaac-sim/cache/ov:/root/.cache/ov:rw \
-v ~/docker/isaac-sim/cache/pip:/root/.cache/pip:rw \
-v ~/docker/isaac-sim/cache/glcache:/root/.cache/nvidia/GLCache:rw \
-v ~/docker/isaac-sim/cache/computecache:/root/.nv/ComputeCache:rw \
-v ~/docker/isaac-sim/logs:/root/.nvidia-omniverse/logs:rw \
-v ~/docker/isaac-sim/data:/root/.local/share/ov/data:rw \
-v ~/docker/isaac-sim/documents:/root/Documents:rw \
-v ***:/root/VolleyBots:rw \
-v /data:/data \
-e "WANDB_API_KEY=***" \
jimmyzhangruize/isaac-sim:2023.1.0-hotfix.1Note that:
- You need to replace *** in
--name ***with the your docker image name. - You need to replace *** in
-v ***:/root/VolleyBots:rwwith the directory where VolleyBots is locally located in. - You need to replace *** in
-e "WANDB_API_KEY=***"with your own WANDB_API_KEY. If you do not need to use Wandb, you can omit this line. - In the container, VolleyBots is located at
/root/VolleyBotsand Isaac-sim is located at/isaac-sim.
Install VolleyBots in the container:
conda activate sim2
cd /root/VolleyBots
cp -r conda_setup/etc $CONDA_PREFIX
conda activate sim2 # re-activate the environment
pip install -e . # install VolleyBotsVerify you can successfully run VolleyBots in the container (use deploy branch):
cd /root/VolleyBots/scripts
python train.py headless=true wandb.mode=disabled total_frames=50000 task=SingleJuggleVolleyball task.env.num_envs=16We use hydra to manage the configurations. You can find the configuration files in cfg.
File structure of the configuration files:
cfg
├── train.yaml # training configurations
├── task
│ ├── <TASKNAME>.yaml # configurations for each task
├── algo
│ ├── <ALGONAME>.yaml # configurations for each algorithmWhen you run the training script, you can modify the configurations in these yaml files, or you can pass the configurations as command line arguments. For example:
python train.py headless=true \
total_frames=1_000_000_000 \
task=Volleyball1v1 \
task.drone_model=Iris \
task.env.num_envs=2048 \
task.env.env_spacing=18 \
task.random_turn=True \
task.initial.drone_0_near_side=True \
task.symmetric_obs=True \
eval_interval=300 \
save_interval=300 \
algo=mappo \
algo.share_actor=true \
algo.critic_input="obs" \
seed=0 \For experiment tracking, we use wandb. You need to have a wandb account, and set the wandb configurations in the train.yaml file. For example:
wandb:
group: <EXPERIMENT_GROUP>
run_name: <EXPERIMENT_NAME>
job_type: train
entity: <YOUR_WANDB_ENTITY>
project: <YOUR_WANDB_PROJECT>
mode: online # set to 'disabled' when debugging locally
run_id:
monitor_gym: True
tags:To train a model, you can run the following command:
# at VolleyBots/scripts
python train.py <TRAIN_CONFIGS> <TASK_CONFIGS> <ALGO_CONFIGS>We also provide example training scripts for each task included in our paper. You can find them in the scripts/training_shell directory. For example, to train a model for the Volleyball1v1 task using self-play, you can run:
# at VolleyBots/scripts/training_shell
bash volleyball1v1_sp.shWe include the example evaluation scripts from our paper in the scripts/evalution_shell directory. For example, to evalute the cross-play win rate between psro_uniform and psro_nash on the Volleyball3v3 task, you can run:
# at VolleyBots/scripts/evaluation_shell
bash volleyball3v3_crossplay.shWe offers trained checkpoints at VolleyBots/checkpoints directory for evaluating exploitability and cross-play win rates, and we also include hierarchical policy checkpoints for competing against the self-play baseline in the 3 vs 3 task.
Our experiments were run on a workstation with:
- GPU: NVIDIA GeForce RTX 4090
- RAM: 128 GB
- OS: Ubuntu 20.04 LTS
- Software: CUDA 12.4
Please cite our paper if you find our work useful:
@misc{xu2025volleybotstestbedmultidronevolleyball,
title={VolleyBots: A Testbed for Multi-Drone Volleyball Game Combining Motion Control and Strategic Play},
author={Zelai Xu and Ruize Zhang and Chao Yu and Huining Yuan and Xiangmin Yi and Shilong Ji and Chuqi Wang and Wenhao Tang and Feng Gao and Wenbo Ding and Xinlei Chen and Yu Wang},
year={2025},
eprint={2502.01932},
archivePrefix={arXiv},
primaryClass={cs.RO},
url={https://arxiv.org/abs/2502.01932},
}This repository is heavily based on OmniDrones.
Some of the abstractions and implementation was heavily inspired by Isaac Orbit.