Large-scale Dataset and Benchmark for Egocentric Robot Perception and Navigation in Crowded and Unstructured Environments
Haisheng Su1,2, Feixiang Song2, Cong Ma2, Wei Wu2,3, Junchi Yan1 📧
1 School of Computer Science and School of AI, SJTU
2 SenseAuto Research, 3 Tsinghua University
📧 Corresponding author, yanjunchi@sjtu.edu.cn
RoboSense is a large-scale multimodal dataset constructed to facilitate egocentric robot perception capabilities especially in crowded and unstructured environments.
- News
- Key Features
- Sensor Setup and Coordinate System
- Dataset Example
- Getting started
- Contact
- Citation
[2025/06/05]: 🤖 RoboSense dataset released, including training/validation splits.[2025/03/09]: Our paper has been accepted to CVPR 2025, [Poster]![2024/08/25]: We released our paper on Arxiv. Code and dataset are coming soon. Please stay tuned! ☕️
- 133k+ synchronized frames of 4C+4F+4L sensor data.
- 1.4 million+ 3D bounding boxes and IDs annotated in the full 360°view.
- 7.6K temporal sequences across 6 kinds of target domains (i.e., scenic spots, parks, squares, campuses, streets and sidewalks).
- 216K+ trajectories of objects.
- 270x and 18x as many annotations of near-field obstacles as KITTI and nuScenes.
- 6 benchmarks for both perception and prediction tasks.
Download our source code:
git clone https://github.com/suhaisheng/RoboSense.git
cd RoboSense- Download data from our HuggingFace page.
huggingface-cli download --resume-download --repo-type dataset suhaisheng0527/RoboSense --local-dir ./- Combine all splitted files for image and LiDAR&OCC respectively.
cd dataset
cat image_trainval_part_* > image_trainval.tar.gz
cat lidar_occ_trainval_part_* > lidar_occ_trainval.tar.gz
rm image_trainval_part_*
rm lidar_occ_trainval_part_*
tar -xzf image_trainval.tar.gz
tar -xzf lidar_occ_trainval.tar.gz
cd ..Evaluating perception models with our proposed metrics (CCDP: Closest- Collision Distance Proportion matching function).
Coming soon... ☕️
All assets and code within this repo are under the CC BY-NC-SA 4.0 unless specified otherwise.
If you have any questions, please contact Haisheng Su via email (suhaisheng@sjtu.edu.cn).
If you find RoboSense is useful in your research or applications, please consider giving us a star 🌟 and citing it by the following BibTeX entry.
@inproceedings{su2025robosense,
title={RoboSense: Large-scale Dataset and Benchmark for Egocentric Robot Perception and Navigation in Crowded and Unstructured Environments},
author={Su, Haisheng and Song, Feixiang and Ma, Cong and Wu, Wei and Yan, Junchi},
booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
year={2025}
}