Let Occ Flow: Self-Supervised 3D Occupancy Flow Prediction, CoRL 2024
Yili Liu*, Linzhan Mou*, Xuan Yu, Chenrui Han, Sitong Mao, Rong Xiong, Yue Wang
$\dagger$
* Equal contribution
- [2025/04/16] - Code and checkpoints released.
- [2024/09/04] - Our paper has been accepted by CoRL 2024.
- [2024/07/18] - We released our paper on arXiv.
Follow detailed instructions in Installation.
Follow detailed instructions in Prepare Dataset.
# Stage 1: train occ model
## for kitti odometry dataset (tab.1)
python train.py --py-config config/kitti/kitti_occ_odom.py --work-dir out/train/kitti/occ_odom --dataset kitti --depth-metric
## for nuscenes dataset (tab.3)
python train.py --py-config config/nuscenes/nuscenes_occ_voxelaffm.py --work-dir out/train/nuscenes/occ_static_train --dataset nuscenes --depth-metric
# Stage 2: train occ flow model (modify the parameter 'load_from' in the config file)
python train.py --py-config config/nuscenes/nuscenes_occ_flow_voxelaffm.py --work-dir out/train/nuscenes/occ_flow_train --dataset nuscenes --depth-metric Reproduce our results by downloading the checkpoints run by us from here.
# save occupancy, occupancy flow and render results
# (modify the parameter 'load_from' in the config file)
## for kitti dataset
python eval.py --py-config config/kitti/kitti_occ_odom.py --work-dir out/visualization/kitti/kitti_odom --resolution 0.2 --dataset kitti
## for nuscenes dataset
python eval.py --py-config config/nuscenes/nuscenes_occ_flow_voxelaffm.py --work-dir out/visualization/nuscenes/occ_flow --resolution 0.4 --dataset nuscenes
# prepare ray casting for ray_iou_geo metric
## for kitti dataset
python utils/ray_iou_geo/ray_casting_kitti.py --pred-occ-path out/visualization/nuscenes/kitti/kitti_odom --output-dir /path/to/project/ray_iou_output/kitti_odom
## for nuscenes dataset
python utils/ray_iou_geo/ray_casting_nus.py --pred-occ-path out/visualization/nuscenes/occ_flow/occupancy --output-dir /path/to/project/ray_iou_output/occ_flow
# eval ray_iou_geo metric
## for kitti dataset
python utils/ray_iou_geo/metric_kitti.py --work-dir /path/to/project/ray_iou_output/kitti_odom
## for nuscenes dataset
python utils/ray_iou_geo/metric.py --work-dir /path/to/project/ray_iou_output/occ_flow
# visualize occupancy and occupancy flow
python visualize_occupancy.pyMany thanks to these excellent projects.
If this work is helpful for your research, please consider citing the following paper:
@article{liu2024letoccflow,
title={Let Occ Flow: Self-Supervised 3D Occupancy Flow Prediction},
author={Yili Liu and Linzhan Mou and Xuan Yu and Chenrui Han and Sitong Mao and Rong Xiong and Yue Wang},
journal={arXiv preprint arXiv:2407.07587},
year={2024},
}