Memory-Efficient Optical Flow via Radius-Distribution Orthogonal Cost Volume
Gangwei Xu, Shujun Chen, Hao Jia, Miaojie Feng, Xin Yang
- NVIDIA RTX 3090
- python 3.8
- torch 1.12.1+cu113
conda create -n meflow python=3.8
conda activate meflowpip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu113
pip install tensorboard
pip install opencv-python
pip install scipy
pip install pillow==9.5.0All pretrained models can be downloaded from google drive.
You can run a trained model on a sequence of images and visualize the results:
bash ./scripts/demo.sh
The datasets used to train and evaluate HCVFlow are as follows:
By default the dataloader datasets.py assumes the datasets are located in folder datasets and are organized as follows:
datasets
├── FlyingChairs
│ └── data
├── FlyingThings3D
│ ├── frames_cleanpass
│ ├── frames_finalpass
│ └── optical_flow
├── HD1K
│ ├── hd1k_challenge
│ ├── hd1k_flow_gt
│ ├── hd1k_flow_uncertainty
│ └── hd1k_input
├── KITTI_2015
│ ├── testing
│ └── training
├── Sintel
│ ├── test
│ └── training
You can evaluate a trained MeFlow model by running:
bash ./scripts/evaluate.sh
All training scripts on FlyingChairs, FlyingThings3D, Sintel and KITTI datasets can be found in scripts/train.sh. You can train a MeFlow model by running:
bash ./scripts/train.sh
Following Flow1D, we support the use of tensorboard to monitor and visualize the training process. To begin, you can start a tensorboard session by
tensorboard --logdir checkpoints --port 6688and then access http://localhost:6688 in your browser.
If you find our work useful in your research, please consider citing our paper:
@article{xu2023memory,
title={Memory-efficient optical flow via radius-distribution orthogonal cost volume},
author={Xu, Gangwei and Chen, Shujun and Jia, Hao and Feng, Miaojie and Yang, Xin},
journal={arXiv preprint arXiv:2312.03790},
year={2023}
}This project is based on Flow1D, we thank the original authors for their excellent work.