🤗 If you find TrajectoryCrafter useful, please help ⭐ this repo, which is important to Open-Source projects. Thanks!
- [2025-03-10]: 🔥🔥 Update the arXiv preprint.
- [2025-02-23]: Launch the project page.
TrajectoryCrafter can generate high-fidelity novel views from casually captured monocular video, while also supporting highly precise pose control. Below shows some examples:
| Input Video New Camera Trajectory |
|
|
|
|
|
|
|
|
We recommend deploying it on a GPU with VRAM ≥ 28GB.
git clone --recursive https://github.com/TrajectoryCrafter/TrajectoryCrafter.git
cd TrajectoryCrafterconda create -n trajcrafter python=3.10
conda activate trajcrafter
pip install -r requirements.txtIdeally, you can load pretrained models directly from HuggingFace. If you encounter issues connecting to HuggingFace, you can download the pretrained models locally instead. To do so, you can:
- Download the pretrained models using HuggingFace or using git-lfs
# HuggingFace (recommend)
sh download/download_hf.sh
# git-lfs (much slower but more stable)
sh download/download_lfs.sh - Change default path of the pretrained models to your local path in inference.py.
Run inference.py using the following script. Please refer to the configuration document to set up inference parameters and camera trajectory.
sh run.sh python gradio_app.pyOur model excels at handling videos with well-defined objects and clear motion, as demonstrated in the demo videos. However, since it is built upon a pretrained video diffusion model, it may struggle with complex cases that go beyond the generation capabilities of the base model.
Including but not limited to: CogVideo-Fun, ViewCrafter, DepthCrafter, GCD, NVS-Solver, DimensionX, ReCapture, TrajAttention, GS-DiT, DaS, RecamMaster, GEN3C, CAT4D...