This repository contains the implementation of OctGPT.
OctGPT: Octree-based Multiscale Autoregressive Models for 3D Shape Generation
Si-Tong Wei, Rui-Huan Wang, Chuan-Zhi Zhou, Baoquan Chen, Peng-Shuai Wang
Accepted by SIGGRAPH 2025
The code has been tested on Ubuntu 20.04 and CUDA 12.4.
-
Install Conda and create a
Condaenvironment.conda create --name octgpt python=3.10 conda activate octgpt
-
Install PyTorch-2.5 with conda according to the official documentation.
pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 \ --index-url https://download.pytorch.org/whl/cu124 -
Clone this repository and install the requirements.
git clone https://github.com/octree-nn/octgpt.git cd octgpt pip install -r requirements.txt
We provide the pretrained models for unconditional and category-condition generation. Please download the pretrained models from Hugging Face and put them in saved_ckpt.
-
Unconditional generation in category
airplane,car,chair,rifle,table.export category=airplane && \ python main_octgpt.py \ --config configs/ShapeNet/shapenet_uncond.yaml \ SOLVER.run generate \ SOLVER.ckpt saved_ckpt/octgpt_${category}.pth \ SOLVER.logdir logs/${category} \ MODEL.vqvae_ckpt saved_ckpt/vqvae_large_im5_uncond_bsq32.pth \ MODEL.OctGPT.patch_size 2048 \ MODEL.OctGPT.dilation 2
-
Category-conditioned generation
export category=airplane && \ python main_octgpt.py \ --config configs/ShapeNet/shapenet_uncond.yaml \ SOLVER.run generate \ SOLVER.ckpt saved_ckpt/octgpt_im5.pth \ SOLVER.logdir logs/im5 \ MODEL.vqvae_ckpt saved_ckpt/vqvae_large_im5_cond_bsq32.pth \ MODEL.OctGPT.condition_type category \ MODEL.OctGPT.num_classes 5 \ MODEL.OctGPT.patch_size 1024 \ MODEL.OctGPT.dilation 16 \ DATA.test.category ${category}
-
Download
ShapeNetCore.v1.zip(31G) from ShapeNet and place it indata/ShapeNet/ShapeNetCore.v1.zip. DownloadShapeNetfrom HuggingFace and place it indata/ShapeNet/filelist. -
Convert the meshes in
ShapeNetCore.v1to signed distance fields (SDFs). We use the same data preparation as DualOctreeGNN and OctFusion. We utilize mesh2sdf.python tools/sample_sdf.py --mode cpu --dataset ShapeNet
-
Unconditional Generation
export category=airplane && \ python main_octgpt.py \ --config configs/ShapeNet/shapenet_uncond.yaml \ SOLVER.run train \ SOLVER.gpu 0,1,2,3 \ SOLVER.logdir logs/octgpt_${category} \ DATA.train.filelist data/ShapeNet/filelist/train_${category}.txt \ DATA.test.filelist data/ShapeNet/filelist/test_${category}.txt \ MODEL.vqvae_ckpt saved_ckpt/vqvae_large_im5_uncond_bsq32.pth
-
Category-condition Generation
python main_octgpt.py \ --config configs/ShapeNet/shapenet_uncond.yaml \ SOLVER.run train \ SOLVER.gpu 0,1,2,3 \ SOLVER.logdir logs/octgpt_im_5 \ DATA.train.filelist data/ShapeNet/filelist/train_im_5.txt \ DATA.test.filelist data/ShapeNet/filelist/test_im_5.txt \ MODEL.vqvae_ckpt saved_ckpt/vqvae_large_im5_cond_bsq32.pth \ MODEL.OctGPT.condition_type category \ MODEL.OctGPT.num_classes 5 -
VQVAE
python main_vae.py \ --config configs/ShapeNet/shapenet_vae.yaml \ SOLVER.run train \ SOLVER.gpu 0,1,2,3 \ SOLVER.logdir logs/vqvae_im_5 \ DATA.train.filelist data/ShapeNet/filelist/train_im_5.txt \ DATA.test.filelist data/ShapeNet/filelist/test_im_5.txt
Download the pretrained models from Hugging Face and put them in saved_ckpt.
Generate based on a specific text prompt
python main_octgpt.py \
--config configs/Objaverse/objaverse_octar_text.yaml \
SOLVER.run generate \
SOLVER.logdir logs/obja_text \
SOLVER.ckpt saved_ckpt/octgpt_objv_text.pth \
MODEL.vqvae_ckpt saved_ckpt/vqvae_large_objv_bsq32.pth \
DATA.test.text_prompt "A 3D model of a Pokémon character."We adopt the data filtering and preprocessing pipeline from from LGM. Our model is trained on a subset of Objaverse containing 4.5w 3D meshes. Text annotations are provided by Cap3D. Download Objaverse from HuggingFace and place it in data/Objaverse/filelist.
To replicate our experimental setup, please follow these steps:
- Place the raw dataset in
data/Objaverse/raw. - Conduct mesh repairing and save the processed meshes to
data/Objaverse/datasets_512.
python tools/sample_sdf.py --mode cpu --dataset Objaverse --depth 9-
Text-condition Generation
python main_octgpt.py \ --config configs/Objaverse/objaverse_octar_text.yaml \ SOLVER.run train \ SOLVER.gpu 0,1,2,3 \ SOLVER.logdir logs/obja_text \ MODEL.vqvae_ckpt saved_ckpt/vqvae_large_objv_bsq32.pth -
VQVAE
python main_vae.py \ --config configs/Objaverse/objaverse_vae.yaml \ SOLVER.run train \ SOLVER.gpu 0,1,2,3 \ SOLVER.logdir logs/vqvae_im_5 \
@inproceedings {wei2025octgpt,
title = {OctGPT: Octree-based Multiscale Autoregressive Models
for 3D Shape Generation},
author = {Wei, Si-Tong and Wang, Rui-Huan and Zhou, Chuan-Zhi and
Chen, Baoquan and Wang Peng-Shuai},
booktitle = {SIGGRAPH},
year = {2025},
}