This repository contains the PyTorch implementation for PoinTr, which was adapted for a study on point cloud completion of terrestrial laserscanning data of individual trees.
The folder tree_workflow contains jupyter notebooks for:
- Creating training samples from individual tree point clouds (make_samples_for_treePoinTr.ipynb)
- Applying the completion approach to individual trees (whole_tree_completion_workflow.ipynb)
The data used in the study and the models trained in the study can be downloaded from Zenodo: doi.org/10.5281/zenodo.13303159
The folder treehull_workflow contains scripts and instructions for the completion of the outer hull shape of individual trees.
For installation and usage, please follow the original instructions below, provided by the authors of PoinTr.
Created by Xumin Yu*, Yongming Rao*, Ziyi Wang, Zuyan Liu, Jiwen Lu, Jie Zhou
[arXiv] [Video] [Dataset] [Models] [supp]
This repository contains PyTorch implementation for PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers (ICCV 2021 Oral Presentation).
PoinTr is a transformer-based model for point cloud completion. By representing the point cloud as a set of unordered groups of points with position embeddings, we convert the point cloud to a sequence of point proxies and employ a transformer encoder-decoder architecture for generation. We also propose two more challenging benchmarks ShapeNet-55/34 with more diverse incomplete point clouds that can better reflect the real-world scenarios to promote future research.
- 2023-9-2 AdaPoinTr accepted by T-PAMI, Projected-ShapeNet dataset see here
- 2023-1-11 Release AdaPoinTr (PoinTr + Adaptive Denoising Queries), achieving SOTA performance on various benchmarks. Arxiv.
- 2022-06-01 Implement SnowFlakeNet.
- 2021-10-07 Our solution based on PoinTr wins the Championship on MVP Completion Challenge (ICCV Workshop 2021). The code will come soon.
- 2021-09-09 Fix a bug in
datasets/PCNDataset.py(#27) 8000 , and update the performance of PoinTr on PCN benchmark (CD from 8.38 to 7.26).
We provide pretrained PoinTr models:
| dataset | url | performance |
|---|---|---|
| ShapeNet-55 | [Tsinghua Cloud] / [Google Drive] / [BaiDuYun] (code:erdh) | CD = 1.09e-3 |
| ShapeNet-34 | [Tsinghua Cloud] / [Google Drive] / [BaiDuYun] (code:atbb ) | CD = 2.05e-3 |
| PCN | [Tsinghua Cloud] / [Google Drive] / [BaiDuYun] (code:9g79) | CD = 8.38e-3 |
| PCN_new | [Tsinghua Cloud] / [Google Drive] / [BaiDuYun] (code:aru3 ) | CD = 7.26e-3 |
| KITTI | [Tsinghua Cloud] / [Google Drive] / [BaiDuYun] (code:99om) | MMD = 5.04e-4 |
We provide pretrained AdaPoinTr models (coming soon):
| dataset | url | performance |
|---|---|---|
| ShapeNet-55 | Tsinghua Cloud / Google Drive / BaiDuYun | CD = 0.81e-3 |
| ShapeNet-34 | Tsinghua Cloud / Google Drive / BaiDuYun | CD = 1.23e-3 |
| Projected_ShapeNet-55 | Tsinghua Cloud / Google Drive / [BaiDuYun](code:dycc) | CD = 9.58e-3 |
| Projected_ShapeNet-34 | Tsinghua Cloud / Google Drive / [BaiDuYun](code:dycc) | CD = 9.12e-3 |
| PCN | [Tsinghua Cloud] / [Google Drive] / [BaiDuYun](code:rc7p) | CD = 6.53e-3 |
- PyTorch >= 1.7.0
- python >= 3.7
- CUDA >= 9.0
- GCC >= 4.9
- torchvision
- timm
- open3d
- tensorboardX
pip install -r requirements.txt
NOTE: PyTorch >= 1.7 and GCC >= 4.9 are required.
# Chamfer Distance
bash install.sh
The solution for a common bug in chamfer distance installation can be found in Issue #6
# PointNet++
pip install "git+https://github.com/erikwijmans/Pointnet2_PyTorch.git#egg=pointnet2_ops&subdirectory=pointnet2_ops_lib"
# GPU kNN
pip install --upgrade https://github.com/unlimblue/KNN_CUDA/releases/download/0.2/KNN_CUDA-0.2-py3-none-any.whl
Note: If you still get ModuleNotFoundError: No module named 'gridding' or something similar then run these steps
1. cd into extensions/Module (eg extensions/gridding)
2. run `python setup.py install`
That will fix the ModuleNotFoundError.
The details of our new ShapeNet-55/34 datasets and other existing datasets can be found in DATASET.md.
To inference sample(s) with pretrained model
python tools/inference.py \
${POINTR_CONFIG_FILE} ${POINTR_CHECKPOINT_FILE} \
[--pc_root <path> or --pc <file>] \
[--save_vis_img] \
[--out_pc_root <dir>] \
For example, inference all samples under demo/ and save the results under inference_result/
python tools/inference.py \
cfgs/PCN_models/AdaPoinTr.yaml ckpts/AdaPoinTr_PCN.pth \
--pc_root demo/ \
--save_vis_img \
--out_pc_root inference_result/ \
To evaluate a pre-trained PoinTr model on the Three Dataset with single GPU, run:
bash ./scripts/test.sh <GPU_IDS> \
--ckpts <path> \
--config <config> \
--exp_name <name> \
[--mode <easy/median/hard>]
Test the PoinTr (AdaPoinTr) pretrained model on the PCN benchmark or Projected_ShapeNet:
bash ./scripts/test.sh 0 \
--ckpts ./pretrained/PoinTr_PCN.pth \
--config ./cfgs/PCN_models/PoinTr.yaml \
--exp_name example
bash ./scripts/test.sh 0 \
--ckpts ./pretrained/PoinTr_ps55.pth \
--config ./cfgs/Projected_ShapeNet55_models/AdaPoinTr.yaml \
--exp_name example
Test the PoinTr pretrained model on ShapeNet55 benchmark (easy mode):
bash ./scripts/test.sh 0 \
--ckpts ./pretrained/PoinTr_ShapeNet55.pth \
--config ./cfgs/ShapeNet55_models/PoinTr.yaml \
--mode easy \
--exp_name example
Test the PoinTr pretrained model on the KITTI benchmark:
bash ./scripts/test.sh 0 \
--ckpts ./pretrained/PoinTr_KITTI.pth \
--config ./cfgs/KITTI_models/PoinTr.yaml \
--exp_name example
CUDA_VISIBLE_DEVICES=0 python KITTI_metric.py \
--vis <visualization_path>
To train a point cloud completion model from scratch, run:
# Use DistributedDataParallel (DDP)
bash ./scripts/dist_train.sh <NUM_GPU> <port> \
--config <config> \
--exp_name <name> \
[--resume] \
[--start_ckpts <path>] \
[--val_freq <int>]
# or just use DataParallel (DP)
bash ./scripts/train.sh <GPUIDS> \
--config <config> \
--exp_name <name> \
[--resume] \
[--start_ckpts <path>] \
[--val_freq <int>]
Train a PoinTr model on PCN benchmark with 2 gpus:
CUDA_VISIBLE_DEVICES=0,1 bash ./scripts/dist_train.sh 2 13232 \
--config ./cfgs/PCN_models/PoinTr.yaml \
--exp_name example
Resume a checkpoint:
CUDA_VISIBLE_DEVICES=0,1 bash ./scripts/dist_train.sh 2 13232 \
--config ./cfgs/PCN_models/PoinTr.yaml \
--exp_name example --resume
Finetune a PoinTr on PCNCars
CUDA_VISIBLE_DEVICES=0,1 bash ./scripts/dist_train.sh 2 13232 \
--config ./cfgs/KITTI_models/PoinTr.yaml \
--exp_name example \
--start_ckpts ./weight.pth
Train a PoinTr model with a single GPU:
bash ./scripts/train.sh 0 \
--config ./cfgs/KITTI_models/PoinTr.yaml \
--exp_name example
We also provide the Pytorch implementation of several baseline models including GRNet, PCN, TopNet and FoldingNet. For example, to train a GRNet model on ShapeNet-55, run:
CUDA_VISIBLE_DEVICES=0,1 bash ./scripts/dist_train.sh 2 13232 \
--config ./cfgs/ShapeNet55_models/GRNet.yaml \
--exp_name example
MIT License
Our code is inspired by GRNet and mmdetection3d.
If you find our work useful in your research, please consider citing:
@inproceedings{yu2021pointr,
title={PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers},
author={Yu, Xumin and Rao, Yongming and Wang, Ziyi and Liu, Zuyan and Lu, Jiwen and Zhou, Jie},
booktitle={ICCV},
year={2021}
}