Pre-training a Density-Aware Pose Transformer for Robust LiDAR-based 3D Human Pose Estimation (AAAI 2025)
We propose a simple yet powerful framework, which provides insights both on modeling and augmentation of point clouds. Specifically, we first propose a concise and effective density-aware pose transformer (DAPT) to get stable keypoint representations. By using a set of joint anchors and a carefully designed exchange module, valid information is extracted from point 8000 clouds with different densities. Then 1D heatmaps are utilized to represent the precise locations of the keypoints. Secondly, a comprehensive LiDAR human synthesis and augmentation method is proposed to pre-train the model, enabling it to acquire a better human body prior. We increase the diversity of point clouds by randomly sampling human positions and orientations and by simulating occlusions through the addition of laser-level masks.
git clone --recursive https://github.com/AnxQ/daptPlease make sure your connection to anaconda and pypi is stable.
conda env create -f environment.yml
conda activate daptaccelerate configFor details, please refer to Accelerate documentation. Note that differences in computing facilities can lead to some differences in training results.
pip install wandb
wandb loginPlease refer to DATA.md for data download and preparation.
Pre-trained checkpoints can be downloaded from here.
accelerate launch main.py configs/dapt-waymo-50b64-finetune.py --test \
--ckpt checkpoints/dapt-waymo-50b64-finetune.safetensorsChange the config name for evaluation on different dataset .
# Pre-Training
accelerate launch main.py configs/dapt-synpretrain-50b64.py
# Finetuning
accelerate launch main.py configs/dapt-waymo-50b64-finetune.py \
--options model.pretrained=work_dir/dapt-synpretrain-50b64/checkpoints/checkpoint_4/model.safetensorsChange the config name for training on different dataset.
We thank the following open source projects for helping us implement our idea.
This will be modified after publication.
@article{an2025dapt,
title={Pre-training a Density-Aware Pose Transformer for Robust LiDAR-based 3D Human Pose Estimation},
volume={39},
doi={10.1609/aaai.v39i2.32169},
number={2},
journal={Proceedings of the AAAI Conference on Artificial Intelligence},
author={An, Xiaoqi and Zhao, Lin and Gong, Chen and Li, Jun and Yang, Jian},
year={2025},
pages={1755-1763}
}