This repository includes Torch code for training our SPPE+SSTN model presented in our arXiv paper.
To run this code, the following must be installed:
- Torch7
- hdf5 (and the torch-hdf5 package)
- cudnn
- qlua (for displaying results)
- STN module(use this stable repo)
To install STN module, do
git clone https://github.com/Fang-Haoshu/stnbhwd.git
cd stnbhwd && luarocks make stnbhwd-scm-1.rockspecAfter all the installation is done, get the code. We will call the directory that you cloned this repo into $SPPE_ROOT
git clone https://github.com/Fang-Haoshu/multi-human-pose.git
cd multi-human-pose- Download pre-trained SPPE+SSTN model(
79DD
Google drive|Baidu cloud). By default, we assume the models are stored in
$SPPE_ROOT/predict/model/.
-
Download stacked hourglass networks. By default, we assume the model is stored in
$SPPE_ROOT/train/. -
Download MPII images and COCO14 training set. By default, we assume the images are stored in
/data/MPII_COCO14/images/.
You will need to generate bounding box first. Here we have already generated the bounding box in $SPPE_ROOT/predict/annot/mpii-test0.09/. To generate it yourself, please follow the guidlines in the main repo.
cd $SPPE_ROOT/predict
# make a soft link to the images then test
ln -s /data/MPII_COCO14/images/ data/images
# get the predicted results
th main.lua predict-testYou can also use the following command to visualize the single person pose results.
qlua main.lua demoWe finetune our model based on the pre-trained stacked-hourglass model.
cd $SPPE_ROOT/train/src
ln -s /data/MPII_COCO14/images ../data/mpii/images
# First finetune the model using DPG
th main.lua -expID finetune -addDPG
th main.lua -expID finetune -addDPG -continue -LR 0.5e-4 -nEpochs 10
# Then add parallel SPPE and SSTN on the finetuned model
# It should reach a final mAP of 80.*
th main.lua -expID final_model -loadModel '../exp/mpii/finetune/final_model.t7' -LR 0.5e-4 -addParallelSPPE -addSTN -addDPG
th main.lua -expID final_model -continue -Npochs 8 -LR 0.1e-4 -addParallelSPPE -addSTN -addDPGThanks to Wei Liu, Alejandro Newell, Pfister, T., Kaichun Mo, Maxime Oquab for contributing their codes. Thanks to the authors of Caffe and Torch7!