Open
Description
[Bug] gt_instances keypoints are all zeros when fine-tuning on COCO
Description
I encountered an issue where the keypoints in gt_instances are always an array of zeros during training (see the slurm-6618226.txt at the bottom). I have checked browse_dataset.py and PackPoseInputs, and both seem to be working correctly. However, I can't figure out where the problem might be.
Please suggest any potential errors or solutions. Thank you!
Steps to Reproduce
Attempt to fine-tune the model on the COCO dataset using the following command:
python ./mmpose/tools/train.py \
./mmpose/configs/body_2d_keypoint/topdown_heatmap/ap2d/td-hm_hrnet-w48_8xb32-210e_coco-256x192_testing.pyExpected Behavior
The gt_instances.keypoints should contain the correct annotated keypoint coordinates from the dataset instead of an array of zeros.
Modifications
The only modification I made was updating the settings in the configuration file and print out the datasample inside the CocoMetric.
Environment
- Python 3.8.20
- mmpose 1.3.2
- mmengine 0.10.6
- mmcv 2.2.0
- mmdet 3.3.0
The settings
_base_ = ['../../../_base_/default_runtime.py']
# runtime
train_cfg = dict(max_epochs=1, val_interval=1) #test
# optimizer
optim_wrapper = dict(optimizer=dict(
type='Adam',
lr=5e-4,
))
# use the pre-trained model for the whole HRNet
load_from = './mmpose/configs/body_2d_keypoint/topdown_heatmap/ap2d/td-hm_hrnet-w48_8xb32-210e_coco-384x288-c161b7de_20220915.pth'
# fintune learning policy
param_scheduler = [
dict(
type='LinearLR',
begin=0,
end=10,
start_factor=1e-4,
by_epoch=True
), # Warm-up phase
dict(
type='MultiStepLR',
begin=10,
end=20,
milestones=[15],
gamma=0.1,
by_epoch=True
) # Learning rate decay
]
# automatically scaling LR based on the actual training batch size
auto_scale_lr = dict(base_batch_size=512)
# hooks
# default_hooks = dict(checkpoint=dict(save_best='tPCK', rule='greater'))
default_hooks = dict(checkpoint=dict(type='CheckpointHook', interval=1))
# codec settings
codec = dict(
type='MSRAHeatmap', input_size=(288, 384), heatmap_size=(72, 96), sigma=3)
# model settings
model = dict(
type='TopdownPoseEstimator',
data_preprocessor=dict(
type='PoseDataPreprocessor',
mean=[123.675, 116.28, 103.53],
std=[58.395, 57.12, 57.375],
bgr_to_rgb=True),
backbone=dict(
type='HRNet',
in_channels=3,
extra=dict(
stage1=dict(
num_modules=1,
num_branches=1,
block='BOTTLENECK',
num_blocks=(4, ),
num_channels=(64, )),
stage2=dict(
num_modules=1,
num_branches=2,
block='BASIC',
num_blocks=(4, 4),
num_channels=(48, 96)),
stage3=dict(
num_modules=4,
num_branches=3,
block='BASIC',
num_blocks=(4, 4, 4),
num_channels=(48, 96, 192)),
stage4=dict(
num_modules=3,
num_branches=4,
block='BASIC',
num_blocks=(4, 4, 4, 4),
num_channels=(48, 96, 192, 384))),
init_cfg=dict(
type='Pretrained',
checkpoint='https://download.openmmlab.com/mmpose/'
'pretrain_models/hrnet_w48-8ef0771d.pth'),
),
head=dict(
type='HeatmapHead',
in_channels=48,
out_channels=17,
deconv_out_channels=None,
loss=dict(type='KeypointMSELoss', use_target_weight=True),
decoder=codec),
test_cfg=dict(
flip_test=True,
flip_mode='heatmap',
shift_heatmap=True,
))
# base dataset settings
dataset_type = 'CocoDataset'
data_mode = 'topdown'
data_root = '/fsws1/share/database/coco/'
# pipelines
train_pipeline = [
dict(type='LoadImage'),
dict(type='GetBBoxCenterScale'),
dict(type='RandomFlip', direction='horizontal'),
dict(type='RandomHalfBody'),
dict(type='RandomBBoxTransform'),
dict(type='TopdownAffine', input_size=codec['input_size']),
dict(type='GenerateTarget', encoder=codec),
dict(type='PackPoseInputs')
]
val_pipeline = [
dict(type='LoadImage'),
dict(type='GetBBoxCenterScale'),
dict(type='TopdownAffine', input_size=codec['input_size']),
dict(type='PackPoseInputs')
]
train_dataloader = dict(
batch_size=64,
num_workers=4,
persistent_workers=True,
sampler=dict(type='DefaultSampler', shuffle=True),
dataset=dict(
type=dataset_type,
data_root=data_root,
data_mode=data_mode,
ann_file='annotations/person_keypoints_train2017.json',
data_prefix=dict(img='images/train2017/'),
pipeline=train_pipeline,
metainfo = dict(from_file='configs/_base_/datasets/coco.py')
))
val_dataloader = dict(
batch_size=64,
num_workers=4,
persistent_workers=True,
drop_last=False,
sampler=dict(type='DefaultSampler', shuffle=False, round_up=False),
dataset=dict(
type=dataset_type,
data_root=data_root,
data_mode=data_mode,
ann_file='annotations/person_keypoints_val2017.json',
bbox_file='./mmpose_train/'
'COCO_val2017_detections_AP_H_70_person.json',
data_prefix=dict(img='images/val2017/'),
test_mode=True,
pipeline=val_pipeline,
metainfo = dict(from_file='configs/_base_/datasets/coco.py')
))
test_dataloader = val_dataloader
# evaluators
val_evaluator = dict(
type='CocoMetric',
ann_file=data_root + 'annotations/person_keypoints_val2017.json')
test_evaluator = val_evaluator
Metadata
Metadata
Assignees
Labels
No labels