8000 KeyError: 'normal' when adaptation using 5s data · Issue #13 · Fictionarry/InsTaG · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

KeyError: 'normal' when adaptation using 5s data #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
whitenoise-tzx opened this issue Mar 21, 2025 · 1 comment
Open

KeyError: 'normal' when adaptation using 5s data #13

whitenoise-tzx opened this issue Mar 21, 2025 · 1 comment

Comments

@whitenoise-tzx
Copy link

Hi, I was trying to do adaptation using 5s training data, but the bug was like:

Training progress:  50%|##########################################                                          | 5000/10000 [02:07<02:14, 37.20it/s, Loss=0.00766, Mouth=10.6-25.0]Traceback (most recent call last):
  File "InsTaG-main/train_face.py", line 450, in <module>
    training(lp.extract(args), op.extract(args), pp.extract(args), args.test_iterations, args.save_iterations, args.checkpoint_iterations, args.start_checkpoint, args.debug_from, args.long, args.pretrain_path)
  File "InsTaG-main/train_face.py", line 199, in training
    loss += 0.01 * (1 - viewpoint_cam.talking_dict["normal"].cuda() * render_pkg["normal"]).sum(0)[head_mask^mouth_mask].mean()
KeyError: 'normal'

I performed 'data_utils/split.py' and set train_val_split = 25 * 5. In the adaptation script, I set n_views=-1

For all the test case that I tried, I noticed that this error only happens when training progress is 50%.

If I increase the n_views to 250, the error won't happen.
@Fictionarry
Copy link
Owner

Hi, n_views=-1 will skip the loading of normal priors, as shown below.

InsTaG/scene/dataset_readers.py

Lines 284 to 293 in a919a3b

if "train" in transformsfile and N_views > 0:
normal_path_candidates = glob.glob(os.path.join(path, 'sapiens/normal/sapiens_*'))
normal_path_candidates.sort(reverse = True)
normal_path = os.path.join(normal_path_candidates[0], str(frame['img_id']) + '.npy')
talking_dict['normal_path'] = normal_path
depth_path_candidates = glob.glob(os.path.join(path, 'sapiens/depth/sapiens_*'))
depth_path_candidates.sort(reverse = True)
depth_path = os.path.join(depth_path_candidates[0], str(frame['img_id']) + '.npy')
talking_dict['depth_path'] = depth_path

So if you want to skip the geometry prior regularization, you may also specify --long during adaptation when using a long training video (seems not suitable for your situation), or modify the logic of the code. Otherwise, an N_views larger than 0 is required to be given.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Fictionarry @whitenoise-tzx
0