BackPACK is built on top of PyTorch. It efficiently computes quantities other than the gradient.
- Website: https://backpack.pt
- Documentation: https://docs.backpack.pt/en/master/
- Bug reports & feature requests: https://github.com/f-dangel/backpack/issues
Provided quantities include:
- Individual gradients from a mini-batch
- Estimates of the gradient variance or second moment
- Approximate second-order information (diagonal and Kronecker approximations)
Motivation: Computation of most quantities is not necessarily expensive (often just a small modification of the existing backward pass where backpropagated information can be reused). But it is difficult to do in the current software environment.
pip install backpack-for-pytorch