Open
Description
🐛 Bug
Using a GammaPrior in conjunction with the GaussianLikelihood can lead to sampling errors with Pyro. This is because the default GaussianLikelihood noise_constraint is [1e-4, \infty), while the GammaLikelihood has support over (0, \infty). When the GammaLikelihood samples a < 1e-4 value, the resulting noise parameter is NaN.
In general, we should set the default noise constraint to match the support of the prior, or we should remove the default noise constraint when using a prior.
@Balandat what would be the least breaking approach with BoTorch?
(See #2643)
To reproduce
** Code snippet to reproduce **
# need pyro installed
import gpytorch
import torch
noise_prior = gpytorch.priors.GammaPrior(1.0, 0.5)
likelihood = gpytorch.likelihoods.GaussianLikelihood(
noise_prior=noise_prior,
# noise_constraint=gpytorch.constraints.Positive(), # <- error goes away if we uncomment this line
batch_shape=torch.Size([100,])
)
for i in range(1000):
sampled_likelihood = likelihood.pyro_sample_from_prior()
noise = sampled_likelihood.noise
if not torch.equal(noise, noise):
raise RuntimeError("Sampled noise has NaNs")** Stack trace/error message **
RuntimeError: Sampled noise has NaNs
System information
- GPyTorch Version 1.4
- PyTorch Version 2
- MacOS X