Module laplace.utils.subnetmask
Classes
class SubnetMask (model: nn.Module)-
Baseclass for all subnetwork masks in this library (for subnetwork Laplace).
Parameters
model:torch.nn.Module
Subclasses
- ModuleNameSubnetMask
- ParamNameSubnetMask
- laplace.utils.subnetmask.ScoreBasedSubnetMask
Instance variables
var indices : torch.LongTensorvar n_params_subnet : int
Methods
def convert_subnet_mask_to_indices(self, subnet_mask: torch.Tensor) ‑> torch.LongTensor-
Converts a subnetwork mask into subnetwork indices.
Parameters
subnet_mask:torch.Tensor- a binary vector of size (n_params) where 1s locate the subnetwork parameters
within the vectorized model parameters
(i.e.
torch.nn.utils.parameters_to_vector(model.parameters()))
Returns
subnet_mask_indices:torch.LongTensor- a vector of indices of the vectorized model parameters
(i.e.
torch.nn.utils.parameters_to_vector(model.parameters())) that define the subnetwork
def select(self, train_loader: DataLoader | None = None) ‑> torch.LongTensor-
Select the subnetwork mask.
Parameters
train_loader:torch.data.utils.DataLoader, default=None- each iterate is a training batch (X, y);
train_loader.datasetneeds to be set to access N, size of the data set
Returns
subnet_mask_indices:torch.LongTensor- a vector of indices of the vectorized model parameters
(i.e.
torch.nn.utils.parameters_to_vector(model.parameters())) that define the subnetwork
def get_subnet_mask(self, train_loader: DataLoader) ‑> torch.Tensor-
Get the subnetwork mask.
Parameters
train_loader:torch.data.utils.DataLoader- each iterate is a training batch (X, y);
train_loader.datasetneeds to be set to access N, size of the data set
Returns
subnet_mask:torch.Tensor- a binary vector of size (n_params) where 1s locate the subnetwork parameters
within the vectorized model parameters
(i.e.
torch.nn.utils.parameters_to_vector(model.parameters()))
class RandomSubnetMask (model: nn.Module, n_params_subnet: int)-
Subnetwork mask of parameters sampled uniformly at random.
Ancestors
- laplace.utils.subnetmask.ScoreBasedSubnetMask
- SubnetMask
Methods
def compute_param_scores(self, train_loader: DataLoader) ‑> torch.Tensor
Inherited members
class LargestMagnitudeSubnetMask (model: nn.Module, n_params_subnet: int)-
Subnetwork mask identifying the parameters with the largest magnitude.
Ancestors
- laplace.utils.subnetmask.ScoreBasedSubnetMask
- SubnetMask
Methods
def compute_param_scores(self, train_loader: DataLoader) ‑> torch.Tensor
Inherited members
class LargestVarianceDiagLaplaceSubnetMask (model: nn.Module, n_params_subnet: int, diag_laplace_model: DiagLaplace)-
Subnetwork mask identifying the parameters with the largest marginal variances (estimated using a diagonal Laplace approximation over all model parameters).
Parameters
model:torch.nn.Modulen_params_subnet:int- number of parameters in the subnetwork (i.e. number of top-scoring parameters to select)
diag_laplace_model:DiagLaplace- diagonal Laplace model to use for variance estimation
Ancestors
- laplace.utils.subnetmask.ScoreBasedSubnetMask
- SubnetMask
Methods
def compute_param_scores(self, train_loader: DataLoader) ‑> torch.Tensor
Inherited members
class LargestVarianceSWAGSubnetMask (model: nn.Module, n_params_subnet: int, likelihood: Likelihood | str = Likelihood.CLASSIFICATION, swag_n_snapshots: int = 40, swag_snapshot_freq: int = 1, swag_lr: float = 0.01)-
Subnetwork mask identifying the parameters with the largest marginal variances (estimated using diagonal SWAG over all model parameters).
Parameters
model:torch.nn.Modulen_params_subnet:int- number of parameters in the subnetwork (i.e. number of top-scoring parameters to select)
likelihood:str- 'classification' or 'regression'
swag_n_snapshots:int- number of model snapshots to collect for SWAG
swag_snapshot_freq:int- SWAG snapshot collection frequency (in epochs)
swag_lr:float- learning rate for SWAG snapshot collection
Ancestors
- laplace.utils.subnetmask.ScoreBasedSubnetMask
- SubnetMask
Methods
def compute_param_scores(self, train_loader: DataLoader) ‑> torch.Tensor
Inherited members
class ParamNameSubnetMask (model: nn.Module, parameter_names: list[str])-
Subnetwork mask corresponding to the specified parameters of the neural network.
Parameters
model:torch.nn.Moduleparameter_names:List[str]- list of names of the parameters (as in
model.named_parameters()) that define the subnetwork
Ancestors
Methods
def get_subnet_mask(self, train_loader: DataLoader) ‑> torch.Tensor-
Get the subnetwork mask identifying the specified parameters.
Inherited members
class ModuleNameSubnetMask (model: nn.Module, module_names: list[str])-
Subnetwork mask corresponding to the specified modules of the neural network.
Parameters
model:torch.nn.Moduleparameter_names:List[str]- list of names of the modules (as in
model.named_modules()) that define the subnetwork; the modules cannot have children, i.e. need to be leaf modules
Ancestors
Subclasses
Methods
def get_subnet_mask(self, train_loader: DataLoader) ‑> torch.Tensor-
Get the subnetwork mask identifying the specified modules.
Inherited members
class LastLayerSubnetMask (model: nn.Module, last_layer_name: str | None = None)-
Subnetwork mask corresponding to the last layer of the neural network.
Parameters
model:torch.nn.Modulelast_layer_name:str, default=None- name of the model's last layer, if None it will be determined automatically
Ancestors
Methods
def get_subnet_mask(self, train_loader: DataLoader) ‑> torch.Tensor-
Get the subnetwork mask identifying the last layer.
Inherited members