Registers hooks on supported layers to track activations and gradients. Warning: Avoid in-place operations (e.g., relu(i

Prepares the model for optimization by registering forward and backward hooks on supported modules. These hooks are crucial for captu

Resets num_bad_epochs counter and cooldown counter.

Do eigen decomposition for computing inverse of the ~ fisher. :param m: The layer :return: no returns.

Do eigen decomposition for computing inverse of the ~ fisher. :param m: The layer :return: no returns.

Aggregates Kronecker factors across multiple GPUs in a distributed setting. This function performs an all-reduce operation to sum th

Apply the attention. This computes the output of the attention, for all the sequences and all the heads in the batch. For a single se

r"""Densenet-161 model from `"Densely Connected Convolutional Networks" <https://arxiv.org/pdf/1608.06993.pdf>`_ Args: memory_efficie

r"""Densenet-169 model from `"Densely Connected Convolutional Networks" <https://arxiv.org/pdf/1608.06993.pdf>`_ Args: memory_efficie

r"""Densenet-201 model from `"Densely Connected Convolutional Networks" <https://arxiv.org/pdf/1608.06993.pdf>`_ Args: memory_efficie

Compute the attention weights. This computes the attention weights for all the sequences and all the heads in the batch. For a single

Get the embeddings for some input sequence. This function computes the embedding vectors based on the input sequence (and the positio

compute the Hessian vector product with a random vector v, at the current gradient point, i.e., compute the gradient of <gradsH,v>.

compute the Hessian vector product with a random vector v, at the current gradient point, i.e., compute the gradient of <gradsH,v>.

Merge the head vectors. This function concatenates the head vectors in a single vector. This function also transposes the `sequence_l

r"""ResNet-152 model from `"Deep Residual Learning for Image Recognition" <https://arxiv.org/pdf/1512.03385.pdf>`_

r"""ResNet-18 model from `"Deep Residual Learning for Image Recognition" <https://arxiv.org/pdf/1512.03385.pdf>`_

r"""ResNet-34 model from `"Deep Residual Learning for Image Recognition" <https://arxiv.org/pdf/1512.03385.pdf>`_ Args: pretrained (b

Seed the pseudorandom number generator, for repeatability. Args: seed (int): random seed

Replaces positive infinity in the tensor with 1. and negative infinity with 0.. Parameters: tensor (torch.Tensor): Input tensor that

Delegates computation to layer-specific methods. Args: s: Output gradients layer: PyTorch layer

Args: img (Tensor): Tensor image of size (C, H, W). Returns: Tensor: Image with n_holes of dimension length x

@monitor: value to monitor for early stopping (e.g. train_loss, test_loss, ...) @mode: specify whether you want to

Directly calls the instance to compute the diagonal and the full matrix of the covariance matrix by delegating to layer-specific met