The model Loading stage, which should be overridden when surrogate model is customized (e.g., DSM, SETR, etc.) Prioritize the model i

Log a value of some diagnostic Call this once for each diagnostic quantity, each iteration If called many times, last value will be used.

Get the loss function

Copy the master parameter data back into the model parameters.

Copy the gradients from the model parameters into the master parameters from make_master_params().

Modify the model with IAA - Replace ReLU with Softplus - Decrease the weight for certain residual modules

Copied from: https://github.com/openai/baselines/blob/ea25b9e8b234e6ee1bca43083f8f3cf974143998/baselines/common/mpi_util.py#L110 Perform a we

See http://www.codinghorror.com/blog/archives/001018.html

See http://www.codinghorror.com/blog/archives/001018.html

Make uniform grid points, and add noise except for edge points.

Differentiable version of torchvision.functional.normalize

Differentiable version of torchvision.functional.normalize

Create the drift and diffusion functions for the reverse SDE

Optimizes with warmup and gradient clipping (disabled if negative).

Generate samples from the model and yield intermediate samples from each timestep of diffusion. Arguments are the same as p_

Parse filenames of the form path/to/modelNNNNNN.pt, where NNNNNN is the checkpoint's number of steps.

Monte Carlo algorithm for evaluating the prediction of g at x. With probability at least 1 - alpha, the class returned by this method will e

Print timing statistics

Prune the model parameters with the smallest 'p' weights Arguments: model (torch.nn.Module): the surrogate model to be p

Inplace prune the module with mask, and save the original weights in module.weight_orig

Get the distribution q(x_t | x_0). :param x_start: the [N x C x ...] tensor of noiseless inputs. :param t: the number of dif

read image by cv2 or from lmdb return: Numpy float32, HWC, BGR, [0,1]

register hook for densenet models (This code is copied from https://github.com/csdongxian/skip-connections-matter)

register hook for resnet models (This code is copied from https://github.com/csdongxian/skip-connections-matter)

Replace the old layer with the new layer in the model Inputs: model (nn.Module): the model to be modified ol

Rescale the gradient

Reset all stored states to None

r"""ResNet-101 model from `"Deep Residual Learning for Image Recognition" <https://arxiv.org/pdf/1512.03385.pdf>`_ Args: pretrained (b

r"""ResNet-152 model from `"Deep Residual Learning for Image Recognition" <https://arxiv.org/pdf/1512.03385.pdf>`_ Args: pretrained (b

r"""ResNet-18 model from `"Deep Residual Learning for Image Recognition" <https://arxiv.org/pdf/1512.03385.pdf>'_ Args: pretrained (bo

Forward hook for residual modules Inputs: lamb (float): the weight for the residual modules

Restore the parameters stored with the `store` method. Useful to validate the model with EMA parameters without affecting the original op

Restore the original weights in model