Create a 1D, 2D, or 3D convolution module.

Make a standard normalization layer. :param channels: number of input channels. :return: an nn.Module for normalization.

Write the sequence of args, with no separators, to the console and output files (if you've configured an output file).

Create a linear module.

Zero out the parameters of a module and return it.

Convert the torso of the model to float16.

Returns a vector containing the singular values. The shape of the vector should be the same as the smaller dimension (like U)

Multiplies the input vector by H

Multiplies the input vector by the pseudo inverse of H

Multiplies the input vector by U transposed

Multiplies the input vector by V

Adds trailing zeros to turn a vector from the small dimension (U) to the big dimension (V)

Evaluate a function without caching intermediate activations, allowing for reduced memory at the expense of extra compute in the backward pas

Defaults for classifier models.

If comm is provided, average all numerical stats across that comm

A counter for the `thop` package to count the operations in an attention operation. Meant to be used like: macs, params = thop.pr

Defaults for image and classifier training.

Copy model parameters into a (differently-shaped) list of full-precision parameters.

Python2 support for os.makedirs(.., exist_ok=True)

Create sinusoidal timestep embeddings. :param timesteps: a 1-D Tensor of N indices, one per batch element. These may b

Multiplies the input vector by U

Multiplies the input vector by V transposed

Create a 1D, 2D, or 3D average pooling module.

Download a Google Drive file from and place it in root. Args: file_id (str): id of file to be downloaded root (str): Directory t

This matches the implementation in Denoising Diffusion Probabilistic Models: From Fairseq. Build sinusoidal embeddings. This matches

Log a value of some diagnostic Call this once for each diagnostic quantity, each iteration If called many times, last value will be used.

Copy the master parameter data back into the model parameters.

Defaults for image training.

Copy the gradients from the model parameters into the master parameters from make_master_params().

Copied from: https://github.com/openai/baselines/blob/ea25b9e8b234e6ee1bca43083f8f3cf974143998/baselines/common/mpi_util.py#L110 Perform a we