r"""Fills the input Tensor with values drawn from a truncated normal distribution. The values are effectively drawn from the normal distributi

preprocess a 1-length pytorch Variable or Tensor to scalar

Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks). This is the same as the DropConnect impl I created for E

create a dataset description file, which consists of images, labels

create a dataset description file, which consists of images, labels

Note: torch.save() reserves device type and id of tensors to save. So when loading ckpt, you have to inform torch.load() to load

Transfer an optimizer.state to cpu or specified gpu, which means transferring tensors of the optimizer.state to specified device. The mod

Multiplied by a factor at the beging of specified epochs. Different params groups specify thier own base learning rates. Args: pa

retain topk elements of a matrix and set others 0 Args: matrix (object): np.array 2d

load state_dict of module & optimizer from file Args: modules_optims: A two-element list which contains module and optimizer

copy parameter from src_state_dict to models Arguments: model: A torch.nn.Module object src_state_dict: a dict containing par

maybe_modules, an object or a list of objects.

Args: sys_device_ids: a tuple; which GPUs to use e.g. sys_device_ids = (), only use cpu sys_device_ids = (3,),