r"""Returns true if `self.data` stored on in pinned memory

r""" When the type of io args is tuple or list, map the leaf of the arguments into map_function(leaf).

r"""Computes and returns a pruned version of input tensor ``t`` according to the pruning rule specified in :meth:`compute_mask`. Args:

r"""ResNet-5 `"Deep Residual Learning for Image Recognition" <https://arxiv.org/pdf/1512.03385.pdf>`_.

signal(2) is really not portable. So use sigaction. http://man7.org/linux/man-pages/man2/signal.2.html

Setters

Getters

tp means "this pointer"

transpose the last two dims of the tensor. Reshape it to 3D if it is 2D.

r"""Updates BatchNorm running_mean, running_var buffers in the model. The documentation is referenced from: https://pytorch.org/docs/stable/o

We can not just simply merging two same split For example, shape = [6], we are trying to merge [2, 2]: (S0, S0) -> [4]: S0 For each rank, [4]: S0 has

check whether the sbp_parallel is legal

Compute the weighted sum of the time and memory cost