Method all_reduce_half

python/singa/opt.py:767–773

Performs all reduce of a tensor after converting to FP16. Args: tensor(Tensor): a tensor to be all-reduced

(self, tensor)

        self.communicator.fusedSynch(tensor, send)

    def all_reduce_half(self, tensor):
        """Performs all reduce of a tensor after converting to FP16.

        Args:
                tensor(Tensor): a tensor to be all-reduced
        """
        self.communicator.synchHalf(tensor)

    def fused_all_reduce_half(self, tensor, send=True):
        """Performs all reduce of the tensors after fusing and converting them to FP16.