Performs all reduce of the tensors after fusing and converting them to FP16. Args: tensor(List of Tensors): a list of tensors to be all-reduced send(bool): When send is False, the tensor won't be send to the target device immediately, it will
(self, tensor, send=True)
| 773 | self.communicator.synchHalf(tensor) |
| 774 | |
| 775 | def fused_all_reduce_half(self, tensor, send=True): |
| 776 | """Performs all reduce of the tensors after fusing and converting them to FP16. |
| 777 | |
| 778 | Args: |
| 779 | tensor(List of Tensors): a list of tensors to be all-reduced |
| 780 | send(bool): When send is False, the tensor won't be send to the |
| 781 | target device immediately, it will be copied to the buffer first |
| 782 | """ |
| 783 | tensor = singa.VecTensor(tensor) |
| 784 | self.communicator.fusedSynchHalf(tensor, send) |
| 785 | |
| 786 | def sparsification(self, tensor, accumulation, spars, topK): |
| 787 | """Performs all reduce of a tensor after sparsification. |