Returns a list of tensors with the all-reduce max across `tensors`. The computation is done with an all-reduce operation, so if only some of the returned tensors are evaluated then the computation will hang. Args: tensors: The input tensors across which to reduce; must be assigned
(tensors)
| 111 | |
| 112 | |
| 113 | def all_max(tensors): |
| 114 | """Returns a list of tensors with the all-reduce max across `tensors`. |
| 115 | |
| 116 | The computation is done with an all-reduce operation, so if only some of the |
| 117 | returned tensors are evaluated then the computation will hang. |
| 118 | |
| 119 | Args: |
| 120 | tensors: The input tensors across which to reduce; must be assigned |
| 121 | to GPU devices. |
| 122 | |
| 123 | Returns: |
| 124 | List of tensors, each with the maximum of the input tensors, where tensor i |
| 125 | has the same device as `tensors[i]`. |
| 126 | """ |
| 127 | return _apply_all_reduce('max', tensors) |
| 128 | |
| 129 | |
| 130 | def reduce_sum(tensors): |
nothing calls this directly
no test coverage detected