Copy the ``.grad`` attribute from stored references to fp16 parameters to the ``.grad`` attribute of the fp32 master parameters that are directly updated by the optimizer. :attr:`update_master_grads` only needs to be called if ``fp16_optimizer_obj.backward`` was c
(self)
| 554 | self.update_master_grads() |
| 555 | |
| 556 | def update_master_grads(self): |
| 557 | """ |
| 558 | Copy the ``.grad`` attribute from stored references to fp16 parameters to |
| 559 | the ``.grad`` attribute of the fp32 master parameters that are directly |
| 560 | updated by the optimizer. :attr:`update_master_grads` only needs to be called if |
| 561 | ``fp16_optimizer_obj.backward`` was called with ``update_master_grads=False``. |
| 562 | """ |
| 563 | if self.dynamic_loss_scale: |
| 564 | self._check_overflow() |
| 565 | if self.overflow: return |
| 566 | self._model_grads_to_master_grads() |
| 567 | self._downscale_master() |
| 568 | |
| 569 | def inspect_master_grad_data(self): |
| 570 | """ |
no test coverage detected