Update target parameters to be closer to those of source parameters using an exponential moving average. :param target_params: the target parameter sequence. :param source_params: the source parameter sequence. :param rate: the EMA rate (closer to 1 means slower).
(target_params, source_params, rate=0.99)
| 53 | |
| 54 | |
| 55 | def update_ema(target_params, source_params, rate=0.99): |
| 56 | """ |
| 57 | Update target parameters to be closer to those of source parameters using |
| 58 | an exponential moving average. |
| 59 | |
| 60 | :param target_params: the target parameter sequence. |
| 61 | :param source_params: the source parameter sequence. |
| 62 | :param rate: the EMA rate (closer to 1 means slower). |
| 63 | """ |
| 64 | for targ, src in zip(target_params, source_params): |
| 65 | targ.detach().mul_(rate).add_(src, alpha=1 - rate) |
| 66 | |
| 67 | |
| 68 | def zero_module(module): |