This DDP communication hook just calls ``allreduce`` using ``GradBucket`` tensors. Once gradient tensors are aggregated across all workers, its ``then`` callback takes the mean and returns the result. If user registers this hook, DDP results is expected to be same as the case where
(
process_group: dist.ProcessGroup,
bucket: dist.GradBucket)
source not stored for this graph (policy: none)
nothing calls this directly
no test coverage detected