MCPcopy Create free account
hub / github.com/AIS-SNU/Smart-Infinity / sparse_allreduce_bucket

Method sparse_allreduce_bucket

deepspeed/runtime/engine.py:2344–2348  ·  view source on GitHub ↗
(self, bucket, dp_group)

Source from the content-addressed store, hash-verified

2342 tensor.orig_dense_tensor.copy_(tensor.to_dense())
2343
2344 def sparse_allreduce_bucket(self, bucket, dp_group):
2345 sparse_list = []
2346 for sparse in bucket:
2347 sparse_list.append(self.sparse_allreduce(sparse, dp_group))
2348 return sparse_list
2349
2350 def sparse_allreduce(self, sparse, dp_group):
2351 original_data_type = sparse.values.dtype

Callers 1

Calls 2

sparse_allreduceMethod · 0.95
appendMethod · 0.45

Tested by

no test coverage detected