hub / github.com/zai-org/CodeGeeX / param_is_not_tensor_parallel_duplicate

Function param_is_not_tensor_parallel_duplicate

codegeex/megatron/mpu/layers.py:49–52 · view source on GitHub ↗

(param)

Source from the content-addressed store, hash-verified

47
48
49	def param_is_not_tensor_parallel_duplicate(param):
50	return (
51	hasattr(param, "tensor_model_parallel") and param.tensor_model_parallel
52	) or (get_tensor_model_parallel_rank() == 0)
53
54
55	def set_tensor_model_parallel_attributes(tensor, is_parallel, dim, stride):

calc_params_l2_normFunction · 0.90

clip_grad_norm_fp32Function · 0.90

count_zeros_fp32Function · 0.90

get_tensor_model_parallel_rankFunction · 0.85

no test coverage detected