Function get_model_parallel_group

SwissArmyTransformer/sat/mpu/initialize.py:109–113 · view source on GitHub ↗

Get the model parallel group the caller rank belongs to.

()

Source from the content-addressed store, hash-verified

107
108
109	def get_model_parallel_group():
110	"""Get the model parallel group the caller rank belongs to."""
111	assert _MODEL_PARALLEL_GROUP is not None, \
112	'model parallel group is not initialized'
113	return _MODEL_PARALLEL_GROUP
114
115
116	def get_data_parallel_group():

forwardMethod · 0.90

mp_split_model_rank0Function · 0.85

mp_split_model_receiveFunction · 0.85

forwardMethod · 0.85

_build_key_size_numel_dictionariesFunction · 0.85

broadcast_dataFunction · 0.85

get_model_parallel_world_sizeFunction · 0.85

get_model_parallel_rankFunction · 0.85

_reduceFunction · 0.85

_splitFunction · 0.85

_gatherFunction · 0.85

no outgoing calls

no test coverage detected