Configuration for Mcore module wrapper.
| 162 | |
| 163 | @dataclass |
| 164 | class McoreModuleWrapperConfig: |
| 165 | """Configuration for Mcore module wrapper.""" |
| 166 | |
| 167 | is_value_model: bool = False |
| 168 | share_embeddings_and_output_weights: bool = False |
| 169 | wrap_with_ddp: bool = True |
| 170 | use_distributed_optimizer: bool = True |
| 171 | |
| 172 | |
| 173 | def make_megatron_module( |
no outgoing calls
no test coverage detected