Configuration for the distributed execution. Args: pipeline_parallel_size: Number of pipeline parallel groups. tensor_parallel_size: Number of tensor parallel groups. worker_use_ray: Whether to use Ray for model workers. Will be set to True if either pipeline
| 308 | |
| 309 | |
| 310 | class ParallelConfig: |
| 311 | """Configuration for the distributed execution. |
| 312 | |
| 313 | Args: |
| 314 | pipeline_parallel_size: Number of pipeline parallel groups. |
| 315 | tensor_parallel_size: Number of tensor parallel groups. |
| 316 | worker_use_ray: Whether to use Ray for model workers. Will be set to |
| 317 | True if either pipeline_parallel_size or tensor_parallel_size is |
| 318 | greater than 1. |
| 319 | """ |
| 320 | |
| 321 | def __init__( |
| 322 | self, |
| 323 | pipeline_parallel_size: int, |
| 324 | tensor_parallel_size: int, |
| 325 | worker_use_ray: bool, |
| 326 | max_parallel_loading_workers: Optional[int] = None, |
| 327 | ) -> None: |
| 328 | self.pipeline_parallel_size = pipeline_parallel_size |
| 329 | self.tensor_parallel_size = tensor_parallel_size |
| 330 | self.worker_use_ray = worker_use_ray |
| 331 | self.max_parallel_loading_workers = max_parallel_loading_workers |
| 332 | |
| 333 | self.world_size = pipeline_parallel_size * tensor_parallel_size |
| 334 | if self.world_size > 1: |
| 335 | self.worker_use_ray = True |
| 336 | |
| 337 | self.start = 0 |
| 338 | self.end = 0 |
| 339 | self.is_first = True |
| 340 | self.is_last = True |
| 341 | |
| 342 | |
| 343 | class SchedulerConfig: |
no outgoing calls
no test coverage detected