Method
__init__
(self, model_config: ModelConfig, cache_config: CacheConfig, parallel_config: ParallelConfig, scheduler_config: SchedulerConfig, placement_group: PlacementGroup | None, log_stats: bool)
Source from the content-addressed store, hash-verified
| 176 | """Extension of LLMEngine to add async methods.""" |
| 177 | |
| 178 | def __init__(self, model_config: ModelConfig, cache_config: CacheConfig, parallel_config: ParallelConfig, scheduler_config: SchedulerConfig, placement_group: PlacementGroup | None, log_stats: bool) -> None: |
| 179 | super().__init__(model_config, cache_config, parallel_config, scheduler_config, placement_group, log_stats) |
| 180 | |
| 181 | self.pipeline_tasks = [] |
| 182 | self.pipeline_outputs = [] |
| 183 | |
| 184 | async def step_async(self) -> List[RequestOutput]: |
| 185 | """Performs one decoding iteration and returns newly generated results. |
Callers
nothing calls this directly
Tested by
no test coverage detected