Metadata for a sequence group. Used to create `InputMetadata`. Args: request_id: The ID of the request. is_prompt: Whether the request is at prompt stage. seq_data: The sequence data. (Seq id -> sequence data) sampling_params: The sampling parameters used to gen
| 335 | |
| 336 | |
| 337 | class SequenceGroupMetadata: |
| 338 | """Metadata for a sequence group. Used to create `InputMetadata`. |
| 339 | |
| 340 | |
| 341 | Args: |
| 342 | request_id: The ID of the request. |
| 343 | is_prompt: Whether the request is at prompt stage. |
| 344 | seq_data: The sequence data. (Seq id -> sequence data) |
| 345 | sampling_params: The sampling parameters used to generate the outputs. |
| 346 | block_tables: The block tables. (Seq id -> list of physical block |
| 347 | numbers) |
| 348 | """ |
| 349 | |
| 350 | def __init__( |
| 351 | self, |
| 352 | request_id: str, |
| 353 | is_prompt: bool, |
| 354 | seq_data: Dict[int, SequenceData], |
| 355 | sampling_params: SamplingParams, |
| 356 | block_tables: Dict[int, List[int]], |
| 357 | ) -> None: |
| 358 | self.request_id = request_id |
| 359 | self.is_prompt = is_prompt |
| 360 | self.seq_data = seq_data |
| 361 | self.sampling_params = sampling_params |
| 362 | self.block_tables = block_tables |
| 363 | |
| 364 | |
| 365 | class SequenceOutput: |
no outgoing calls
no test coverage detected