Class BatchRequest

optillm/batching.py:27–33

Container for a single request in a batch

@dataclass
class BatchRequest:
    """Container for a single request in a batch"""
    request_data: Dict[str, Any]
    future: Future
    timestamp: float
    model: str
    approach: Optional[str] = None

class BatchingError(Exception):
    """Raised when batch processing fails"""