Results from a throughput test.
| 50 | |
| 51 | @dataclass |
| 52 | class ThroughputResult: |
| 53 | """Results from a throughput test.""" |
| 54 | thread_count: int |
| 55 | tokenizer_name: str |
| 56 | total_text_size_mb: float |
| 57 | total_tokens: int |
| 58 | total_time_seconds: float |
| 59 | throughput_mb_per_sec: float |
| 60 | throughput_tokens_per_sec: float |
| 61 | avg_latency_ms: float |
| 62 | |
| 63 | |
| 64 | class ThroughputBenchmark: |
no outgoing calls