Group-wise quantization.
| 8 | |
| 9 | @dataclasses.dataclass |
| 10 | class CompressionConfig: |
| 11 | """Group-wise quantization.""" |
| 12 | |
| 13 | num_bits: int |
| 14 | group_size: int |
| 15 | group_dim: int |
| 16 | symmetric: bool |
| 17 | enabled: bool = True |
| 18 | |
| 19 | |
| 20 | default_compression_config = CompressionConfig( |