Tolerance specification for a workload. Specifies the conditions for a particular problem to be deemed numerically correct against the reference implementation.
| 79 | |
| 80 | |
| 81 | class ToleranceSpec(BaseModelWithDocstrings): |
| 82 | """Tolerance specification for a workload. |
| 83 | |
| 84 | Specifies the conditions for a particular problem to be deemed numerically correct against the reference implementation. |
| 85 | """ |
| 86 | |
| 87 | max_atol: float = Field(default=1e-2) |
| 88 | """The maximum absolute error allowed for the problem.""" |
| 89 | max_rtol: float = Field(default=1e-2) |
| 90 | """The maximum relative error allowed for the problem.""" |
| 91 | required_matched_ratio: float = Field(default=0.99) |
| 92 | """The ratio of elements that must pass the correctness bounds to be considered correct.""" |
| 93 | max_error_cap: Optional[float] = Field(default=None) |
| 94 | """Hard ceiling on maximum absolute error. If set, correctness fails when any |
| 95 | element's absolute error exceeds this cap, regardless of matched ratio.""" |
| 96 | allow_negative_inf: bool = Field(default=False) |
| 97 | """When True, matching -inf values in both output and reference are treated as |
| 98 | correct and excluded from error computation. Positions where only one tensor |
| 99 | has -inf still fail. +inf and NaN are unaffected by this flag.""" |
| 100 | |
| 101 | |
| 102 | class Workload(BaseModelWithDocstrings): |