Configuration object for ONNXRuntime and ONNXRuntime Genai - these parameters are consumed by the ONNXGenerativeModel class in module llmware.models. In most cases, the parameters do not require attention, but provided for more options to adapt to particular environments and use cases.
| 504 | |
| 505 | |
| 506 | class ONNXConfig: |
| 507 | |
| 508 | """ Configuration object for ONNXRuntime and ONNXRuntime Genai - these parameters are consumed by |
| 509 | the ONNXGenerativeModel class in module llmware.models. In most cases, the parameters do not |
| 510 | require attention, but provided for more options to adapt to particular environments and use cases. """ |
| 511 | |
| 512 | # note: breaking changes in the onnxruntime_genai api starting in version 0.6, which was released in |
| 513 | # February 2025 - if you pip install, you should get version 0.6 |
| 514 | # if using an older version of onnxruntime_genai, then set the legacy flag to True |
| 515 | |
| 516 | _conf = {"version": "0.6", |
| 517 | "legacy": False} |
| 518 | |
| 519 | @classmethod |
| 520 | def get_config(cls, param): |
| 521 | return cls._conf[param] |
| 522 | |
| 523 | @classmethod |
| 524 | def set_config(cls, param, value): |
| 525 | cls._conf[param] = value |
| 526 | |
| 527 | @classmethod |
| 528 | def get_legacy_flag(cls): |
| 529 | return cls._conf["legacy"] |
| 530 | |
| 531 | @classmethod |
| 532 | def set_legacy_flag(cls, boolean_val): |
| 533 | cls._conf["legacy"] = boolean_val |
| 534 | return boolean_val |
| 535 | |
| 536 | |
| 537 | class OVConfig: |
no outgoing calls
no test coverage detected