Base API class providing common functionality for all API models. This class handles: - Message preprocessing and formatting - Image encoding (base64) - Retry logic with exponential backoff - Parallel request processing
| 31 | |
| 32 | |
| 33 | class BaseAPI: |
| 34 | """Base API class providing common functionality for all API models. |
| 35 | |
| 36 | This class handles: |
| 37 | - Message preprocessing and formatting |
| 38 | - Image encoding (base64) |
| 39 | - Retry logic with exponential backoff |
| 40 | - Parallel request processing |
| 41 | """ |
| 42 | |
| 43 | allowed_types = ["text", "image", "image_url", "system"] |
| 44 | |
| 45 | def __init__( |
| 46 | self, |
| 47 | retry=10, |
| 48 | wait=5, |
| 49 | timeout=(30, 1800), |
| 50 | logger=None, |
| 51 | thread_num=384, |
| 52 | fail_msg="Failed to obtain answer via API.", |
| 53 | openai_format=True, |
| 54 | image_key_name="image", |
| 55 | return_dict=True, |
| 56 | use_system_proxy=False, |
| 57 | **kwargs, |
| 58 | ): |
| 59 | """Initialize the base API. |
| 60 | |
| 61 | Args: |
| 62 | retry: Number of retry attempts on API failure. |
| 63 | wait: Wait time between retries (seconds). |
| 64 | timeout: Request timeout as (connect_timeout, read_timeout) tuple. |
| 65 | logger: Logger instance. |
| 66 | thread_num: Number of parallel threads. |
| 67 | fail_msg: Message returned on failure. |
| 68 | openai_format: Whether to use OpenAI message format. |
| 69 | image_key_name: Key name for image data. |
| 70 | return_dict: Whether to parse response as dict. |
| 71 | use_system_proxy: Whether to use system proxy. |
| 72 | **kwargs: Additional arguments passed to generate_inner. |
| 73 | """ |
| 74 | self.retry = retry |
| 75 | self.wait = wait |
| 76 | self.timeout = timeout |
| 77 | self.logger = logger |
| 78 | self.fail_msg = fail_msg |
| 79 | self.thread_num = thread_num |
| 80 | self.default_kwargs = kwargs if kwargs else {} |
| 81 | self.image_key_name = image_key_name |
| 82 | self.openai_format = openai_format |
| 83 | self.return_dict = return_dict |
| 84 | self.use_system_proxy = use_system_proxy |
| 85 | |
| 86 | @staticmethod |
| 87 | def _collect_memory(trim: bool = False): |
| 88 | """Run garbage collection and optionally trim glibc malloc arenas. |
| 89 | |
| 90 | Args: |
no outgoing calls