Initialize multiprocessing OpenAI API wrapper. Args: model: Model name, e.g., gpt-4, Qwen2-VL-72B. key: API key (uses OPENAI_API_KEY env var if not provided). temperature: Generation temperature. top_p: Nucleus sampling threshold (0~1).
(
self,
model: str,
key: str = None,
temperature: float = 0.6,
top_p: float = 0.95,
top_k: int = 20,
min_p: float = 0.0,
presence_penalty: float = 1.0,
repetition_penalty: float = 1.0,
api_base: str = None,
port: int = None,
retry: int = 10,
wait: int = 3,
timeout: tuple = (30, 1800),
max_tokens: int = 16384,
thread_num: int = 8192,
return_dict: bool = False,
logger=None,
max_connections: int = None,
num_workers: int = None,
worker_concurrency: int = None,
enable_thinking: bool = False,
**kwargs,
)
| 98 | """ |
| 99 | |
| 100 | def __init__( |
| 101 | self, |
| 102 | model: str, |
| 103 | key: str = None, |
| 104 | temperature: float = 0.6, |
| 105 | top_p: float = 0.95, |
| 106 | top_k: int = 20, |
| 107 | min_p: float = 0.0, |
| 108 | presence_penalty: float = 1.0, |
| 109 | repetition_penalty: float = 1.0, |
| 110 | api_base: str = None, |
| 111 | port: int = None, |
| 112 | retry: int = 10, |
| 113 | wait: int = 3, |
| 114 | timeout: tuple = (30, 1800), |
| 115 | max_tokens: int = 16384, |
| 116 | thread_num: int = 8192, |
| 117 | return_dict: bool = False, |
| 118 | logger=None, |
| 119 | max_connections: int = None, |
| 120 | num_workers: int = None, |
| 121 | worker_concurrency: int = None, |
| 122 | enable_thinking: bool = False, |
| 123 | **kwargs, |
| 124 | ): |
| 125 | """Initialize multiprocessing OpenAI API wrapper. |
| 126 | |
| 127 | Args: |
| 128 | model: Model name, e.g., gpt-4, Qwen2-VL-72B. |
| 129 | key: API key (uses OPENAI_API_KEY env var if not provided). |
| 130 | temperature: Generation temperature. |
| 131 | top_p: Nucleus sampling threshold (0~1). |
| 132 | top_k: Top-K sampling (0 to disable). vLLM/SGLang extra param. |
| 133 | min_p: Minimum probability threshold. vLLM/SGLang extra param. |
| 134 | presence_penalty: Penalize tokens already present in the output. |
| 135 | repetition_penalty: Penalize repeated tokens. vLLM/SGLang extra param. |
| 136 | api_base: API base URL (full URL to chat/completions endpoint). |
| 137 | port: Port number for local deployments. |
| 138 | retry: Number of retry attempts on API failure. |
| 139 | wait: Max wait time between retries (seconds). |
| 140 | timeout: Request timeout as (connect_timeout, read_timeout) tuple. |
| 141 | max_tokens: Maximum tokens in response. |
| 142 | thread_num: Total maximum concurrent requests across all workers. |
| 143 | return_dict: Whether to parse response as dict. |
| 144 | logger: Logger instance. |
| 145 | max_connections: Max TCP connections per worker. Defaults to |
| 146 | min(worker_concurrency, 16384). |
| 147 | num_workers: Number of worker processes. Defaults to |
| 148 | min(cpu_count, 8). Set based on available CPU cores. |
| 149 | worker_concurrency: Async concurrency limit per worker. Defaults to |
| 150 | thread_num // num_workers. |
| 151 | enable_thinking: Whether to enable thinking mode for models that support it |
| 152 | (e.g., Qwen3 on vLLM/SGLang). Default False. |
| 153 | **kwargs: Additional API parameters passed to payload. |
| 154 | """ |
| 155 | super().__init__( |
| 156 | retry=retry, |
| 157 | wait=wait, |