MCPcopy Create free account
hub / github.com/NVIDIA/SkillSpector / acquire

Method acquire

contrib/multilingual/api_pool.py:165–237  ·  view source on GitHub ↗

Acquire a slot on the least-loaded available key. Scheduling priority: 1. **Recovered keys** — rate-limited keys whose backoff has expired become available again. 2. **Least-loaded key** — among available keys, pick the one with the fewest ``active_req

(self, timeout: float | None = None)

Source from the content-addressed store, hash-verified

163 # -- Public API -----------------------------------------------------------
164
165 def acquire(self, timeout: float | None = None) -> ApiKey:
166 """Acquire a slot on the least-loaded available key.
167
168 Scheduling priority:
169
170 1. **Recovered keys** — rate-limited keys whose backoff has expired
171 become available again.
172 2. **Least-loaded key** — among available keys, pick the one with
173 the fewest ``active_requests``.
174 3. **Block** — if every non-rate-limited key is at capacity, wait
175 for a slot to free up or a rate-limited key to recover.
176
177 Parameters
178 ----------
179 timeout :
180 Maximum seconds to wait. ``None`` means wait indefinitely.
181
182 Returns
183 -------
184 ApiKey
185 A key with at least one available slot.
186
187 Raises
188 ------
189 RuntimeError
190 If *timeout* expires before a slot becomes available.
191 """
192 deadline = time.monotonic() + timeout if timeout is not None else None
193
194 with self._condition:
195 while True:
196 now = time.monotonic()
197
198 # Step 1: recover rate-limited keys whose backoff has expired
199 self._recover_expired_keys(now)
200
201 # Step 2: find available keys (not rate-limited, slots open)
202 available = [k for k in self._keys if k.available]
203 if available:
204 key = min(available, key=lambda k: k.active_requests)
205 key.active_requests += 1
206 key.total_requests += 1
207 self._total_requests_served += 1
208 _now_active = sum(k.active_requests for k in self._keys)
209 if _now_active > self._peak_active_requests:
210 self._peak_active_requests = _now_active
211 logger.debug(
212 "Pool: slot on key …%s (%d/%d active)",
213 key.key[-8:],
214 key.active_requests,
215 key.max_concurrent,
216 )
217 return key
218
219 # Step 3: no capacity — compute wait time
220 wait_for = self._next_available_in(now)
221 remaining = self._remaining_timeout(deadline)
222 if remaining is not None and remaining <= 0:

Calls 4

_recover_expired_keysMethod · 0.95
_next_available_inMethod · 0.95
_remaining_timeoutMethod · 0.95
_capacity_summaryMethod · 0.95