Estimate KV cache VRAM for a given context length. 5-path architecture-aware estimation: 1. MLA -- compressed KV latent + RoPE

True iff the live server already satisfies these load kwargs. Mirrors ``routes/inference.py:_request_matches_loaded_settings`` but co

Block a load when HF's security scan flags unsafe serialized files. Call UNCONDITIONALLY before any load (independent of trust_remote_code): a ma

Hard stop. We terminate the subprocess. Quick + reliable.

Send a status update to the parent process.

Send a response to the parent process; stamps ``ts`` if absent.

Return live model-load progress, or None if not loading. During warm-up llama-server mmaps weight shards into page cache before pushi

True when Origin is missing or matches request's scheme://host:port. Missing Origin counts as same-origin (top-level GETs omit it). Both sides

Draft KV cache bytes at n_ctx, sized from GGUF dims (K and V types are independent). Separate drafter (Gemma): its own KV via _estimate_kv_cac

Unified decode — dispatches to the right codec decoder.

Record a failure but keep running so we report ALL failures. `msg` must be credential-free.

MTP draft reserve at ``n_ctx`` = draft KV (grows with ctx) + separate- drafter weights + (MTP + MLA only) a duplicated target KV context. The

Return the largest context length that fits in GPU VRAM. Budget caps occupancy at ``_CTX_FIT_VRAM_FRACTION`` of the card: an absolute

Rewrite `\\ue200cite\\ue202SOURCE_ID[\\ue202LOCATOR]\\ue201` markers into `[[N]](URL)` per resolvable id. Multi-source markers expand to one link

Strip flags that shadow first-class Studio settings. Used when inheriting a previous load's ``llama_extra_args`` so an inherited `-c 4096` ca

Drive the post-metadata portion of load_model with stubbed inputs. Mirrors llama_cpp.py:1137-1296 to assert the built command, without subpro

Return a clean, path-free public id for *identifier*. - Local GGUF path -> the file stem with ``.gguf`` stripped, e.g. ``/srv/models/Qwen3-

Create a LlamaCppBackend with parsed GGUF metadata from given fields. `general` injects extra `general.*` metadata, to verify the dynamic SWA

Call OpenAI's /v1/responses endpoint and translate its SSE stream back into OpenAI Chat Completions chunk format. The Respon

Record a model's perplexity in the comparison tracker.

Return the llama-server flag list for the requested spec mode. Side effects: sets ``self._speculative_type`` (resolved internal emit)

Install a MockTransport, drive one streamed call, return body+lines.

Format an error as an SSE data line in OpenAI error format.

Call the Anthropic Messages API and translate its SSE to OpenAI format. Anthropic SSE event types: content_block_delta →

Process one generator event, return SSE strings.

Return the user-facing backend name for API responses. ROCm hosts stay ``DeviceType.CUDA`` internally (ROCm reuses ``torch.cuda.*``), but "cu

Patch huggingface_hub.model_info to return one fixed security_repo_status.

Return Windows-only subprocess kwargs that suppress console windows.

Direct import when possible, else AST extraction from vision.py.

Convert ChatMessage list to OpenAI-compatible dicts for external providers. Behaviour per content-part type: - `text`: always preserved.

Callable mimicking transformers' lru_cache-decorated gates. Tracks call count and exposes `cache_clear`. Flip `.next_return` to mimic install

Detect audio/TTS codec; swallows errors (use _strict to distinguish).

Signal generation is complete.

Get a connection to the auth database, creating tables if needed.