Helper function to get the appropriate Bedrock chat config based on model and route. Args: model: The model name/identifier Ret

Cached read of a LiteLLM_Config row; returns row, _ConfigRow shim, or None.

Retrieve API credentials for a model from the LLM Router. Args: llm_router: LiteLLM Router instance model_id: Model name or

Read form data from request Handles when OpenAI SDKs pass form keys as `timestamp_granularities[]="word"` instead of `timestamp_granularitie

Returns: - List of model name strings - Empty list if no models set - If model_access_groups is provided, only return models that are

Returns the vector store from the registry

curl http://localhost:11434/api/show -d '{ "name": "mistral" }'

Get user, org, end-user, and team-membership objects. Returns ``(..., effective_user_id)``: JWT claim unless fuzzy lookup matched a l

Get a prompt template by ID and optional version. Args: prompt_id: The base prompt ID version: Optional vers

Lazily create the SSL context reused by realtime websocket clients so we avoid import-order cycles during startup while keeping a single shar

Get the stored API key from token file. If expected_base_url is provided, the key is only returned when it was originally issued for that URL

Returns a list of valid LLMs based on the set environment variables Args: check_provider_endpoint: If True, will check the provider'

Return the OCI vendor enum for a model name. OCI GenAI uses two ``apiFormat`` values: - ``"COHERE"`` for Cohere models (``cohere.*``) -

Parse the Akto guardrail response. Returns (allowed, reason).

Handles messages with content list conversion

Handle non-streaming A2A request. Args: request_id: A2A JSON-RPC request ID params: A2A MessageSendParams co

Return True if any inspected content is not a plain string. Used by hooks whose mask/redact path operates on string offsets and therefore can

Check if tool search tools are present in the tools list.

List available tools from the server. Args: raise_on_error: When True, re-raise exceptions instead of returning a

List all tools available across all MCP Servers. Args: user_api_key_auth: User authentication mcp_auth_heade

Scan GitLab for all .prompt files under prompts_path, load and parse each, and return the mapping of repo file path -> JSON-like dict

Map OpenAI parameters to Moonshot AI parameters Handles Moonshot AI specific limitations: - tool_choice doesn't support "req

Enable using Milvus `/vectors` endpoint as a pass-through endpoint.

Build a router from a `model_list` using the default `simple-shuffle` strategy.

Handle Anthropic OAuth token detection and header setup. If an OAuth token is detected in the Authorization header, extracts it and sets

Process output response by applying guardrails to text content. Args: response: LiteLLM ModelResponse object

Apply one turn, push session snapshot + bandit deltas to the queue.

Recreate both writer and reader Prisma clients. The writer reconnect path in PrismaClient calls `self.db.recreate_prisma_client(...)`

Removes messages, prompts, input, response from logging. This modifies the data in-place only redacts when litellm.turn_off_message_logging =

Register a key so the replenisher starts warming it. Idempotent. The gateway calls this once per known deployment at startup; the pool only warms keys

Return ``llm_router`` iff it serves ``embedding_model`` as a deployment.

Walk URL path segments and resolve any passthrough managed IDs to raw provider IDs. Returns *path* unchanged when no managed IDs are found.

Store a routing override for a session. Called by guardrails when they detect sensitive data and want to route the session t

1. raise an exception for ContextWindowExceededError if context_window_fallbacks is not None 2. raise an exception for ContentPolicyV

Sign a request using manually provided OCI credentials (user/fingerprint/tenancy/key).

Store message in list

Transform OpenAI-style file creation request into Manus's two-step format. Manus API spec (https://open.manus.im/docs/openai-compati

Transform Bedrock Nova Sonic response to OpenAI realtime format. Args: message: Bedrock format message (JSON string)

Transform the overall request to be sent to the API. Returns: dict: The transformed request. Sent as the body of the API

Transform retrieve file response.

Transform the Veo operation status response. Veo returns: { "name": "operations/generate_1234567890",

Translate Anthropic tools to OpenAI format. Returns: Tuple of (translated_tools, tool_name_mapping) - tool_n

Transform litellm completion response to Google GenAI generate_content format Args: response: ModelResponse from litellm

Translate system message to anthropic format. Removes system message from the original list and returns a new list of anthropic syst

Validate that a team can be moved to an organization. - The org must have access to the team's models - The team budget cannot be greate

Rewrite every text fragment in place via ``visit``. Mutates ``data["messages"]`` and ``data["input"]``. Returns the number of fragments visit