Convert a single MCP content item to OpenAI format. For text/image/audio content, returns a single content-part dict. For tool_use/tool_resul

Decode a managed container ID to extract provider, model, and original container ID. Returns: DecodedResponseId with custom_llm_p

Estimate total tokens this request will consume so we can reserve them upfront (input + output budget): estimated = input_tok

Execute tool calls and return results.

Extract error dictionary from SSE format chunk. Args: event_line: SSE format event line, e.g. "data: {"error": {...}}\n\n" Retu

Convert the output items in a ``response.completed`` event into Responses API message dicts suitable for the next turn's ``input``.

Resolve MCP access groups to server IDs by querying BOTH the MCP server table (DB) AND config-loaded servers

Helper function to get a metric from the registry by name.

OpenAI Batch metadata only accepts string values.

Get the request body for the Bedrock Image Generation API Checks the model/provider and transforms the request body accordingly

Get the model this session should be routed to, if any.

Resolves the routing strategy and selector to use for the given model. Every model belongs to exactly one group: an explicit entry f

Get tools from available_tools by their names, preserving the semantic router's ordering. Matching is tolerant of client-sid

Get the New Relic trace ID for AI monitoring events. This integration runs in LiteLLM's async logging worker, outside the Ne

Takes raw chunks from Anthropic passthrough endpoint and logs them in litellm callbacks - Builds complete response from chunks

Lazy import to avoid pulling orjson at collection time.

Check if the model is a Nova 2 model that supports reasoningConfig. Nova 2 models use a different reasoning configuration structure

Create a _PROXY_LiteLLMManagedFiles instance with mocked DB and batches. Args: file_id: The unified file ID batches: Lis

Create a mock Request for fetch_mcp_server tests (IP used for access control).

Create a base64-encoded unified file ID.

Apply Presidio output masking (apply_to_output=True) to the ``response.completed`` event before it is forwarded to the client.

Return a mock module so 'import pyroscope' succeeds in _init_pyroscope.

Patch everything after the prompt management block so tests stay unit-level.

Given a media URL (image, audio, or video), return the appropriate PartType for Gemini By the way, actually video_metadata can only be used w

Return a copy of the MCP server object with credentials removed.

Remove a key from both cache_dict and ttl_dict

Resolve a single value that may be a passthrough managed ID. Returns the raw provider ID on success. Returns *managed_id* unchanged when

Check if route matches the wildcard pattern eg. pattern: "/scim/v2/*" route: "/scim/v2/Users" - returns: Tr

Build PolicyDBResponse from a Prisma LiteLLM_PolicyTable row.

Send a message to the backend WebSocket. If a provider_config is set the message is first passed through transform_realtime_request s

Helper that decides if a deployment should be put in cooldown Returns True if the deployment should be put in cooldown Returns False if

The generic `openai` provider also reaches OpenAI-compatible endpoints (a LiteLLM proxy, vLLM, an Anthropic-compatible gateway) via a

Transform Responses API output items into OTEL GenAI 1.38 format. The Responses API returns output as a list of items, each with a

Apply Presidio PII unmasking to backend events before forwarding to the client. Handles two shapes: - ``response.com

Apply a merge-patch of per-member budget fields to a team membership. ``budget_patch`` holds only the budget columns the caller explicitly s

Validate the format of the key_alias. Gated behind ``litellm.enable_key_alias_format_validation`` (default **False**). When disabled, no

Helper method to verify Langfuse API calls

Accept the WebSocket connection

Parameters: - deployment: Deployment - the deployment to be added to the Router Returns: - The added deployment

Apply clear_tool_uses; return (messages, AppliedEdit or None).

Apply Prompt Security guardrail to the given inputs. This method is called by LiteLLM's guardrail framework for ALL endpoints:

Apply the Generic Guardrail API to the given inputs. This is the main method that gets called by the framework. Args:

Implementing async callbacks, to handle asyncio event loop issues when custom integrations need to use async functions.

If the request ``input`` contains litellm-encoded item IDs, decode the embedded ``model_id`` and pin the request to that deployment.

Allow user to modify outgoing streaming data -> Given a whole response iterator. This hook is best used when you need to modify multi

Allow modifying the request just before it's sent to the deployment. Use this instead of 'async_pre_call_hook' when you need to modi

Pre-call hook to scan the request for security threats before sending to LLM. Args: user_api_key_dict: User API key auth

Mock pre MCP tool call hook

Sends the in memory logs queue to datadog api Logs sent to /api/v2/logs DD Ref: https://docs.datadoghq.com/api/latest/logs/

- For counting if the redis, postgres call is successful

Shared streaming data generator: runs proxy iterator hook, per-chunk hook, cost injection, then yields chunks via serialize_chunk; on

Take a blocking function and create an async one that receives the same positional and keyword arguments, and that when called, calls the ori

Return auth contexts that reflect the actual teams for UI session tokens.

Append permissions to existing teams. Either pass team_ids to target specific teams, or set apply_to_all_teams=True to update every team

Returns True if the team can access a specific model. 1. First checks native team-level model permissions (current implementation) 2. If

For a given model, get its available tpm Params: - model: str, the name of the model in the router model_list - prio

Check EU AI Act compliance. Returns: List of compliance check results for: - Art. 9: Guardrails applied

Check GDPR compliance. Returns: List of compliance check results for: - Art. 32: Data protection applied (pr