Retrieves the HTTP client from the cache If not present, creates a new client Caches the new client and returns it.

Context manager that overrides ``user_api_key_auth`` for a role. Usage:: def test_admin_only(client, auth_as): from litellm.

For a given model id, return the model info Returns - dict: the model in list with 'model_name', 'litellm_params', Optional[

Example usage: response = router.completion(model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Hey, how's it going?"}]

Returns the provider for a given model name - e.g. 'azure/chatgpt-v-2' -> 'azure' For router -> Can also give the whole litellm param dict -

Load providers from JSON configuration file

Replace volatile field values with "<VOLATILE>" so dict equality works. Recursive over dicts and lists. Pass an explicit ``volatile`` set to

Recursively serialize data while detecting circular references. If a circular reference is detected then a marker string is returned. NUL

Get a compiled regex pattern by name. Args: pattern_name: Name of the prebuilt pattern Returns: Compiled regex pattern

Helper to construct a guardrail with test defaults.

Calculate the cost of a given completion call fot GPT-3.5-turbo, llama2, any litellm supported llm. Parameters: completion_response

Get the complete URL for the CometAPI call. Returns: str: The complete URL for the API call.

Generate a mock UserAPIKeyAuth object

Adds LiteLLM-specific data to the request. Args: data (dict): The data dictionary to be modified. request (Request): The inc

Common request processing logic for both chat completions and responses API endpoints

Add a callback to litellm.callbacks Ensures no duplicates are added.

Factory for test handlers with standard defaults.

Clear all policies from the registry.

Create a new credential. Args: credential_name (str): Name of the credential credential_info (Dict[str, Any]

Start a span for ``role`` without dedup or attribute mapping. For callers that own and manage their own span lifecycle. ``tracer`` ov

Raises ProxyException (OpenAI API compatible) if an exception is raised

Percent-encode one user-controlled URL path segment. ``urllib.parse.quote(..., safe="")`` intentionally leaves RFC 3986 unreserved characters