Function _should_use_max_completion_tokens

codewiki/src/be/llm_services.py:23–38

Determine whether to use max_completion_tokens instead of max_tokens. Newer OpenAI models (o1, o3, o4, gpt-4o, gpt-5, etc.) require max_completion_tokens. Anthropic and other providers still use max_tokens.

(model_name: str, base_url: str)

# (base_url, model_name) pairs whose provider rejected cache_control markers.
# Module-level because sub-agent runs re-create model instances per tool call
# (see generate_sub_module_documentations); the fallback probe should only
# happen once per provider/model, not once per sub-module.
_CACHE_UNSUPPORTED: set = set()

_EPHEMERAL_CACHE = {"type": "ephemeral"}


def _add_cache_control_to_message(message) -> None:
    """Attach an ephemeral ``cache_control`` marker to an OpenAI-format message in place.

    Tool messages get the marker at message level: OpenAI-compatible proxies
    (e.g. LiteLLM) map that onto the Anthropic ``tool_result`` block, whereas a
    part-level marker would land on a nested block, which Anthropic rejects.
    Other roles get it on the last content part (string content is converted to
    a single text part).
    """
    if message.get("role") == "tool":
        message.setdefault("cache_control", dict(_EPHEMERAL_CACHE))