Function create_fallback_model

codewiki/src/be/llm_services.py:121–130

Create the fallback LLM model from configuration.

(config: Config)

                if choice.index is None:
                    choice.index = i
        return super()._validate_completion(response)


class CachingOpenAIModel(CompatibleOpenAIModel):
    """CompatibleOpenAIModel that adds Anthropic-style prompt-cache breakpoints.

    Injects ``cache_control: {"type": "ephemeral"}`` markers into the
    OpenAI-format payload at two points: the last system message (on Anthropic
    this caches the tools + system prefix) and the final message (incremental
    multi-turn caching — each turn reads the longest previously cached prefix
    and extends it). OpenAI-compatible proxies such as LiteLLM forward the
    markers to providers that support prompt caching; providers that don't
    either ignore the field or reject the request with a 4xx, in which case the