Functions — github.com/abetlen/llama-cpp-python

↓ 2 callersMethod_decode_token_piece

(piece: Any)

llama_cpp/llama_chat_format.py:3387

↓ 2 callersMethod_evict_if_needed

(self)

examples/server/server.py:12545

↓ 2 callersMethod_evict_until

(self, target_bytes: int)

examples/server/server.py:12538

↓ 2 callersMethod_finalize_response_stream_items

( self, state: "OpenAIFormatter.ResponsesStream", *, finish_reason: Optional[s

examples/server/server.py:9363

↓ 2 callersMethod_find_longest_prefix_key

( self, key: Tuple[int, ...], )

llama_cpp/llama_cache.py:63

↓ 2 callersMethod_find_longest_prefix_key

( self, key: Tuple[int, ...], )

llama_cpp/llama_cache.py:117

↓ 2 callersMethod_find_regex_group_end

(pattern: str, start: int)

examples/server/server.py:4554

↓ 2 callersMethod_finish_tool_call_state

( self, item_state: Dict[str, Any], item_plan: Dict[str, Any], )

examples/server/server.py:6317

↓ 2 callersFunction_format_add_colon_two

Format the prompt with the add-colon-two style.

llama_cpp/llama_chat_format.py:932

↓ 2 callersMethod_format_literal

(self, literal: str)

examples/server/server.py:307

↓ 2 callersMethod_gemma4_tool_call_to_json

(text: str)

examples/server/server.py:4507

↓ 2 callersMethod_generate_constant_rule

(self, value)

llama_cpp/llama_grammar.py:694

↓ 2 callersMethod_generate_constant_rule

(self, value: Any)

examples/server/server.py:523

↓ 2 callersMethod_generate_union_rule

(self, name, alt_schemas)

llama_cpp/llama_grammar.py:492

↓ 2 callersMethod_generate_union_rule

(self, name: str, alt_schemas: List[Dict[str, Any]])

examples/server/server.py:358

↓ 2 callersFunction_grammar_for_json

(verbose: bool = False)

llama_cpp/llama_chat_format.py:1000

↓ 2 callersMethod_init_sampler

( self, top_k: int = 40, top_p: float = 0.95, min_p: float = 0.05, typ

llama_cpp/llama.py:697

↓ 2 callersMethod_locate_prefix_node

( self, sequence_id: int, keep_len: int, )

examples/server/server.py:853

↓ 2 callersMethod_match_single_file

(pattern: str, file_list: Sequence[str])

examples/server/server.py:3259

↓ 2 callersMethod_model_meta_value

(llama_model: Any, key: str)

examples/server/server.py:11849

↓ 2 callersMethod_normalize_tool_call_item

( self, tool_call: Any, *, partial: bool, )

examples/server/server.py:7172

↓ 2 callersMethod_path_for_key

(self, key: str)

examples/server/server.py:10466

↓ 2 callersMethod_pending_allocation_for_capacity

( self, source: Union[CompletionRequest, Completion], already_allocated: int,

examples/server/server.py:13788

↓ 2 callersMethod_raw_object_tool_arguments

(cls, value: str)

examples/server/server.py:5364

↓ 2 callersMethod_regex_capture

(text: str, pattern: str)

examples/server/server.py:4497

↓ 2 callersMethod_regex_capture_end_literal_specs

(cls, pattern: str)

examples/server/server.py:4671

↓ 2 callersMethod_regex_capture_parts

( cls, pattern: str, )

examples/server/server.py:4655

↓ 2 callersMethod_render

( self, *, messages: List[ChatCompletionRequestMessage], media_marker: Optiona

examples/server/server.py:3646

↓ 2 callersMethod_response_custom_tool_call_item

( *, item_id: str, call_id: str, name: str, input_text: str, )

examples/server/server.py:8841

↓ 2 callersMethod_response_function_call_item

( *, item_id: str, call_id: str, name: str, arguments: str, st

examples/server/server.py:8818

↓ 2 callersMethod_response_message_item

( *, item_id: str, text: str, status: str, phase: Optional[str],

examples/server/server.py:8791

↓ 2 callersMethod_response_object

( self, *, body: CreateResponseRequest, response_id: str, created_at:

examples/server/server.py:8995

↓ 2 callersMethod_response_reasoning_effort

( body: CreateResponseRequest, )

examples/server/server.py:7933

↓ 2 callersMethod_response_reasoning_item

( *, item_id: str, text: str, status: str, )

examples/server/server.py:8771

↓ 2 callersMethod_response_status_and_incomplete_details

( self, *, finish_reason: Optional[str], )

examples/server/server.py:8986

↓ 2 callersMethod_response_stream_response

( self, state: "OpenAIFormatter.ResponsesStream", *, status: str, usag

examples/server/server.py:9140

↓ 2 callersMethod_response_tool_call_input_message

( *, name: str, arguments: str, call_id: str, content_type: Optional[s

examples/server/server.py:8072

↓ 2 callersMethod_responses_tool_type_by_name

( tools: Optional[List[Any]], )

examples/server/server.py:8862

↓ 2 callersMethod_responses_usage_from_completion

( self, *, usage: Optional[CompletionUsage], output_items: Sequence[Dict[str,

examples/server/server.py:8952

↓ 2 callersMethod_safe_open

(path: Path)

examples/server/server.py:12388

↓ 2 callersMethod_serialize_tool_arguments

(cls, arguments: Any, *, partial: bool = False)

examples/server/server.py:7357

↓ 2 callersMethod_split_child

( self, parent: "RadixTrie.Node", child: "RadixTrie.Node", prefix_len: int,

examples/server/server.py:833

↓ 2 callersMethod_start_direct_tool_call

(self, tool_call_index: int)

examples/server/server.py:5562

↓ 2 callersMethod_text_tool_argument_from_object

( self, tool_name: str, arguments: Dict[str, Any], )

examples/server/server.py:5314

↓ 2 callersMethod_text_tool_arguments

( self, tool_name: str, arguments: Any, *, partial: bool, )

examples/server/server.py:5333

↓ 2 callersMethod_touch

(entry: "SequenceDiskCache.Entry")

examples/server/server.py:12523

↓ 2 callersMethod_uses_harmony_channels

(self)

examples/server/server.py:7917

↓ 2 callersMethod_validate_text_input

(text: str)

examples/server/server.py:2577

↓ 2 callersMethod_validate_token_input

(tokens: List[int])

examples/server/server.py:2583

↓ 2 callersMethodaccept

(self, seq_id: int, accepted_draft_tokens: int)

examples/server/server.py:1186

↓ 2 callersMethodactivate_request

( self, request: CompletionRequest, *, base_seq_id: int, sibling_seq_i

examples/server/server.py:14968

↓ 2 callersMethodadd_batch_tokens

( self, *, seq_id: int, start_pos: int, tokens: Sequence[int],

examples/server/server.py:12088

↓ 2 callersMethodadd_temp

(self, temp: float)

llama_cpp/_internals.py:729

↓ 2 callersMethodadd_top_k

(self, k: int)

llama_cpp/_internals.py:713

↓ 2 callersMethodadd_top_p

(self, p: float, min_keep: int = 1)

llama_cpp/_internals.py:717

↓ 2 callersMethodbuild_chat_prompt

( self, messages: List[ChatCompletionRequestMessage], *, functions: Optional[L

examples/server/server.py:11992

↓ 2 callersMethodbuild_context_params

( *, n_ctx: Optional[int], n_batch: Optional[int], n_ubatch: Optional[int],

examples/server/server.py:11656

↓ 2 callersMethodclaim_free_sequence

(self, seq_id: int)

examples/server/server.py:14963

↓ 2 callersMethodclear_resident_state

(self)

examples/server/server.py:13124

↓ 2 callersMethodcompletion_request_from_response_chat_parts

( self, body: ResponsesChatRequestParts, )

examples/server/server.py:8612

↓ 2 callersMethodconvert_chat_chunk_to_response_events

( self, chunk: ChatCompletionChunk | Dict[str, Any], state: "OpenAIFormatter.Responses

examples/server/server.py:9492

↓ 2 callersMethodconvert_completion_response_to_chat

( self, completion: OpenAICompletion, tool_name: Optional[str] = None, *,

examples/server/server.py:9783

↓ 2 callersMethodcopy_prompt_state

( self, source_sequence_id: int, dest_sequence_id: int, keep_len: int, )

examples/server/server.py:12676

↓ 2 callersFunctioncreate_app

( settings: Settings | None = None, server_settings: ServerSettings | None = None, model_settings:

llama_cpp/server/app.py:100

↓ 2 callersMethoddetokenize

(self, tokens: List[int], special: bool = False)

llama_cpp/_internals.py:191

↓ 2 callersFunctiondisconnected_cancelled_response_or_raise

( http_request: Request, exc: BaseException, )

examples/server/server.py:15758

↓ 2 callersMethoddraft_batch_size_allowed

( self, batch_items: Sequence["CompletionScheduler.BatchItem"], )

examples/server/server.py:13500

↓ 2 callersMethoddraft_input_ids

(self, completion: Completion)

examples/server/server.py:14194

↓ 2 callersMethodexact_checkpoint_match

(self, tokens: Sequence[int])

examples/server/server.py:12858

↓ 2 callersMethodfinalize_request_if_ready

(self, request: CompletionRequest)

examples/server/server.py:14851

↓ 2 callersMethodflush_stream_updates

( self, completion: Completion, finish_reason: Optional[str], )

examples/server/server.py:14818

↓ 2 callersMethodgenerate

Create a generator of tokens from a prompt. Examples: >>> llama = Llama("models/ggml-7b.bin") >>> tokens = llama.toke

llama_cpp/llama.py:847

↓ 2 callersFunctiongenerate_shared_definitions

(shared_defs, indent_level: int)

llama_cpp/llama_chat_format.py:1524

↓ 2 callersMethodgeneration_kv_for_request

( request: CompletionRequest, prompt_length: int, )

examples/server/server.py:12627

↓ 2 callersMethodget_image_urls

(messages: List[llama_types.ChatCompletionRequestMessage])

llama_cpp/llama_chat_format.py:3673

↓ 2 callersFunctiongpt_params_parse

(argv=None)

examples/low_level_api/common.py:77

↓ 2 callersFunctionhf_tokenizer_config_to_chat_formatter

( tokenizer_config: Dict[str, Any], add_generation_prompt: bool = True, )

llama_cpp/llama_chat_format.py:794

↓ 2 callersMethodhydrate_sequence_cache_match

( self, request: CompletionRequest, seq_id: int, )

examples/server/server.py:13272

↓ 2 callersMethoditem_allows_mtp_processing

(self, item: "CompletionScheduler.BatchItem")

examples/server/server.py:13461

↓ 2 callersMethodload_llama_from_model_settings

(settings: ModelSettings)

llama_cpp/server/model.py:74

↓ 2 callersFunctionmake_request

(url, params=None)

docker/open_llama/hug_model.py:7

↓ 2 callersMethodmaybe_fill_batched_draft_tokens

(self)

examples/server/server.py:14123

↓ 2 callersMethodmaybe_save_prompt_checkpoint

(self, request: CompletionRequest)

examples/server/server.py:14579

↓ 2 callersMethodmedia_slice

( self, row_offset: int, row_count: int, )

examples/server/server.py:3891

↓ 2 callersMethodn_vocab

Return the vocabulary size.

llama_cpp/llama.py:2256

↓ 2 callersMethodnormalized_input

(self)

examples/server/server.py:2595

↓ 2 callersMethodon_stream_chunk

(chunk: CompletionChunk)

examples/server/server.py:15578

↓ 2 callersMethodoutput

(self)

examples/low_level_api/low_level_api_chat_cpp.py:664

↓ 2 callersFunctionparse_model_from_args

Parse a pydantic model from an argparse namespace.

llama_cpp/server/cli.py:121

↓ 2 callersMethodposition_length_for_prefix

(self, sequence_id: int, keep_len: int)

examples/server/server.py:1085

↓ 2 callersMethodprefill

( cls, *, request_id: str, seq_id: int, start_pos:

examples/server/server.py:13018

↓ 2 callersFunctionprepare_messages_for_inference

( messages: List[llama_types.ChatCompletionRequestMessage], functions: Optional[List[llama_typ

llama_cpp/llama_chat_format.py:1575

↓ 2 callersMethodread_input

(self)

examples/low_level_api/low_level_api_chat_cpp.py:692

↓ 2 callersMethodregister_chat_completion_handler

( self, name: str, chat_handler: LlamaChatCompletionHandler, overwrite: bool =

llama_cpp/llama_chat_format.py:120

↓ 2 callersMethodrelease_request

(self, request: CompletionRequest)

examples/server/server.py:14981

↓ 2 callersMethodresolve_refs

Resolves all $ref fields in the given schema, fetching any remote schemas, replacing $ref with absolute reference URL and populating

llama_cpp/llama_grammar.py:439

↓ 2 callersMethodresponse_request_to_chat_parts

( self, body: CreateResponseRequest, )

examples/server/server.py:8426

↓ 2 callersMethodresponse_stream_terminal_events

( self, state: "OpenAIFormatter.ResponsesStream", completion: Optional[OpenAICompletio

examples/server/server.py:9621

↓ 2 callersMethodreturned_tokens

( self, completion: Completion, finish_reason: Optional[str], *, start

examples/server/server.py:10046

↓ 2 callersMethodrollback_draft_verification

( self, item: "CompletionScheduler.BatchItem", keep_len: int, accepted_draft_c

examples/server/server.py:14296

Functions1,336 in github.com/abetlen/llama-cpp-python