Functions — github.com/abetlen/llama-cpp-python

hub / github.com/abetlen/llama-cpp-python / functions

Functions1,336 in github.com/abetlen/llama-cpp-python

⨍ Functions 1,336◇ Types & classes 256↳ Endpoints 95

↓ 1 callersMethodcompatibility_key_for_model

(cls, model: "Model")

examples/server/server.py:12366

↓ 1 callersMethodcompletion_finish_chunk

( self, request: CompletionRequest, completion: Completion, finish_reason: str

examples/server/server.py:10167

↓ 1 callersMethodcompletion_request_from_chat_request

( self, body: CreateChatCompletionRequest, )

examples/server/server.py:8520

↓ 1 callersMethodconsume_completion_chunk

( self, text: str, *, chunk_id: str, created: int, model: str,

examples/server/server.py:7687

↓ 1 callersMethodconvert_completion_response_to_response

( self, completion: OpenAICompletion, body: CreateResponseRequest, tool_name:

examples/server/server.py:9092

↓ 1 callersMethodcopy_draft_sequence

( self, source_seq_id: int, dest_seq_id: int, p0: int, p1: int, )

examples/server/server.py:12188

↓ 1 callersMethodcopy_sequence

( self, source_seq_id: int, dest_seq_id: int, p0: int, p1: int, )

examples/server/server.py:1192

↓ 1 callersFunctioncreate_app

()

examples/server/server.py:15737

↓ 1 callersMethodcreate_chat_completion_openai_v1

Generate a chat completion with return type based on the the OpenAI v1 API. OpenAI python package is required to use this method. Yo

llama_cpp/llama.py:2100

↓ 1 callersMethodcreate_embedding

( self, payload: CreateEmbeddingRequest, )

examples/server/server.py:15534

↓ 1 callersMethoddetect_embedding_model

(cls, llama_model: Any)

examples/server/server.py:11883

↓ 1 callersMethoddetokenize

( self, tokens: List[int], prev_tokens: Optional[List[int]] = None, special: b

llama_cpp/llama_tokenizer.py:54

↓ 1 callersFunctiondownload_file

(url, destination)

docker/open_llama/hug_model.py:29

↓ 1 callersMethoddraft

( self, input_ids: np.ndarray, /, *, seq_id: int, max_tokens:

examples/server/server.py:1170

↓ 1 callersMethoddraft

( self, input_ids: np.ndarray, /, *, seq_id: int, max_tokens:

examples/server/server.py:1726

↓ 1 callersMethoddraft_acceptance_length_metric_lines

(self)

examples/server/server.py:15152

↓ 1 callersMethodembed

( self, inputs: Sequence[Union[str, List[int]]], )

examples/server/server.py:12204

↓ 1 callersMethodencode_embedding

( embedding: Sequence[float], encoding_format: Literal["float", "base64"], dimensions:

examples/server/server.py:2641

↓ 1 callersFunctionenv_or_def

(env, default)

examples/low_level_api/ReasonAct.py:7

↓ 1 callersMethoderror_message_wrapper

Wraps error message in OpenAI style error response

llama_cpp/server/errors.py:125

↓ 1 callersMethodevict_if_needed

(self)

examples/server/server.py:10503

↓ 1 callersMethodfind_candidate_pred_tokens

( input_ids: npt.NDArray[np.intc], max_ngram_size: int, num_pred_tokens: int, )

llama_cpp/llama_speculative.py:25

↓ 1 callersMethodfind_sequence_cache_match

( self, request: CompletionRequest, resident_reuse_len: int, )

examples/server/server.py:13197

↓ 1 callersFunctionformat

( messages: List[llama_types.ChatCompletionRequestMessage], **kwargs: Any, )

llama_cpp/llama_chat_format.py:1115

↓ 1 callersMethodformat_grammar

(self)

llama_cpp/llama_grammar.py:937

↓ 1 callersMethodformat_grammar

(self)

examples/server/server.py:764

↓ 1 callersMethodfrom_embeddings

( cls, *, model: str, embeddings: Sequence[Sequence[float]], total_tok

examples/server/server.py:2658

↓ 1 callersMethodfrom_prepared

( cls, *, payload: CreateCompletionRequest, prompt_text: str, prompt_p

examples/server/server.py:4201

↓ 1 callersMethodfrom_pretrained

( cls, repo_id: str, filename: Optional[str], local_dir: Optional[Union[str, o

llama_cpp/llama_chat_format.py:3692

↓ 1 callersMethodgenerate

(self)

examples/low_level_api/low_level_api_chat_cpp.py:338

↓ 1 callersFunctiongenerate_streaming

(tools, functions, function_call, prompt)

llama_cpp/llama_chat_format.py:2140

↓ 1 callersMethodget_chat_completion_handler_by_name

( self, name: str )

llama_cpp/llama_chat_format.py:138

↓ 1 callersMethodget_image_urls

(messages: List[llama_types.ChatCompletionRequestMessage])

llama_cpp/llama_chat_format.py:3140

↓ 1 callersMethodget_logits

(self)

llama_cpp/_internals.py:345

↓ 1 callersMethodget_sampler

(self)

llama_cpp/_internals.py:671

↓ 1 callersFunctionget_server_settings

()

llama_cpp/server/app.py:58

↓ 1 callersFunctionget_user_choice

(model_list)

docker/open_llama/hug_model.py:51

↓ 1 callersFunctiongpt_random_prompt

(rng)

examples/low_level_api/common.py:389

↓ 1 callersFunctionhf_autotokenizer_to_chat_formatter

( pretrained_model_name_or_path: Union[str, os.PathLike[str]], )

llama_cpp/llama_chat_format.py:760

↓ 1 callersMethodinitial_prompt_records

( self, payload: CreateCompletionRequest, prompt_plan: PromptPlan, prompt_visi

examples/server/server.py:15711

↓ 1 callersMethodis_boundary

(self, pos: int)

examples/server/server.py:3971

↓ 1 callersMethodis_sequence_cache_match_usable

( self, request: CompletionRequest, match: SequenceCache.Match, resident_reuse

examples/server/server.py:13177

↓ 1 callersFunctionjson_schema_to_gbnf

(schema: str, prop_order: Optional[List[str]] = None)

llama_cpp/llama_grammar.py:944

↓ 1 callersMethodkey_for_media

(self, kind: Literal["image", "audio", "video"], media_bytes: bytes)

examples/server/server.py:10458

↓ 1 callersMethodlast_output_index

(output_indices: Sequence[Optional[int]])

examples/server/server.py:14611

↓ 1 callersFunctionllama_set_adapter_cvec

Apply a loaded control vector to a llama_context, or if data is NULL, clear the currently loaded vector. n_embd should be the size of a single

llama_cpp/llama_cpp.py:2227

↓ 1 callersFunctionllama_set_adapters_lora

Set LoRA adapters on the context if they differ from the current adapters.

llama_cpp/llama_cpp.py:2177

↓ 1 callersMethodload_image

(self, image_url: str)

llama_cpp/llama_chat_format.py:2830

↓ 1 callersMethodload_image

(self, image_url: str)

llama_cpp/llama_chat_format.py:3324

↓ 1 callersMethodload_media

(self, media: MediaInput)

examples/server/server.py:10711

↓ 1 callersMethodload_sequence_state_bytes

( self, seq_id: int, state_bytes: np.ndarray, )

examples/server/server.py:13234

↓ 1 callersMethodlogits_to_logprobs

(logits: np.ndarray)

examples/server/server.py:13357

↓ 1 callersMethodlookup

(self, tokens: Sequence[int])

examples/server/server.py:4149

↓ 1 callersFunctionmain

()

llama_cpp/server/__main__.py:43

↓ 1 callersFunctionmain

()

docker/open_llama/hug_model.py:73

↓ 1 callersFunctionmain

()

examples/server/server.py:16352

↓ 1 callersFunctionmain

(args)

examples/low_level_api/quantize.py:6

↓ 1 callersMethodmaybe_fill_draft_tokens

(self, completion: Completion)

examples/server/server.py:14080

↓ 1 callersMethodmedia_inputs_from_messages

( messages: Sequence[ChatCompletionRequestMessage], )

examples/server/server.py:3503

↓ 1 callersFunctionmessage_to_str

(msg: llama_types.ChatCompletionRequestMessage)

llama_cpp/llama_chat_format.py:1625

↓ 1 callersFunctionmtmd_get_audio_sample_rate

Get the audio sample rate in Hz. Returns -1 if audio is not supported.

llama_cpp/mtmd_cpp.py:309

↓ 1 callersFunctionmtmd_helper_bitmap_init_from_buf_wrapper

( ctx: mtmd_context_p, buf: CtypesArray[c_uint8], length: Union[c_size_t, int], placeholder: U

llama_cpp/mtmd_cpp.py:744

↓ 1 callersFunctionmtmd_helper_bitmap_init_from_file_wrapper

Initialize an MTMD bitmap wrapper from a file.

llama_cpp/mtmd_cpp.py:724

↓ 1 callersMethodn_embd

Return the embedding size.

llama_cpp/llama.py:2252

↓ 1 callersMethodobserve_decode

( self, items: Sequence[Any], elapsed_seconds: float, )

examples/server/server.py:4097

↓ 1 callersMethodobserve_draft_process

( self, items: Sequence["CompletionScheduler.BatchItem"], elapsed_seconds: float,

examples/server/server.py:13452

↓ 1 callersMethodobserve_predicted_token

(self)

examples/server/server.py:4127

↓ 1 callersMethodon_done

(result: OpenAICompletion)

examples/server/server.py:15581

↓ 1 callersMethodon_error

(exc: BaseException)

examples/server/server.py:15586

↓ 1 callersMethodparse_completion_message

(self, response_text: str)

examples/server/server.py:7529

↓ 1 callersMethodpooling_type

Return the pooling type.

llama_cpp/llama.py:2276

↓ 1 callersMethodposition_increments_up_to

(self, pos: int)

examples/server/server.py:3958

↓ 1 callersMethodposition_length

(self, sequence_id: int)

examples/server/server.py:1082

↓ 1 callersMethodprepare_completion_prompt

( self, payload: CreateCompletionRequest, )

examples/server/server.py:15681

↓ 1 callersMethodprev_text_tokens_at

(self, pos: int)

examples/server/server.py:4024

↓ 1 callersMethodprint_timings

(self)

llama_cpp/_internals.py:457

↓ 1 callersMethodprocess

(self, batch: Any, /)

examples/server/server.py:1183

↓ 1 callersMethodprocess_batch

(self, items: List[CompletionScheduler.BatchItem])

examples/server/server.py:14005

↓ 1 callersMethodprocess_draft_batch

(self)

examples/server/server.py:12172

↓ 1 callersMethodprocess_generation_item

( self, completion: Completion, item: CompletionScheduler.BatchItem, output_co

examples/server/server.py:14366

↓ 1 callersMethodprocess_sampled_draft_batch

( self, items: Sequence["CompletionScheduler.BatchItem"], )

examples/server/server.py:14200

↓ 1 callersMethodprompt_visible_start

(self, prompt_plan: PromptPlan)

examples/server/server.py:15705

↓ 1 callersMethodremove_pending_request

(self, request: CompletionRequest)

examples/server/server.py:14992

↓ 1 callersMethodrequest_from_payload

( self, payload: CreateCompletionRequest, )

examples/server/server.py:15637

↓ 1 callersMethodrequest_needs_uncached_prompt_logprobs

(request: CompletionRequest)

examples/server/server.py:13162

↓ 1 callersMethodrequire_prefill

(self)

examples/server/server.py:12998

↓ 1 callersMethodreset

(self)

llama_cpp/_internals.py:502

↓ 1 callersMethodreset_timings

(self)

llama_cpp/_internals.py:454

↓ 1 callersMethodresolve_draft_model_path

(self)

examples/server/server.py:3443

↓ 1 callersMethodresolve_embedding_mode

( cls, llama_model: Any, embedding: Optional[bool], )

examples/server/server.py:11900

↓ 1 callersMethodresolve_mmproj_path

(self)

examples/server/server.py:3362

↓ 1 callersMethodresolve_path

(self)

examples/server/server.py:3331

↓ 1 callersMethodresolve_refs

(self, schema: Dict[str, Any], url: str)

examples/server/server.py:329

↓ 1 callersMethodresponses_input_to_chat_messages

( self, body: CreateResponseRequest, )

examples/server/server.py:8137

↓ 1 callersMethodreturned_output_end

( self, completion: Completion, finish_reason: Optional[str], )

examples/server/server.py:10021

↓ 1 callersMethodsample

(self, ctx: LlamaContext, idx: int = -1)

llama_cpp/_internals.py:857

↓ 1 callersMethodsample

(self, ctx: llama_cpp.llama_context_p, output_index: int)

examples/server/server.py:10338

↓ 1 callersMethodsample_completion

( self, completion: Completion, output_index: Optional[int], )

examples/server/server.py:14698

↓ 1 callersMethodsample_completion_from_logits

( self, completion: Completion, logits: np.ndarray, )

examples/server/server.py:14739

↓ 1 callersMethodsample_logits

(self, logits: np.ndarray)

examples/server/server.py:10366

← previous next →501–600 of 1,336, ranked by callers