MCPcopy Create free account

hub / github.com/AtomaAI/atoma-infer / functions

Functions613 in github.com/AtomaAI/atoma-infer

⨍ Functions 613◇ Types & classes 173
↓ 319 callersMethodclone
(&self)
models/src/phi3.rs:145
↓ 37 callersFunctioncreate_dummy_prompt
Create a dummy prompt sequence and sequence group.
backends/vllm/src/sequence.rs:2268
↓ 30 callersFunctioncreate_random_tensor
(device: &Device, dtype: DType)
csrc/tests/cache_manager_tests.rs:13
↓ 29 callersMethodcontains
(&self, block_number: u32)
backends/vllm/src/evictor.rs:84
↓ 28 callersFunctioncopy
csrc/kernels/utils.h:344
↓ 28 callersMethodsequence_id
Returns the unique identifier of the sequence. # Returns The `u64` sequence ID. # Example ``` let sequence = Sequence::new(42, /* ... */); assert_
backends/vllm/src/sequence.rs:968
↓ 26 callersMethodsample
Samples next tokens for a batch of sequence groups # Arguments `logits` - The output logits from the forward pass `sequence_groups_metadata` - Metada
backends/vllm/src/model_executor.rs:173
↓ 24 callersMethoddevice
Getter for `device`
backends/vllm/src/block.rs:192
↓ 24 callersMethodsend
(&self, request: ExecuteModelRequest)
backends/vllm/src/model_executor.rs:522
↓ 22 callersFunctionlinear
(size1: usize, size2: usize, vb: VarBuilder)
models/src/llama_nccl.rs:17
↓ 21 callersMethodread_lock
(&self)
backends/vllm/src/block.rs:271
↓ 18 callersMethodset_sequence_status
Updates the status of the sequence. # Arguments `sequence_status` - The new `SequenceStatus` to set.
backends/vllm/src/sequence.rs:844
↓ 18 callersMethodwrite_lock
(&self)
backends/vllm/src/block.rs:281
↓ 15 callersMethodblock_number
Getter for `block_number`
backends/vllm/src/block.rs:61
↓ 15 callersMethodis_empty
Checks if the current instance is empty
backends/vllm/src/scheduler.rs:292
↓ 14 callersMethodallocate
(&mut self, seq_group: &SequenceGroup)
backends/vllm/src/block_manager.rs:195
↓ 13 callersMethodclone
(&self)
models/src/mistral.rs:149
↓ 13 callersMethodget_number_of_free_gpu_blocks
Returns the number of free GPU blocks available in the block manager. This method provides a quick way to check the current availability of GPU memor
backends/vllm/src/block_manager.rs:974
↓ 13 callersMethodis_empty
Checks if `token_ids` is empty
backends/vllm/src/block.rs:71
↓ 13 callersMethodlength
Returns the total number of tokens in the sequence. This method calculates the sum of prompt tokens and generated output tokens. # Returns `usize`
backends/vllm/src/sequence.rs:240
↓ 13 callersFunctionschedule_and_update_computed_tokens
( scheduler: &mut Scheduler<FcfsPolicy>, )
backends/vllm/src/scheduler.rs:2286
↓ 12 callersMethodadd_sequence_group
(&mut self, sequence_group: SequenceGroup)
backends/vllm/src/scheduler.rs:1739
↓ 12 callersMethodconfig
(&self)
backends/vllm/src/tests/mod.rs:161
↓ 12 callersMethodget_sequence_status
Returns the current status of the sequence. # Returns `SequenceStatus` - The current status of the sequence (e.g., Waiting, Running, Finished).
backends/vllm/src/sequence.rs:835
↓ 12 callersFunctionmessages_to_hermes3_prompt
Function to convert a list of messages to a prompt string in Hermes3 format.
server/src/api/chat_completions.rs:393
↓ 12 callersFunctionround_multiple
(x: usize, m: usize)
csrc/src/lib.rs:2112
↓ 11 callersMethodadd_number_sequences
(&mut self, request_id: String, num_current_sequences: usize)
backends/vllm/src/scheduler.rs:118
↓ 11 callersMethodadd_token_id
(&mut self, token_id: u32, logprob: f32)
backends/vllm/src/sequence.rs:219
↓ 11 callersMethodallocate_and_set_running
Allocates blocks to a `SequenceGroup` and sets its sequences' status to `Running`. This function performs two main tasks: 1. Allocates memory blocks
backends/vllm/src/scheduler.rs:2166
↓ 11 callersMethodget_max_num_running_seqs
Returns the maximum number of sequences that could be running in parallel for this request. This method determines the upper bound of concurrent sequ
backends/vllm/src/sequence.rs:1380
↓ 11 callersFunctionget_sequence_groups
(scheduler_outputs: &SchedulerOutputs)
backends/vllm/src/scheduler.rs:2278
↓ 11 callersFunctionmessages_to_llama3_prompt
Function to convert a list of messages to a prompt string in Llama3 format.
server/src/api/chat_completions.rs:324
↓ 11 callersMethodupdate_num_computed_tokens
( &mut self, num_new_computed_tokens: usize, )
backends/vllm/src/sequence.rs:382
↓ 10 callersMethodadd_num_batched_tokens
(&mut self, request_id: String, num_batched_tokens: usize)
backends/vllm/src/scheduler.rs:94
↓ 10 callersMethodforward
( &mut self, x: &Tensor, input_positions: &Tensor, kv_cache: &Tensor,
models/src/llama.rs:253
↓ 10 callersMethodfree
(&mut self, sequence_id: u64)
backends/vllm/src/block_manager.rs:870
↓ 9 callersFunctionadd_new_token_to_sequence_group
( token_chunk_size: usize, sequence_group: &mut SequenceGroup, token_id: u32, )
backends/vllm/src/scheduler.rs:2335
↓ 9 callersFunctioncopy_blocks
Launches the `copy_blocks_kernel` on the given `key_caches` and `value_caches`, following the `block_mapping`, to copy the blocks on both `key_cache`
csrc/src/cache_manager.rs:148
↓ 9 callersFunctionreshape_thread_tile
csrc/kernels/utils.h:322
↓ 9 callersMethodschedule_prefills
( &mut self, mut waiting_queue: VecDeque<SequenceGroup>, budget: &mut SchedulingBudget
backends/vllm/src/scheduler.rs:897
↓ 9 callersFunctionswap_blocks
Swaps blocks from `src` to `dst` tensors, using the block_mapping. Both `src` and `dst` tensors must have the same dtype, and either be on the same cu
csrc/src/cache_manager.rs:18
↓ 8 callersMethodis_finished
Checks if the sequence has finished processing. Returns `true` if the sequence has reached a terminal state (aborted, ignored, length capped, or stop
backends/vllm/src/sequence.rs:93
↓ 8 callersMethodmessages_to_prompt
(&self, messages: &[Message])
server/src/api/chat_completions.rs:159
↓ 7 callersMethodget_block_size
(&self)
backends/vllm/src/worker.rs:644
↓ 7 callersMethodis_prefill
Checks if the sequence is in the `Prefill` stage. # Returns `true` if the sequence is in the Prefill stage, `false` otherwise. # Example ``` let s
backends/vllm/src/sequence.rs:952
↓ 7 callersMethodrank
Getter for `rank`
backends/vllm/src/sequence.rs:47
↓ 7 callersFunctionreshape_and_cache_flash
Launches the `reshape_and_cache_kernel_flash` on the given `key_caches` and `value_caches`, respecting a slot mapping. # Arguments `key` - A `Tensor
csrc/src/cache_manager.rs:319
↓ 7 callersMethodschedule_swapped
( &mut self, swapped_queue: VecDeque<SequenceGroup>, budget: &mut SchedulingBudget,
backends/vllm/src/scheduler.rs:773
↓ 7 callersMethodswap_out
( &mut self, sequence_group: &mut SequenceGroup, blocks_to_swap_out: &mut HashMap<u32,
backends/vllm/src/scheduler.rs:1976
↓ 6 callersMethodfork
( &mut self, parent_sequence: RwLockReadGuard<Sequence>, child_sequence: RwLockReadGua
backends/vllm/src/block_manager.rs:408
↓ 6 callersMethodforward
( &mut self, xs: &Tensor, input_positions: &Tensor, kv_cache: &Tensor,
models/src/phi3.rs:209
↓ 6 callersMethodforward
( &mut self, xs: &Tensor, input_positions: &Tensor, kv_cache: &Tensor,
models/src/mistral.rs:218
↓ 6 callersMethodget_block_table_ids
Retrieves the block IDs for a given sequence. # Arguments `sequence_id` - The ID of the sequence to retrieve block IDs for. # Returns `Some(Vec<u32>
backends/vllm/src/block_manager.rs:958
↓ 6 callersMethodget_num_free_blocks
Get the number of free blocks for a given device
backends/vllm/src/block_manager.rs:89
↓ 6 callersMethodget_num_tokens
( &self, sequence_group: &SequenceGroup, sequence_status: SequenceStatus, enab
backends/vllm/src/scheduler.rs:1588
↓ 6 callersMethodget_sequence_from_id
Retrieves a shared reference to a `Sequence` with the specified `sequence_id`. This method searches through the sequences in the group and returns a
backends/vllm/src/sequence.rs:1555
↓ 6 callersMethodsliding_window
Getter for `sliding_window`
backends/vllm/src/config.rs:305
↓ 5 callersFunctionadd_new_token
(scheduler: &mut Scheduler<FcfsPolicy>, token_id: u32)
backends/vllm/src/scheduler.rs:2302
↓ 5 callersMethodget_num_sequences
Returns the number of sequences in the group, optionally filtered by a specific status. # Arguments `status` - An optional `SequenceStatus` to filte
backends/vllm/src/sequence.rs:1750
↓ 5 callersMethodget_token_ids
Getter for `token_ids`
backends/vllm/src/block.rs:123
↓ 5 callersMethodhead_dim
(&self)
models/src/phi3.rs:29
↓ 5 callersMethodhead_dim
(&self)
models/src/mistral.rs:29
↓ 5 callersMethodinto_config
(self)
models/src/llama.rs:65
↓ 5 callersMethodmax_num_batched_tokens
Getter for `max_num_batched_tokens`
backends/vllm/src/config.rs:424
↓ 5 callersMethodsplit_kv_cache
Splits the KV cache
models/src/flash_attention.rs:249
↓ 5 callersMethodto_prompt_string
Converts a message to its string representation in the prompt.
server/src/api/chat_completions.rs:228
↓ 4 callersFunctionadd_token_budget
( budget: &mut SchedulingBudget, num_batched_tokens: usize, num_current_sequences: usi
backends/vllm/src/scheduler.rs:2355
↓ 4 callersMethodappend_slots
( &mut self, sequence_group: &SequenceGroup, blocks_to_copy: &mut HashMap<u32, u32>,
backends/vllm/src/scheduler.rs:1683
↓ 4 callersMethodappend_slots
( &mut self, sequence: RwLockReadGuard<Sequence>, )
backends/vllm/src/block_manager.rs:294
↓ 4 callersFunctionapply_softcap
csrc/kernels/flash_fwd_kernel.h:26
↓ 4 callersMethodcan_schedule
( &self, num_new_tokens: usize, num_new_sequences: usize, )
backends/vllm/src/scheduler.rs:71
↓ 4 callersFunctionconvert_layout_acc_rowcol
csrc/kernels/utils.h:186
↓ 4 callersFunctioncreate_test_tensor
(device: &Device, dtype: DType)
csrc/tests/cache_manager_tests.rs:218
↓ 4 callersMethodcumulative_logprob
Returns the cumulative log probability of all generated tokens in the sequence. # Returns `f32` - The sum of log probabilities for all generated tok
backends/vllm/src/sequence.rs:826
↓ 4 callersFunctionembedding
(cfg: &Config, vb: VarBuilder)
models/src/llama_nccl.rs:12
↓ 4 callersMethodfind
Finds a `Sequence` with the given `sequence_id` in this `SequenceGroup`. # Arguments `sequence_id` - The unique identifier of the sequence to find.
backends/vllm/src/sequence.rs:1872
↓ 4 callersFunctiongemm_rs
csrc/kernels/utils.h:164
↓ 4 callersMethodget_block_table
Retrieves the `BlockTable` associated with a given `Sequence`. # Arguments `sequence` - A read guard to the `Sequence` for which to retrieve the bloc
backends/vllm/src/block_manager.rs:941
↓ 4 callersMethodget_number_of_free_cpu_blocks
Returns the number of free CPU blocks available in the block manager. This method provides a quick way to check the current availability of CPU memor
backends/vllm/src/block_manager.rs:986
↓ 4 callersMethodincrement_ref_count
Increments the `ref_count` variable by +1
backends/vllm/src/block.rs:227
↓ 4 callersFunctioninit_tracing
()
backends/vllm/src/tests/mod.rs:354
↓ 4 callersMethodinto_config
(self)
models/src/phi3.rs:37
↓ 4 callersMethodmax_num_sequences
Getter for `max_num_sequences`
backends/vllm/src/config.rs:429
↓ 4 callersMethodpreempt
( &mut self, sequence_group: &mut SequenceGroup, blocks_to_swap_out: &mut HashMap<u32,
backends/vllm/src/scheduler.rs:1778
↓ 4 callersMethodremove
Removes a `Sequence` from this `SequenceGroup`. This method is idempotent - if no sequence with the given ID exists, this method does nothing. # Arg
backends/vllm/src/sequence.rs:1903
↓ 4 callersMethodrun
(mut self)
backends/vllm/src/llm_engine.rs:92
↓ 4 callersFunctionshard
(dim: usize, rank: usize, world_size: usize)
models/src/multi_gpu.rs:182
↓ 4 callersMethodsubtract_num_batched_tokens
(&mut self, request_id: &str, num_batched_tokens: usize)
backends/vllm/src/scheduler.rs:107
↓ 4 callersMethodsubtracts_number_sequences
(&mut self, request_id: &str, num_current_sequences: usize)
backends/vllm/src/scheduler.rs:131
↓ 4 callersMethodverify_args
Verify `CacheConfig` arguments
backends/vllm/src/config.rs:259
↓ 3 callersMethodallocate
(&mut self)
backends/vllm/src/block_allocator.rs:59
↓ 3 callersFunctioncheck_gpu_compatibility
(device_index: usize)
csrc/src/lib.rs:2216
↓ 3 callersFunctioncompute_num_splits
( batch_size: usize, num_heads: usize, head_size: usize, max_seqlen_k: usize,
csrc/src/lib.rs:2169
↓ 3 callersMethodenable_chunked_prefill
Getter for `enable_chunked_prefill`
backends/vllm/src/config.rs:414
↓ 3 callersFunctionflash_attn_kv_cache_full
Flash-attention v2 layer with key and value tensors cached. This implements scaled dot-product attention, `softmax(Q @ K^T . softmax_scale) @ V`. Mul
csrc/src/lib.rs:2083
↓ 3 callersFunctiongemm
csrc/kernels/utils.h:137
↓ 3 callersMethodget_num_computed_tokens
Returns the number of tokens that have been computed so far. This includes both prefill tokens and any generated tokens that have been processed. #
backends/vllm/src/sequence.rs:328
↓ 3 callersMethodget_num_total_logical_token_blocks
Gets the total number of logical token blocks in this sequence. # Returns The number of `LogicalTokenBlock`s in the `logical_token_blocks` vector.
backends/vllm/src/sequence.rs:703
↓ 3 callersMethodget_sequences_ids
Retrieves sequence IDs from the `SequenceGroup`, optionally filtered by status. # Arguments `status` - An optional `SequenceStatus` to filter the se
backends/vllm/src/sequence.rs:1465
↓ 3 callersFunctionhub_load_safetensors
Helper function to download safetensors from the HF API
backends/vllm/src/models/mod.rs:9
next →1–100 of 613, ranked by callers