Types & classes — github.com/AtomaAI/atoma-infer

CodeHub MCPcopy Create free account

hub / github.com/AtomaAI/atoma-infer / types & classes

Types & classes173 in github.com/AtomaAI/atoma-infer

⨍ Functions 613◇ Types & classes 173

↓ 4 callersClassGenerateRequest

backends/vllm/src/types.rs:19

↓ 1 callersEnumModelWorkerError

backends/vllm/src/worker.rs:464

↓ 1 callersEnumTokenizerError

backends/vllm/src/tokenizer.rs:209

csrc/kernels/alibi.h:17

models/src/multi_gpu.rs:59

models/src/multi_gpu.rs:123

EnumAllocationStatus

backends/vllm/src/block_manager.rs:28

csrc/kernels/utils.h:110

ClassAllreduce<2>

csrc/kernels/utils.h:123

server/src/server.rs:89

server/src/server.rs:49

server/src/main.rs:24

models/src/phi3.rs:113

models/src/mistral.rs:113

models/src/llama.rs:384

models/src/llama_nccl.rs:212

ClassBlockAllocator

backends/vllm/src/block_allocator.rs:18

EnumBlockAllocatorError

backends/vllm/src/block_allocator.rs:123

EnumBlockDevice

backends/vllm/src/block.rs:24

backends/vllm/src/block.rs:288

csrc/kernels/block_info.h:12

ClassBlockSpaceManager

backends/vllm/src/block_manager.rs:37

EnumBlockSpaceManagerError

backends/vllm/src/block_manager.rs:1123

Cache for Llama model

models/src/llama.rs:141

ClassCacheConfig

backends/vllm/src/config.rs:137

EnumCacheConfigError

backends/vllm/src/config.rs:326

ClassCacheEngine

`CacheEngine` - Manages the KV cache. This class is responsible for initializing and managing the GPU and CPU KV caches. It also provides methods for

backends/vllm/src/worker.rs:486

EnumCacheEngineError

backends/vllm/src/worker.rs:658

ClassCausalSelfAttention

models/src/llama.rs:203

ClassCausalSelfAttention

models/src/llama_nccl.rs:31

ClassChatCompletionChunk

server/src/api/chat_completions.rs:1046

ClassChatCompletionResponse

server/src/api/chat_completions.rs:936

EnumChatResponse

server/src/server.rs:171

server/src/api/chat_completions.rs:947

models/src/llama.rs:85

InterfaceConfig

`Config` - trait for a LLM model's configuration type

backends/vllm/src/model_executor.rs:116

EnumConfigError

backends/vllm/src/model_executor.rs:597

ClassDecodeTokenizerRequest

`DecodeTokenizerRequest` - A request to decode a single token ID into text

backends/vllm/src/tokenizer.rs:25

ClassDecoderLayer

models/src/phi3.rs:311

ClassDecoderLayer

models/src/mistral.rs:314

server/src/api/chat_completions.rs:1081

csrc/kernels/dropout.h:12

EnumEarlyStopping

backends/vllm/src/sampling_params.rs:24

ClassEncodeTokenizerRequest

Represents a request to encode a string input into tokens.

backends/vllm/src/tokenizer.rs:10

EnumEngineError

backends/vllm/src/llm_engine.rs:653

EnumEngineRequest

Represents different types of requests that can be sent to the LLM engine.

backends/vllm/src/llm_service.rs:53

InterfaceErrString

server/src/api/validate_schema.rs:42

InterfaceEvictor

backends/vllm/src/evictor.rs:6

EnumEvictorError

backends/vllm/src/evictor.rs:139

ClassExecuteModelRequest

backends/vllm/src/sequence.rs:2158

ClassFcfsPolicy

backends/vllm/src/policy.rs:55

EnumFinishReason

server/src/api/chat_completions.rs:956

ClassFlashAttention

Flash attention It encapsulates the flash attention algorithm for fast attention computation. It is further compatible with the paged attention algor

models/src/flash_attention.rs:175

ClassFlashAttention

Flash-attention v2 layer.

csrc/src/lib.rs:15

ClassFlashAttentionDecodingMetadata

`FlashAttentionDecodingMetadata` - Structure wrapping the metadata required for running flash and paged attention kernels for decoding inference

models/src/flash_attention.rs:11

ClassFlashAttentionKvCache

Flash-attention v2 layer, with Key-Value cache. NOTE: We are not passing in each Key and Value tensor for the decoding phase. This is because we plan

csrc/src/lib.rs:1503

ClassFlashAttentionMetadata

`FlashAttentionMetadata` - Structure wrapping the metadata required for running flash and paged attention kernels

models/src/flash_attention.rs:67

ClassFlashAttentionPrefillMetadata

`FlashAttentionPrefillMetadata` - Structure wrapping the metadata required for running flash and paged attention kernels for prefill inference

models/src/flash_attention.rs:34

ClassFlashAttentionVarLen

Flash-attention v2 layer, with variable sequence lengths.

csrc/src/lib.rs:573

ClassFlash_bwd_kernel_traits

csrc/kernels/kernel_traits.h:186

ClassFlash_bwd_params

csrc/kernels/flash.h:138

ClassFlash_fwd_kernel_traits

csrc/kernels/kernel_traits.h:51

ClassFlash_fwd_params

csrc/kernels/flash.h:46

ClassFlash_kernel_traits

csrc/kernels/kernel_traits.h:16

ClassGenerateParameters

backends/vllm/src/types.rs:32

ClassGenerateRequestOutput

backends/vllm/src/llm_engine.rs:526

ClassGenerateStreamingOutput

backends/vllm/src/llm_engine.rs:621

ClassInferenceOutput

backends/vllm/src/llm_engine.rs:599

ClassLRUEvictor

backends/vllm/src/evictor.rs:62

models/src/llama.rs:432

models/src/llama_nccl.rs:266

ClassLlama3RopeConfig

models/src/llama.rs:22

EnumLlama3RopeType

models/src/llama.rs:13

ClassLlamaConfig

models/src/llama.rs:37

EnumLlamaEosToks

models/src/llama.rs:31

ClassLlamaModel

Represents a Llama language model. This struct encapsulates the configuration, device, data type, and the actual Llama model.

backends/vllm/src/models/llama.rs:29

ClassLlamaModel

Represents a Llama language model. This struct encapsulates the configuration, device, data type, and the actual Llama model.

backends/vllm/src/models/llama_nccl.rs:30

`LlmEngine` - An asynchronous worker responsible for scheduling new requests and communicating with the `ModelExecutor` service to send new requests f

backends/vllm/src/llm_engine.rs:36

ClassLlmService

`LlmService` - the entrypoint of the Atoma's inference service. It receives requests from the Atoma's event subscriber service. It validates and token

backends/vllm/src/llm_service.rs:72

EnumLlmServiceError

backends/vllm/src/llm_service.rs:446

backends/vllm/src/sequence.rs:22

ClassLogicalTokenBlock

backends/vllm/src/block.rs:35

csrc/kernels/mask.h:111

csrc/kernels/utils.h:90

ClassMaxOp<float>

csrc/kernels/utils.h:95

server/src/api/chat_completions.rs:178

EnumMessageContent

server/src/api/chat_completions.rs:448

EnumMessageContentPart

server/src/api/chat_completions.rs:502

ClassMessageContentPartImageUrl

server/src/api/chat_completions.rs:535

ClassMistralConfig

models/src/mistral.rs:12

ClassMistralModel

models/src/mistral.rs:366

models/src/llama.rs:354

models/src/phi3.rs:278

models/src/llama_nccl.rs:184

models/src/mistral.rs:282

backends/vllm/src/tests/mod.rs:35

server/src/api/chat_completions.rs:28

ClassModelConfig

backends/vllm/src/config.rs:17

InterfaceModelExecutor

`ModelExecutor` trait - interface for running AI inference from a LLM

backends/vllm/src/model_executor.rs:144

EnumModelExecutorError

backends/vllm/src/model_executor.rs:585

next →1–100 of 173, ranked by callers