MatrixCoord wraps Coord<2, int> to provide a helper for accessing named dimensions. Classes expecting a coordinate in the rank=2 index space of a matr

Coordinate in pitch-linear space

Full model with streaming attention (GPU0) + EP MoE (all GPUs) + CPU experts.

Per-component quantization config for GPU weights. Components NOT configurable (always BF16): embedding, kv_b_proj/w_kc/w_vc, layernorms, gat

IEEE half-precision floating-point type

Floating-point type with 8 bits of exponent and 7 bits of mantissa.

KV cache state for a single sequence (request). Tracks which pages are allocated and current position. Provides paged KV index arrays.

Tensor Float 32 data type

Object to encapsulate the fast division+modulus operation. This object precomputes two values used to accelerate the computation and is best used whe

GemmCoord is a structure derived from Coord<3> that specifies a location within the coordinate space of a GEMM problem.

Mapping function for pitch-linear memory

One configurable item in the TUI.

Object to encapsulate the fast division+modulus operation for 64b integer division. This object precomputes two values used to accelerate the computa

////////////////////////////////////////////////////////// floating-point 8 type : E4M3 //////////////////////////////////////////////////////////

Forward declaration of float_e5m2_t to define float_e4m3_t <=> float_e5m2_t conversions in class float_e4m3_t

Marlin-packed weight for use with gptq_marlin_gemm. Stores packed quantized weight, permuted scales, workspace buffer, scalar type, and origi

Mask object

Defines a canonical 4D coordinate used by tensor operations.

Defines a canonical 5D coordinate used by tensor operations.

Object to encapsulate the fast division+modulus operation for 64b integer division in which the divisor is a power of two.

Streaming weight loader — loads one tensor at a time from safetensors. Uses safe_open with framework="pt" for random-access tensor reading. E

Manages paged KV cache for a set of layers on one GPU. Allocates a fixed pool of pages at init. Sequences claim pages from the free list as t

ScalarType can represent a wide range of floating point and integer types, in particular it can be used to represent sub-byte data types (something th

BatchedGemmCoord is a structure derived from Coord<4> that specifies a location within the coordinate space of a batched GEMM problem.

Wraps HuggingFace tokenizer with chat template formatting.

! Unsigned 128b integer type

One transformer layer: attention + MLP (dense or MoE).

Holds all launcher config values.

Device-side kernel params Arguments has the untransformed problem shape from the user. Params will have the transformed problem shape.

Packed scalar type descriptor matching sgl_kernel's ScalarType. The .id property produces the same int64 as sgl_kernel's ScalarType, used for

Orchestrates benchmark runs across model × config combinations.

FP4 and FP6 types

Mapping function for scenario in which both rows and columns are separated by a stride. Row stride is smaller than column stride in AffineRank2ColumnM

Mapping function for scenario in which both rows and columns are separated by a stride. Column stride is smaller than row stride in AffineRank2RowMajo

Parameters object for Conv2d DGRAD Filter (w) iterator

Gated DeltaNet linear attention for one layer. Maintains internal recurrent state and conv state across forward calls. State is lazily initia

Predicated tile access iterator descriptor object containing template dependent state

Raised when a Qwen image request cannot fit transient vision VRAM.

Main TUI controller.

Per-rank configuration for pipeline parallelism.

Run diverse prompts through a loaded KrasisModel to catch edge cases.

Result from running one combo.

Mapping function for 4-D NHWC tensors.