RepViT block forward: token_mixer DW → optional SE → channel_mixer + residual

Multi-head attention with fused in_proj weights.

MaxPool2d with kernel=2, stride=2 on spatial dims (ne[1]=W, ne[2]=H). Input: [C, W, H, B] -> output: [C, W/2, H/2, B] ggml_pool_2d operates on ne[0] a

SAM2 preprocessing: resize + ImageNet normalization. Returns [C, H, W] float, mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225].

Build the full segmentation head graph. Python UniversalSegmentationHead.forward: 1. Cross-attend encoder_hidden_states to prompt → updated encoder 2

Build the full text encoder computation graph. token_ids: [L] int32 tensor (BPE token IDs, padded to ctx_len with 0s). Must be marked as input by call

Build the full ViT graph. Input: [img_size, img_size, 3, 1] (ggml convention: [W, H, C, B]) Output: [E, W, H, 1] where E=1024, W=H=72

Precompute RoPE frequencies as [N, head_dim/2, 2] (cos, sin pairs). This matches compute_axial_cis() from vitdet.py stored as real (cos, sin) instead

Fill a pre-allocated causal mask buffer (host-side, F16). mask_data must hold L*L ggml_fp16_t values.

Check if position i in s starts a Unicode letter. Handles ASCII letters + treats any multibyte UTF-8 start byte as a letter. This is a reasonable appr

MLP forward: N layers with ReLU (except last), optional sigmoid on last

Random Fourier positional encoding for a single (x, y) coordinate coords_norm: normalized to [0, 1], pe_gaussian: PyTorch row-major [2, 128] Output: [

Helper: capture op counts from a graph.

Helper: run a single sub-graph timing trial. Allocates, warms up, times n_iter runs, reads output tensors, then frees. output_tensors: list of tensors

return 1 if the sum is valid, 0 on overflow. negative terms are considered invalid.

@OPTIMIZE: provide an option that always forces left-predict or paeth predict

Helper: build one perceiver layer on latents, given features x. latents: [D, N_lat, batch] (D=64, N_lat = n latent tokens) x: [D, N_x, batch

Downsample: pre_block → spatial DW s=2 → channel expand → FFN + residual

Check if a click position lands on any tracked instance mask. Returns the instance_id or -1.

Load a .bin + .shape tensor pair.