Convert int64_t stride to int32_t, with overflow check.

Check if two tensors are close enough Return a bool if avoid_sync is False, else return a tensor

Initializes state from arguments.

Return whether we are running under profiling tools like nsys or ncu NOTE cuda-gdb will also cause conflict with CUPTI (bench_kineto) but cu

Get the expanded shape for a non-contiguous tensor. The last dimension is increased by 128 (for alignment), and all other dimensions are incr

Generate random permutations in batch The return tensor, denoted as `res`, has a shape of [batch_size, perm_size]. `0 <= res[i, :] < perm_ran

Return the quantized blocked_k for Flash MLA

For FP8 cases, we need to quantize the KV cache for Flash MLA. Besides, the quantization error may be too large to be distinguished f

Arguments: q: (batch_size, seq_len_q, num_heads_q, head_dim). k_cache: (num_blocks, page_block_size, num_heads_k, head_dim).

Generate test data from a given configuration Return: [cache_seqlens, q, block_table, blocked_k] Pay attention: This function changes the

Calculate the cosine diff between two tensors Return a float if avoid_sync is False, else return a tensor

Get the end-to-end time usage for a sequence of kernels defined as "last kernel end time" - "first kernel start time" If is_u

Get the average each-run time usage of each kernel provided in `kernel_names` If return_avg_individual_run is False, return sum(time

Returns an empty instance of FlashMLASchedMeta. The actual scheduling metadata will be generated during the first invocation of flash_mla_with_kv

A reference implementation in PyTorch