return compute throughput in TOPS

Heuristic for the number of splits

Heuristic for the number of splits

If the tensors are already stacked on dimension :code:`dim`, \ returns the strides of the stacked tensors. \ Otherwise returns :c

return compute throughput in TOPS

An addition fused with forward. z = layer.increment_and_forward_(x, y) is equivalent to x += y

Perform a fused linear layer followed by a reduce-scatter

Returns the list of operators used inside `function` with *args and **kwargs

Load K/V for a given block. In case of int4/fp8-quantized K/V, dequantize them after loading. If quantization is group-wise, use group_id to advan

run performance benchmark

Experimental: Makes each block causal with local attention, start from bottom right

Produce lhs, rhs and reference output tensors To dodge numerical accuracy differences between our kernels and PyTorch's ones, we avoid random

Calculates the forward pass of :attr:`xformers.ops.memory_efficient_attention`.

Returns a tuple (output, lse), where `lse` can be used to compute the backward pass later. See :attr:`xformers.ops.memory_efficient_attention

Returns a tuple (output, lse), where `output` is the attention in the style of memory_efficient_attention, and `lse` is extra data, a log-su

Combine attention output computed on different parts of K/V for the same query to get attention on the whole K/V. See https://arxiv.org/abs/2

Computes the best operator Returns: SwiGLUOp: The best operator for the configuration

Pack a tensor into a tensorwise fp8 ScaledTensor.

Start a new phase, and terminate the current one, if one is ongoing.

Graph out the contents of a dict. Dash key means that if the result label has this key, then it will be displayed with a dash

Printout the contents of a dict as a human-readable and Markdown compatible array

Auxiliary int4 row quantization function used for benchmarking and tests. Matches the behaviour of torch.ops.llama_cpp.dequantize_int4_cache

Remove a file like rm -f.

RMS Normalization along the last dimension. This is similar to torch.nn.functional.normalize but with eps being added instead of max.

An addition fused with rms_norm. z = rms_norm_add(x, y, weight, eps) is equivalent to x += y z = rms_norm(x, weigh