Compiles a tvm function that computes diagonal_mm args: dtype: str in ['float64', 'float32', 'float16'] device: str in ['cpu'

Loads the function from the disk or compile it

Create parameters that allow fast linear stepping of state

Must be called after self.default_state() is used to construct an initial state!

Pykeops version

v, w: (..., N) z: (..., L) returns: (..., L)

Return the hidden representations and predictions. For a sequence (l_1, l_2, ..., l_N), we predict (l_2, ..., l_N, l_{N+1}).

Get the index of the query that can attend to the given key.

Get the key-query index from query-key index for PAM-TVM

Initializes multi-GPU-friendly python logger.

Get the attention mask of PAM-Naive

Get the index of the key that a given query needs to attend to.

Get the query-key index for PAM-TVM

Return w, p, q, V, B such that (w - p q^*, B) is unitarily equivalent to the original HiPPO A, B by the matrix V i.e. A = V[w - p q^*]V^*, B

Return low-rank matrix L such that A + L is normal

Remark: 1 . Currently, dense matrices with sparse multiplication are not supported by Pytorch. Efficient implementation should deal w

Test the time and CUDA memory consumption of PAM.

Test the time and CUDA memory consumption of Prob-sparse self attention.