Compiles a tvm function that computes diagonal_mm args: dtype: str in ['float64', 'float32', 'float16'] device: str in ['cpu'

Loads the function from the disk or compile it

compute the covariance of the data

Prepare specific parameters for different datasets

Evaluate preptrained models

Evaluate preptrained models

Generate covariates

Get covariates

Get covariates

Generate a mixed sinusoidal sequence

Get covariates

Prepare specific parameters for different datasets

Get the key-query index from query-key index for PAM-TVM

Get the attention mask of PAM-Naive

Get the query-key index for PAM-TVM

Get causal attention mask for decoder.

Main function.

Main function.

Generate multivariate normal distribution

Define the function of covariance decay with distance

Divide the training sequence into windows

Divide the training sequence into windows

synthesis a mixed sinusoidal dataset

Test the time and CUDA memory consumption of PAM.

Test the time and CUDA memory consumption of Prob-sparse self attention.

If you want to thest former's flop, you need to give default value to inputs in model.forward(), the following code can only pass one argument to

SpeedUp version of Autocorrelation (a batch-normalization style design) This is for the inference phase.

SpeedUp version of Autocorrelation (a batch-normalization style design) This is for the training phase.

Returns a list of time features that will be appropriate for the given frequency string. Parameters ---------- freq_str Frequ

Returns a list of time features that will be appropriate for the given frequency string. Parameters ---------- freq_str Frequ

Returns a list of time features that will be appropriate for the given frequency string. Parameters ---------- freq_str Frequ

Start training.

Start training.

Epoch operation in training phase.

Epoch operation in training phase.

Input: batch*seq_len. Output: batch*seq_len*d_model.

Results visualization