x_next: [bxa, ts-1, C, H, W] beq_next_frames

x: (N, L, patch_size**2 *chans) imgs: (N, chans, H, W)

imgs: (N, chs, H, W) x: (N, L, patch_size**2 *chns)

Perform per-sample random masking by per-sample shuffling. Per-sample shuffling is done by argsort random noise. x: [N, L, D]

overwrite the original forward_decoder, now input latent are fused

Encoder encodes each timestamp alone Decoder fuses multi timestamp together

overwrite the original forward_decoder, now input latent are fused

embed_dim: output dimension for each position pos: a list of positions to be encoded: size (M,) out: (M, D)

grid_size: int of the grid height and width return: pos_embed: [grid_size*grid_size, embed_dim] or [1+grid_size*grid_size, embed_dim] (w/

This function load state dict to model and optimizer on cpu, and move it back to device. This avoids a GPU memory surge issue.

This function disables printing when not in master process

Decode the input features. Args: x (tensor): layer-0 features. x_1 (tensor): layer-1 features. x_2 (tenso

aggregation, ego use ground truth input, used specifically in inference time

aggregation, ego use ground truth input, used specifically in inference time

aggregation, ego use ground truth input, used specifically in inference time

aggregation, ego use ground truth input, used specifically in inference time

Encode the input BEV features. Args: x (tensor): the input BEV features. Returns: A list that contains all t

overwrite the original forward_decoder, now input latent are fused

overwrite the original forward_decoder, now input latent are fused

x1: [bxa, C, H, W] x_next: [bxa, ts-1, C, H, W] beq_next_frames

as the pre process of the decoder, handles: 1) add mask, restore feature maps shape like (b x agent x channel x h x w) 2) com

as the pre process of the decoder, handles: 1) add mask, restore feature maps shape like (b x agent x channel x h x w) 2) com

overwrite the original forward_loss, now calculate loss on the entire image

overwrite the original forward_loss, now calculate loss on the entire image

imgs: [N, 3, H, W] pred: [N, L, p*p*3] mask: [N, L], 0 is keep, 1 is remove,

reshape the model's predictions back to 256x256x13, do aggregation on this

reshape the model's predictions back to 256x256x13, do aggregation on this

reshape the model's predictions back to 256x256x13, do aggregation on this