r""" Reshape the tensor from `[batch_size, seq_len, dim]` to `[batch_size, seq_len, heads, dim // heads]` `heads` is the number of hea

r""" Normalize the encoder hidden states. Requires `self.norm_cross` to be specified when constructing the `Attention` class.

r""" Prepare the attention mask for the attention computation. Args: attention_mask (`torch.Tensor`): The

r""" Reshape the tensor from `[batch_size, seq_len, dim]` to `[batch_size // heads, seq_len, dim * heads]`. `heads` is the number of h

r""" Compute the attention scores. Args: query (`torch.Tensor`): The query tensor. key (`torch.Tensor`): The

Get ray directions for all pixels in camera coordinate. Reference: https://www.scratchapixel.com/lessons/3d-basic-rendering/ r

The [`Transformer1DModel`] forward method. Args: hidden_states (`torch.LongTensor` of shape `(batch size, num latent pix

r""" The forward method of the `Attention` class. Args: hidden_states (`torch.Tensor`): The hidden states