Enhanced Transformer with Rotary Position Embedding. Derived from: https://github.com/labmlai/annotated_deep_learning_paper_implementations/b

Precompute the frequency tensor for complex exponential (cis) with given dimensions. (Note: `cis` means `cos + i * sin`, where i is t

grid_size: int of the grid height and width return: pos_embed: [grid_size*grid_size, embed_dim] or [1+grid_size*grid_size, embed_dim] (w/ or

Get n-D meshgrid with start, stop and num. Args: start (int or tuple): If len(args) == 0, start is num; If len(args) ==

This is a n-d version of precompute_freqs_cis, which is a RoPE for tokens with n-d structure. Args: rope_dim_list (list

This matches the implementation in Denoising Diffusion Probabilistic Models: Create sinusoidal timestep embeddings. :param timesteps: a 1-D

enumerate available model architectures based on config files

Returns the names of available CLIP models

return all models having the specified pretrain tag

return all pretrain tags for the specified model architecture

Load a CLIP model Parameters ---------- name : str A model name listed by `clip.available_models()`, or the path to a model check