Causal 3d convolusion.

Causal self-attention with a single head.

`FlowDPMSolverMultistepScheduler` is a fast dedicated high-order solver for diffusion ODEs. This model inherits from [`SchedulerMixin`] and [

`UniPCMultistepScheduler` is a training-free framework designed for the fast sampling of diffusion models. This model inherits from [`Schedu

Low rank fast weight. This is a compromise to keep the number of parameters low when comparing against baselines. Idealy, low-rank parameteri

Causal LaCT with SwiGLU fast weight function and sliding window attention. Suitable for ordered 1D sequence like language. The sliding windo

XLMRobertaModel with no pooler and no LM head.

On init of fast_weight: Let's start with the magnitude of the value. value_proj is initialized with uniform distribution with range [-1.

Configuration for LaCT-SWIGLU model. It implements the LaCT-SWIGLU layer mixed with in-layer sliding window attention Args: hidd

Base class for diffusion noise schedule.

Self-attention layer Reference: https://github.com/facebookresearch/dino/blob/7c446df5b9f45747937fb0d72314eb9f7b66930a/vision_transformer.py#