Convert applicable model parameters to low-precision (bf16 or fp16)

Crops positional embeddings for SD3 compatibility.

Moves the previous layer cache to the CPU

imgs: [B, 3, H, W] of torch.float32. - mean: [0.48145466, 0.4578275, 0.40821073] - std: [0.26862954, 0.2613025

embed_dim: output dimension for each position pos: a list of positions to be encoded: size (M,) out: (M, D)

Helper function to get activation function from string. Args: act_fn (str): Name of activation function. Returns: nn.Module:

Makes broadcastable attention and causal masks so that future and masked tokens are ignored. Arguments: attention_mask (

Return set of symbol pairs in a word. Word is represented as tuple of symbols (symbols being variable-length strings).

This matches the implementation in Denoising Diffusion Probabilistic Models: Create sinusoidal timestep embeddings. :param timesteps: a 1-D

Replace fp16 linear with quantized linear