Encode with proper resize and normalization

Create and initialize fixed RAE model

Decode encoder output to predictions.

Encode masked continuous tokens.

Center cropping implementation from ADM. https://github.com/openai/guided-diffusion/blob/8fb3ad9197f16bbc40620447b2742e13458d2831/guided_diff

Decode with proper denormalization and clamping

This function disables printing when not in master process

Initialize Gaussian-based quantization bins.

Compute mean of truncated normal distribution in [a, b].

Cache RAE latents to disk.

Center crop matching official implementation

Initialize model weights.

Load decoder weights

Load and preprocess image

Load normalization statistics

Create 3D mask based on order.

Add noise during training

Convert image features to patch sequence.

Quantize continuous values to discrete indices.

Apply random per-element masking across spatial and channel dimensions.

Sample mask ratio from truncated Gaussian.

Generate random ordering for 3D (spatial+channel) sampling.

Generate tokens via iterative unmasking.

Convert patch sequence back to image features.

Update target parameters to be closer to those of source parameters using an exponential moving average.

Args: index (int): Index Returns: tuple: (sample, target, filename).

Args: index (int): Index Returns: tuple: (moments, target).

Initialize linear and layernorm weights.

Decay the learning rate with half-cycle cosine after warmup

Full forward pass

Training forward pass with masked prediction loss.

Create a mask based on the specified order and mask length.

Warning: does not synchronize the deque!