This matches the implementation in Denoising Diffusion Probabilistic Models: From Fairseq. Build sinusoidal embeddings. This matches

Device initialization context manager. A context manager under which models are initialized with all parameters on the specified device.

Defines heuristics to log different conditionings. These can be lists of strings (text-to-image), tensors, ints, ...

Make dataset and collator for supervised fine-tuning.

Given a list of sources, each is a conversation list. This transform: 1. Add signal '### ' at the beginning each sentence, with end signal '\

Collects the state dict and dump to disk.

Resize tokenizer and embedding. Note: This is the unoptimized version that may make your embedding size not be divisible by 64.

Split a list into n (roughly) equal-sized chunks

Split a list into n (roughly) equal-sized chunks

Split a list into n (roughly) equal-sized chunks

Tool function to split the image into tiles @param h: height of the image @param w: width of the image @return: tile_

Split a list of indices into `chunks` chunks of roughly equal lengths.

summarize the mean and var and return a function that apply group norm on each tile

Decode a latent vector z into an image in a tiled manner. @param z: latent vector @return: image

Check whether the text violates OpenAI moderation API.

Apply wavelet blur to the input tensor.

np.array[H, w, C] [0, 255] -> Tensor[C, H, W], RGB, [-1, 1]

Tensor[C, H, W], RGB, [-1, 1] -> PIL.Image