r""" Aborts the processes recursively in a bottom-up way.

r""" Sets module as a leaf module to skip partitioning in deepspeed zero3.

r""" Aligned dataset: _prompt: [{"role": "user", "content": "..."}] * (2T - 1) _response: [{"role": "assistant", "content": "..."}

r""" Casts projector output to half precision for fine-tuning quantized VLMs.

r""" Checks the version of the required packages.

r""" Calculates the FLOPs of the device capability per second.

r""" Calculates the FLOPs of model per forward/backward pass.

r""" Computes loss for preference learning.

r""" Computes log probabilities of the reference model.

r""" Computes log probabilities of the reference model.

r""" Priority: PTQ-quantized (train/infer) > AutoGPTQ (export) > On-the-fly quantization (train/infer)

r""" Patches VLMs before loading them.

r""" Creates deepspeed config.

r""" Creates reward model for PPO training.

r""" Dumps the layernorm parameters in the model. The model is already unwrapped (and gathered).

r""" Extract a list of tuples from the response message if using tools. Each tuple consists of function name and function arguments.

Extract the contents of a Document with the specified number. Parameters: text (str) -A string containing more than one Docu

Extract the question part from the given text. Parameters: text (str) -A string containing the question and other content

The question and the corresponding document list are extracted from the HTTP response. parameters: - response: requests.Response re

r""" Finds the modules in the expanded blocks to apply lora.

r""" Generates arguments for previewing.

r""" Plots loss curves in LlamaBoard.

r""" Only applies gradient checkpointing to trainable layers.

r""" Gets the attributes of the datasets.

r""" Returns an iterator over all elements with their names.

r""" Gets scores after evaluation.

r""" Gets a list of slots corresponding to a single function call.

r""" Gets id by element.

r""" Computes the number of special tokens per image.

r""" Gets the model path according to the model name.

r""" Computes the patch size of the vit.

r""" Gets the peak memory usage for the current device (in Bytes).

r""" Gets the template name if the model is a chat model.

r""" Gets current date and time.

r""" Gets training infomation for monitor.