r""" Optionally concatenates image path to dataset dir when loading from local disk.

r""" Optionally concatenates video path to dataset dir when loading from local disk.

r""" Encodes formatted inputs to pairs of token ids. Turn 0: prefix + system + query resp Turn t: sep + query

r""" Replaces the sequential sampler of KTO Trainer created by trl with the random sampler.

r""" Activates gradient checkpointing for the current model. Modification of the original method to enable gradient checkpointing for block-w

r""" Calculates model outputs in multiple batches. Subclass and override to inject custom behavior.

r""" Performs block expansion for LLaMA, Mistral, Qwen1.5 or Yi models. Usage: python llama_pro.py --model_name_or_path meta-llama/Llama-2-7b-

r""" Calculates the flops of pre-trained models. Usage: python cal_flops.py --model_name_or_path path_to_model --batch_size 1 --seq_length 512

r""" Calculates the optimal learning rate for 7B/13B models using LLaMA's hyper-parameters. Usage: python cal_lr.py --model_name_or_path p

r""" Calculates MFU for given model and hyper-params. Usage: python cal_mfu.py --model_name_or_path path_to_model --batch_size 1 --seq_length

r""" Calculates the ppl on the dataset of the pre-trained models. Usage: python cal_ppl.py --model_name_or_path path_to_model --dataset alpaca

r""" Judges if the quantization is available in this finetuning type.

r""" Returns the available quantization bits.

r""" Modifys states after changing the training stage.

r""" Gets a list of responses of the chat model.

r""" Checks if the json schema is valid.

r""" Computes pairwise loss. The first n examples are chosen and the last n examples are rejected. Subclass and override to inject cu

r""" Converts alpaca format dataset to the standard format.

r""" Converts sharegpt format dataset to the standard format.

r""" Computes the token with the largest likelihood to reduce memory footprint.

r""" Computes the DPO loss and other metrics for the given batch of inputs for train or test.

r""" Computes the DPO loss and other metrics for the given batch of inputs for train or test.