r""" Example: top.lang, train.dataset

r""" Gets a standard logger with a stream hander to stdout.

r""" Loads pretrained model and tokenizer. Support both training and inference.

Return the number of layers in a model.

Execute the computed schedule and yield the target values. Yields: Iterator[Tuple[Task, Any]]: An iterator of task-resul

r""" Saves model checkpoint. Subclass and override to inject custom behavior.

r""" Returns a single pair of token ids representing prompt and response respectively.

Return a list of all weights associated with a given layer.

Return a list of all weights following the final layer.

Return a list of all weights preceding the first layer.

r""" Gets the current available device.

r""" Collects GPU memory.

Adds a model to `name`s dependencies if it is not already there and is a merge

r""" The model is already unwrapped. There are three cases: 1. full tuning without ds_zero3: state_dict = {"model.layers.*": ..., "v_head

r""" Gets logits processor that removes NaN and Inf logits.

Determines if the given path is a Hugging Face model repository. Args: path: A string path to check.

Key in config that represents number of layers

Parse a ModelReference. Format: '<MODEL_PATH>(+<LORA_PATH>)?

r""" Converts elements to token ids.

r""" Encodes formatted inputs to pairs of token ids. Turn 0: system + query resp Turn t: sep + query resp

Return all weights associated with a model.

Returns a dictionary of arguments required for this task. The keys of the dictionary are argument names, and the values are Task inst