Prints the number of trainable parameters in the model.

Overwrite the `push_to_hub` method in order to force-add the tag "sft" when pushing the model on the Hub. Please refer to `~transform

Map action to the corresponding page

Rotates half the hidden dims of the input.

r""" Handles the early stopping logic. If the policy KL is greater than the target KL, then the gradient is zeroed and the optimizatio

Tokenize a list of strings.

Llama tokenizer does satisfy `enc(a + b) = enc(a) + enc(b)`. It does ensure `enc(a + b) = enc(a) + enc(a + b)[len(enc(a)):]`.

Event stream format: https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events#event_stream_format

Simulate group-wise quantization.

Computes log probabilities of the reference model for a single padded batch of a DPO specific dataset.

Compute per token rewards from scores and KL-penalty. Args: scores (`torch.FloatTensor`): Scores from th

Strip HTML of tags and add separators to convert observation into simple mode

Create OpenAI-style logprobs.

Args: observation (`str`): HTML string available_actions (): ... Returns:

Gather stats from all processes. Useful in the context of distributed training. Args: stats (dict[str, Any]):

Returns list of available actions at the current step

Compute the DPO loss and other metrics for the given batch of inputs for train or test.

keys: `name`, `small_description`, `review` category_type: `category`, `query`

Get the generate_stream function for inference.

Get model judge explanation.

Get model judge explanation.

Render and return the HTML for a product item page

Make dataset and collator for supervised fine-tuning.

Make dataset and collator for supervised fine-tuning.

Extracts the first color found if exists

Parse a username:password file for gradio authorization.

Prompt for output from a role.

Generate random index by sampling uniformly from sum of all weights, then selecting the `min` between the position to keep the list sorted (via bi

Sort by question id and de-duplication

Create a new session and reset environment variables

Return the correct pairwise judge.