A multiturn version of `SFTDataset`, which is an in-memory dataset for supervised finetuning with `input_ids` and `labels`. Args: data: A

An in-memory dataset for supervised finetuning with `input_ids` and `labels`. Args: data: A list of samples (dicts). The target/label mus

An iterator that cycles through an iterable indefinitely. Example: >>> iterator = CycleIterator([1, 2, 3]) >>> [next(iterator) fo

A class designed to manage the packing and storage of large arrays into binary files with a specified chunk size. This class handles the div

Deita data module for supervised finetuning.

MetaMathQA data module for supervised finetuning.

SlimOrcaIDK data module for supervised finetuning.

The implementation is identical to `lit_gpt.model.Block` with the exception that we replace the attention layer where adaption is implemented.

The implementation is identical to `lit_gpt.model.Block` with the exception that we replace the attention layer where adaption is implemented.

Capybara data module for supervised finetuning.

A modification of `lit_gpt.model.CausalSelfAttention` that adds the attention over the adaption prompt.

This is to be attached to our logging interface. Useful at the end of dataset preparation, to tell user how many errors are there.

Evaluation-related arguments

LlamaInstruct data module for supervised finetuning.

A dummy tokenizer that encodes each character as its ASCII code.

SlimOrcaDedup data module for supervised finetuning.

Logs the training throughput and utilization. +-------------------------------------+-----------------------------------------------------------+

A class to manage splitting based on a specified ratio, indicating the frequency of False outcomes. A ratio of 0.9 will return 1 True for eve

Training-related arguments

UltraChat data module for supervised finetuning.

WizardLMEvolInstructV2 data module for supervised finetuning.

Config subclasses ConfigBase to add on things that is not required outside of training, so that data processing can be done. Independent

Args: r: rank of the weight update matrices. To make sense of using LoRA the rank should be smaller than the rank of the weig

This DataCollator is purpose-made for ChatML templates. Referenced: https://medium.com/@xuebinbin12/fine-tuning-chat-based-llm-with-multi-tur

Base class for all data modules in LitGPT.