Functions — github.com/Lightning-AI/lit-llama

hub / github.com/Lightning-AI/lit-llama / functions

Functions276 in github.com/Lightning-AI/lit-llama

⨍ Functions 276◇ Types & classes 39 ↳ Endpoints 1

↓ 1 callersFunctionload_datasets

(data_dir)

finetune/full.py:260

↓ 1 callersFunctionload_datasets

(data_dir)

finetune/lora.py:255

↓ 1 callersFunctionload_datasets

(data_dir)

finetune/adapter_v2.py:279

↓ 1 callersFunctionload_datasets

(data_dir: str = "data/shakespeare")

pretrain/shakespeare.py:160

↓ 1 callersFunctionload_eval_data

(dataset_name: str)

evaluate/adapter.py:29

↓ 1 callersFunctionload_eval_data

(dataset_name: str)

evaluate/full.py:25

↓ 1 callersFunctionload_eval_data

(dataset_name: str)

evaluate/lora.py:32

↓ 1 callersFunctionload_eval_data

(dataset_name: str)

evaluate/adapter_v2.py:28

↓ 1 callersFunctionlora_model_lookup

Returns the LoRA rank from the adapter checkpoint.

scripts/convert_lora_weights.py:28

↓ 1 callersFunctionmain

()

pretrain/shakespeare.py:50

↓ 1 callersFunctionmark_only_adapter_as_trainable

Sets `requires_grad=False` for all non-adapter weights.

lit_llama/adapter.py:307

↓ 1 callersFunctionmark_only_adapter_v2_as_trainable

Sets `requires_grad=False` for all non-adapter weights.

lit_llama/adapter_v2.py:18

↓ 1 callersMethodpack_weight

(self, weight)

lit_llama/quantization.py:376

↓ 1 callersFunctionprepare_full

Prepare the "Red Pajama" dataset. We assume tokenizer has been trained (i.e. we reuse LLaMA's tokenizer model).

scripts/prepare_redpajama.py:93

↓ 1 callersFunctionprepare_sample

Prepare the "Red Pajama" dataset. We assume tokenizer has been trained (i.e. we reuse LLaMA's tokenizer model).

scripts/prepare_redpajama.py:45

↓ 1 callersFunctionqlinear_4bit_weight

(inp, weight, scales, zeros)

lit_llama/quantization.py:284

↓ 1 callersMethodquantize_weight

(x, scale, zero, maxq)

lit_llama/quantization.py:472

↓ 1 callersMethodreset_parameters

Reset all the weights, even including pretrained ones.

lit_llama/lora.py:196

↓ 1 callersFunctiontokenize

( tokenizer: Tokenizer, string: str, max_length: int, eos=True )

scripts/prepare_any_text.py:90

↓ 1 callersFunctiontrain

The training loop. Loosely based on the nanoGPT implementation: https://github.com/karpathy/nanoGPT.

finetune/adapter.py:116

↓ 1 callersFunctiontrain

The training loop. Loosely based on the nanoGPT implementation: https://github.com/karpathy/nanoGPT.

finetune/full.py:97

↓ 1 callersFunctiontrain

The training loop. Loosely based on the nanoGPT implementation: https://github.com/karpathy/nanoGPT.

finetune/lora.py:92

↓ 1 callersFunctiontrain

The training loop. Loosely based on the nanoGPT implementation: https://github.com/karpathy/nanoGPT.

finetune/adapter_v2.py:121

↓ 1 callersFunctiontrain

The training loop. Loosely based on the nanoGPT implementation: https://github.com/karpathy/nanoGPT.

pretrain/redpajama.py:126

↓ 1 callersFunctiontrain

The training loop. Loosely based on the nanoGPT implementation: https://github.com/karpathy/nanoGPT.

pretrain/shakespeare.py:81

↓ 1 callersFunctiontrain_tokenizer

(destination_path)

tests/test_packed_dataset.py:10

↓ 1 callersFunctionvalidate

(fabric: L.Fabric, model: torch.nn.Module, val_data: np.ndarray)

finetune/adapter.py:189

↓ 1 callersFunctionvalidate

(fabric: L.Fabric, model: torch.nn.Module, val_data: np.ndarray)

finetune/full.py:171

↓ 1 callersFunctionvalidate

(fabric: L.Fabric, model: torch.nn.Module, val_data: np.ndarray, tokenizer_path: str)

finetune/lora.py:167

↓ 1 callersFunctionvalidate

(fabric: L.Fabric, model: torch.nn.Module, val_data: np.ndarray)

finetune/adapter_v2.py:192

↓ 1 callersFunctionvalidate

( fabric: L.Fabric, model: torch.nn.Module, val_dataloader: DataLoader )

pretrain/redpajama.py:218

↓ 1 callersFunctionvalidate

(fabric: L.Fabric, model: torch.nn.Module, val_data: np.ndarray)

pretrain/shakespeare.py:134

MethodT

(w)

lit_llama/lora.py:256

Method__del__

(self)

lit_llama/packed_dataset.py:217

Method__enter__

(self)

lit_llama/utils.py:108

Method__enter__

(self)

lit_llama/utils.py:467

Method__exit__

(self, exc_type, exc_val, exc_tb)

lit_llama/utils.py:114

Method__exit__

(self, type, value, traceback)

lit_llama/utils.py:498

Method__getattr__

(self, name)

lit_llama/utils.py:271

Method__getitem__

(self, i: int)

finetune/adapter.py:243

finetune/adapter_v2.py:246

Method__init__

(self, start, end)

tests/test_packed_dataset.py:138

Method__init__

( self, batch_size: int, dataset: Optional[Dataset] = None, lengths: Optional[

finetune/length_grouped_sampler.py:67

Method__init__

(self, data: list)

finetune/adapter.py:237

finetune/adapter_v2.py:240

Method__init__

(self, config: LLaMAConfig, block_idx: int)

lit_llama/adapter.py:65

Method__init__

(self, config: LLaMAConfig)

lit_llama/adapter.py:226

Method__init__

Create tensors with given device and dtype and don't run initialization (but instead use "empty tensors", i.e. uninitialized memor

lit_llama/utils.py:74

Method__init__

(self, metatensor, archiveinfo, storageinfo, rebuild_args)

lit_llama/utils.py:169

Method__init__

(self, file, zipfile_context)

lit_llama/utils.py:303

Method__init__

(self, obj, saver, protocol_version=5)

lit_llama/utils.py:348

Method__init__

(self, tensor, saver, protocol_version=5)

lit_llama/utils.py:382

Method__init__

(self, saver, *args, **kwargs)

lit_llama/utils.py:404

Method__init__

(self, name)

lit_llama/utils.py:461

Method__init__

(self, model_path: Path)

lit_llama/tokenizer.py:14

Method__init__

(self, filenames, n_chunks, block_size, seed=12345, shuffle=True, wrap=False, num_processes=1, process_rank=0)

lit_llama/packed_dataset.py:40

Method__init__

( self, outdir, prefix, chunk_size, sep_token, dtype="auto",

lit_llama/packed_dataset.py:71

Method__init__

(self, filenames, n_chunks, block_size, seed, shuffle, wrap)

lit_llama/packed_dataset.py:140

Method__init__

(self, datasets, seed, weights=None)

lit_llama/packed_dataset.py:242

Method__init__

(self, datasets, seed, weights)

lit_llama/packed_dataset.py:255

Method__init__

(self, in_features, out_features, bias: bool, *, bits, tile_cols)

lit_llama/quantization.py:341

Method__init__

( self, linear_module, *, bits, perchannel=True, sym=False,

lit_llama/quantization.py:432

Method__init__

LoRA wrapper around linear class that is used for calculation of q, k and v matrices. This class has three weight matrices: 1. Pr

lit_llama/lora.py:94

Method__init__

Causal self-attention with calculating qkv matrices with a single matrix* and Low Ranking Adaptation for parameter-efficient fine-tuning.

lit_llama/lora.py:408

Method__init__

(self, config: LLaMAConfig)

lit_llama/model.py:52

Method__init__

(self, config: LLaMAConfig)

lit_llama/model.py:149

Method__init__

(self, config: LLaMAConfig)

lit_llama/model.py:172

Method__init__

(self, size: int, dim: int = -1, eps: float = 1e-5)

lit_llama/model.py:264

Method__iter__

(self)

tests/test_packed_dataset.py:143

Method__iter__

(self)

finetune/length_grouped_sampler.py:99

Method__iter__

(self)

lit_llama/packed_dataset.py:50

Method__iter__

(self)

lit_llama/packed_dataset.py:222

Method__iter__

(self)

lit_llama/packed_dataset.py:250

Method__len__

(self)

finetune/length_grouped_sampler.py:96

Method__len__

(self)

finetune/adapter.py:240

finetune/adapter_v2.py:243

Method__next__

(self)

lit_llama/packed_dataset.py:225

Method__next__

(self)

lit_llama/packed_dataset.py:260

Method__post_init__

(self)

lit_llama/model.py:34

Method__reduce_ex__

(self, protocol_version)

lit_llama/utils.py:377

Method__repr__

(self)

lit_llama/utils.py:298

Method__torch_function__

(self, func, types, args=(), kwargs=None)

lit_llama/utils.py:119

Method__torch_function__

(cls, func, types, args=(), kwargs=None)

lit_llama/utils.py:261

Method_init_weights

(self, module: nn.Module)

lit_llama/model.py:70

Method_load_from_state_dict

For backward compatibility with old checkpoints that have a single gating value for all heads.

lit_llama/adapter.py:176

Method_load_from_state_dict

(self, local_state_dict, *args, **kwargs)

lit_llama/quantization.py:52

Functionadapter_v2_new_forward

(self, input: Tensor)

lit_llama/adapter_v2.py:30

Functioncollate_fn

(batch)

finetune/adapter.py:257

finetune/adapter_v2.py:260

Methodcollect_input_stats

(self, _1, inp, _2)

lit_llama/quantization.py:515

Functionconvert_hf_checkpoint

Perform the reverse operation of: https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/convert_llama_weights_to_hf

scripts/convert_hf_checkpoint.py:22

Methoddevice

(self)

tests/test_generate.py:68

← previous next →101–200 of 276, ranked by callers