Functions — github.com/architkaila/Fine-Tuning-LLMs-for-Medical-Entity-Extraction

hub / github.com/architkaila/Fine-Tuning-LLMs-for-Medical-Entity-Extraction / functions

Functions184 in github.com/architkaila/Fine-Tuning-LLMs-for-Medical-Entity-Extraction

⨍ Functions 184◇ Types & classes 39

Method__init__

( self, filenames, n_chunks, block_size, seed=12345, shuffle=True, wrap=False, num_processes=1, proces

lit_gpt/packed_dataset.py:28

Method__init__

(self, outdir, prefix, chunk_size, sep_token, dtype="auto", vocab_size=None)

lit_gpt/packed_dataset.py:61

Method__init__

(self, filenames, n_chunks, block_size, seed, shuffle, wrap)

lit_gpt/packed_dataset.py:122

Method__init__

(self, datasets, seed, weights=None)

lit_gpt/packed_dataset.py:215

Method__init__

(self, datasets, seed, weights)

lit_gpt/packed_dataset.py:230

Method__init__

(self, size: int, dim: int = -1, eps: float = 1e-5)

lit_gpt/rmsnorm.py:11

Method__init__

Store LoRA specific attributes in a class. Args: r: rank of the weight update matrices. To make sense of using LoRA the rank shou

lit_gpt/lora.py:63

Method__init__

LoRA wrapper around linear class. This class has three weight matrices: 1. Pretrained weights are stored as `self.linear.weight`

lit_gpt/lora.py:89

Method__init__

LoRA wrapper around linear class that is used for calculation of q, k and v matrices. This class has three weight matrices: 1. Pr

lit_gpt/lora.py:155

Method__init__

(self, config: Config)

lit_gpt/lora.py:523

Method__init__

(self, config: Config)

lit_gpt/lora.py:535

Method__init__

(self, config: Config)

lit_gpt/lora.py:580

Method__init__

(self, config: Config)

lit_gpt/lora.py:614

Method__init__

(self, in_features: int, out_features: int, **kwargs)

lit_gpt/adapter_v2.py:48

Method__init__

(self, config: Config, block_idx: int)

lit_gpt/adapter_v2.py:101

Method__init__

(self, config: Config, block_idx: int)

lit_gpt/adapter_v2.py:116

Method__init__

(self, config: Config)

lit_gpt/adapter_v2.py:154

Method__init__

(self, config: Config)

lit_gpt/adapter_v2.py:174

Method__init__

(self, config: Config)

lit_gpt/model.py:136

Method__init__

(self, config: Config)

lit_gpt/model.py:170

Method__init__

(self, config: Config)

lit_gpt/model.py:265

Method__init__

(self, config: Config)

lit_gpt/model.py:279

Method__init__

( self, k_shape: Tuple[int, int, int, int], v_shape: Tuple[int, int, int, int],

lit_gpt/model.py:323

Method__iter__

(self)

lit_gpt/packed_dataset.py:40

Method__iter__

(self)

lit_gpt/packed_dataset.py:197

Method__iter__

(self)

lit_gpt/packed_dataset.py:225

Method__next__

(self)

lit_gpt/packed_dataset.py:200

Method__next__

(self)

lit_gpt/packed_dataset.py:235

(self, protocol_version)

lit_gpt/utils.py:114

Method_init_weights

Meant to be used with `gpt.apply(gpt._init_weights)`. Unused method left for completeness.

lit_gpt/adapter.py:78

Method_init_weights

Meant to be used with `gpt.apply(gpt._init_weights)`. Unused method left for completeness.

lit_gpt/lora.py:509

Method_init_weights

Meant to be used with `gpt.apply(gpt._init_weights)`. Unused method left for completeness.

lit_gpt/adapter_v2.py:84

Method_init_weights

Meant to be used with `gpt.apply(gpt._init_weights)`.

lit_gpt/model.py:61

Method_load_from_state_dict

For compatibility with older checkpoints.

lit_gpt/adapter.py:151

Method_load_from_state_dict

For compatibility with base checkpoints.

lit_gpt/lora.py:567

Method_load_from_state_dict

For compatibility with base checkpoints.

lit_gpt/lora.py:601

Method_load_from_state_dict

For compatibility with base checkpoints.

lit_gpt/lora.py:641

Method_load_from_state_dict

For compatibility with base checkpoints.

lit_gpt/adapter_v2.py:138

Method_load_from_state_dict

For compatibility with base checkpoints.

lit_gpt/adapter_v2.py:161

Method_load_from_state_dict

For compatibility with base checkpoints.

lit_gpt/adapter_v2.py:180

Methodadd_array

(self, arr)

lit_gpt/packed_dataset.py:106

Functionconvert_hf_checkpoint

( *, checkpoint_dir: Path = Path("checkpoints/stabilityai/stablelm-base-alpha-3b"), model_name: Op

scripts/convert_hf_checkpoint.py:248

Functioncopy_weights_falcon

( model_name: str, state_dict: Dict[str, torch.Tensor], hf_weights: Dict[str, Union[torch.Tensor,

scripts/convert_hf_checkpoint.py:66

Functioncopy_weights_gpt_neox

( state_dict: Dict[str, torch.Tensor], hf_weights: Dict[str, Union[torch.Tensor, NotYetLoadedTensor]],

scripts/convert_hf_checkpoint.py:23

Functioncopy_weights_hf_llama

( config: Config, qkv_weights: Dict[int, List[Optional[NotYetLoadedTensor]]], state_dict: Dict[str

scripts/convert_hf_checkpoint.py:115

Functioncopy_weights_phi

( config: Config, state_dict: Dict[str, torch.Tensor], hf_weights: Dict[str, Union[torch.Tensor, N

scripts/convert_hf_checkpoint.py:177

Functiondownload_from_hub

( repo_id: Optional[str] = None, access_token: Optional[str] = os.getenv("HF_TOKEN"), from_safeten

scripts/download.py:19

Functionestimate_flops

Measures estimated FLOPs for MFU. Refs: * https://ar5iv.labs.arxiv.org/html/2205.05198#A1 * https://ar5iv.labs.arxiv.org/html/220

lit_gpt/utils.py:330

Methodfilenames

(self)

lit_gpt/packed_dataset.py:103

Methodforward

( self, idx: torch.Tensor, input_pos: Optional[torch.Tensor] = None, lm_head_chunk_size: int = 0 )

lit_gpt/adapter.py:47

Methodforward

(self, x: torch.Tensor)

lit_gpt/rmsnorm.py:17

Methodforward

(self, x: torch.Tensor)

lit_gpt/lora.py:143

Methodforward

Do the forward pass. If LoRA's weights are merged with pretrained ones then it's a simple matrix multiplication. If not, then multipl

lit_gpt/lora.py:351

Methodforward

( self, idx: torch.Tensor, input_pos: Optional[torch.Tensor] = None, lm_head_chunk_size: int = 0 )

lit_gpt/lora.py:478

Methodforward

(self, x: torch.Tensor)

lit_gpt/adapter_v2.py:54

Methodforward

(self, idx: torch.Tensor, input_pos: Optional[torch.Tensor] = None)

lit_gpt/model.py:70

Methodforward

( self, x: torch.Tensor, cos: torch.Tensor, sin: torch.Tensor, mask: O

lit_gpt/model.py:145

Methodforward

( self, x: torch.Tensor, cos: torch.Tensor, sin: torch.Tensor, mask: O

lit_gpt/model.py:182

Methodforward

(self, x: torch.Tensor)

lit_gpt/model.py:272

Methodforward

(self, x: torch.Tensor)

lit_gpt/model.py:285

Methodforward

(self, input_pos: torch.Tensor, k: torch.Tensor, v: torch.Tensor)

lit_gpt/model.py:334

Methodfrom_checkpoint

Automatically load `lit_config.json` and if it doesn't exist - a matching config from `lit_gpt/config.py`.

lit_gpt/config.py:120

Methodfrom_name

(cls, name: str, **kwargs: Any)

lit_gpt/adapter.py:75

Methodfrom_name

(cls, name: str, **kwargs: Any)

lit_gpt/adapter_v2.py:81

Methodfrom_name

(cls, name: str, **kwargs: Any)

lit_gpt/model.py:93

Functionlora_filter

(key: str, value: Any)

lit_gpt/lora.py:422

Functionmain

Main function for finetuning the model on the entity extraction task using LoRA Args: fabric: Lightning Fabric object data_d

finetune/lora.py:116

Functionmain

Finetune the adapter v2 model on the entity extraction task. Args: fabric (Fabric): Fabric object. data_dir (Path): Path to

finetune/adapter_v2.py:93

Functionmain

Generates text samples based on a pre-trained model and tokenizer. Args: prompt: The prompt string to use for generating the samples.

generate/base.py:103

Functionmark_only_adapter_as_trainable

Sets `requires_grad=False` for all non-adapter weights.

lit_gpt/adapter.py:158

Merges the LoRA weights into the full-rank weights (W = W + delta_W).

lit_gpt/lora.py:333

Methodmlp_class

(self)

lit_gpt/config.py:129

Prepare custom dataset for medical entity extraction. The output is a training and test dataset saved as `train.pt` and `test.pt`, which stor

scripts/prepare_entity_extraction_data.py:25

Methodreset_parameters

(self)

lit_gpt/rmsnorm.py:25

Methodreset_parameters

(self)

lit_gpt/model.py:343

Methodscaled_dot_product_attention

( self, q: torch.Tensor, k: torch.Tensor, v: torch.Tensor, mask: Optional[torch.Tensor] = None )

lit_gpt/adapter.py:116

Functionscrape_website

Scrapes the drugs.com website for the given drug name and saves the extracted text to a file. Args: args (tuple): Tuple containing t

scripts/scrape_drugs_data.py:40

Functionsetup

This script is used to finetune the model on the entity extraction task using LoRA Args: data_dir: Path to the data directory contai

finetune/lora.py:64

Functionsetup

Finetune the adapter v2 model on the entity extraction task. Args: data_dir (Path): Path to the directory containing the dataset.

finetune/adapter_v2.py:55

Methodvocab_size

(self)

lit_gpt/tokenizer.py:51

Methodwrite_reminder

(self)

lit_gpt/packed_dataset.py:117

← previous101–184 of 184, ranked by callers