Functions — github.com/GCYZSL/MoLA

CodeHub MCPcopy Create free account

hub / github.com/GCYZSL/MoLA / functions

Functions226 in github.com/GCYZSL/MoLA

⨍ Functions 226◇ Types & classes 26 ↳ Endpoints 1

↓ 15 callersMethodget_prompt

Returns the virtual prompts to use for Peft. Only applicable when `peft_config.peft_type != PeftType.LORA`.

src/mola_peft_model_hacked.py:418

↓ 14 callersMethodfrom_pretrained

r""" Instantiate a [`LoraModel`] from a pretrained Lora configuration and weights. Args: model ([`~transformers.PreTraine

src/mola_peft_model_hacked.py:283

↓ 9 callersMethodsave_pretrained

r""" This function saves the adapter model and the adapter configuration files to a directory, so that it can be reloaded using the [`

src/mola_peft_model_hacked.py:237

↓ 8 callersMethod__init__

( self, hidden_size: int, intermediate_size: int, hidden_act: str, con

src/mola_modeling_llama_hacked.py:232

↓ 8 callersMethod_gather_and_numpify

Gather value of `tensors` (tensor or list/tuple of nested tensors) and convert them to numpy before concatenating them to `gathered`

src/mola_trainer_hacked.py:3779

↓ 7 callersMethodis_world_process_zero

Whether or not this process is the global main process (when training in a distributed fashion on several machines, this is only goin

src/mola_trainer_hacked.py:2720

↓ 6 callersMethod_nested_gather

Gather value of `tensors` (tensor or list/tuple of nested tensors) and convert them to numpy before concatenating them to `gathered`

src/mola_trainer_hacked.py:3253

↓ 5 callersMethod__init__

(self, model: PreTrainedModel, peft_config: PeftConfig, adapter_name: str = "default", number

src/mola_peft_model_hacked.py:212

↓ 5 callersMethodnum_examples

Helper to get number of samples in a [`~torch.utils.data.DataLoader`] by accessing its dataset. When dataloader.dataset does not exis

src/mola_trainer_hacked.py:1200

↓ 4 callersMethod_issue_warnings_after_load

(self, load_result)

src/mola_trainer_hacked.py:2202

↓ 4 callersMethod_save

(self, output_dir: Optional[str] = None, state_dict=None)

src/mola_trainer_hacked.py:2824

↓ 4 callersMethodgenerate

(self, **kwargs)

src/mola_peft_model_hacked.py:1029

↓ 4 callersMethodtrain

Main training entry point. Args: resume_from_checkpoint (`str` or `bool`, *optional*): If a `str`, local

src/mola_trainer_hacked.py:1555

↓ 3 callersMethod_get_collator_with_removed_columns

Wrap the data collator in a callable removing unused columns.

src/mola_trainer_hacked.py:752

↓ 3 callersMethod_load_rng_state

(self, checkpoint)

src/mola_trainer_hacked.py:2254

↓ 3 callersMethod_move_model_to_device

(self, model, device)

src/mola_trainer_hacked.py:712

↓ 3 callersMethod_pad_across_processes

Recursively pad the tensors in a nested list/tuple/dictionary of tensors from all devices to the same size so they can safely be gath

src/mola_trainer_hacked.py:3271

↓ 3 callersMethod_prepare_input

Prepares one `data` before feeding it to the model, be it a tensor or a nested list/dictionary of tensors.

src/mola_trainer_hacked.py:2574

↓ 3 callersMethod_prepare_inputs

Prepare `inputs` before feeding them to the model, converting them to tensors if they are not already and handling potential state.

src/mola_trainer_hacked.py:2592

↓ 3 callersMethod_remove_unused_columns

(self, dataset: "datasets.Dataset", description: Optional[str] = None)

src/mola_trainer_hacked.py:726

↓ 3 callersMethod_wrap_model

(self, model, training=True, dataloader=None)

src/mola_trainer_hacked.py:1361

↓ 3 callersMethodcompute_loss_context_manager

A helper wrapper to group together context managers.

src/mola_trainer_hacked.py:2608

↓ 3 callersMethodget_new_parameters

(self, number_experts, top_k, oblance)

src/mola_modeling_llama_hacked.py:1158

↓ 3 callersMethodis_local_process_zero

Whether or not this process is the local (e.g., on one machine if training in a distributed fashion on several machines) main process

src/mola_trainer_hacked.py:2713

↓ 3 callersMethodlog

Log `logs` on the various objects watching training. Subclass and override this method to inject custom behavior. Args:

src/mola_trainer_hacked.py:2557

↓ 3 callersMethodreset_lora_parameters

(self, adapter_name)

src/mola_lora_hacked.py:631

↓ 3 callersMethodreset_lora_parameters

(self, adapter_name)

src/mola_lora_hacked.py:752

↓ 3 callersMethodsave_model

Will save the model, so you can reload it using `from_pretrained()`. Will only save from the main process.

src/mola_trainer_hacked.py:2732

↓ 3 callersFunctionset_peft_model_state_dict_moe

Set the state dict of the Peft model. Args: model ([`PeftModel`]): The Peft model. peft_model_state_dict (`dict`): The state

src/mola_peft_model_hacked.py:132

↓ 3 callersMethodstore_flos

src/mola_trainer_hacked.py:2849

↓ 2 callersMethod__init__

(self, model, config, adapter_name, number_experts, top_k)

src/mola_lora_hacked.py:155

↓ 2 callersMethod_find_and_replace

(self, adapter_name)

src/mola_lora_hacked.py:292

↓ 2 callersMethod_get_eval_sampler

(self, eval_dataset: Dataset)

src/mola_trainer_hacked.py:891

↓ 2 callersMethod_get_output_dir

src/mola_trainer_hacked.py:2064

↓ 2 callersMethod_load_from_checkpoint

(self, resume_from_checkpoint, model=None)

src/mola_trainer_hacked.py:2084

↓ 2 callersMethod_load_optimizer_and_scheduler

If optimizer and scheduler states exist, load them.

src/mola_trainer_hacked.py:2401

↓ 2 callersMethod_maybe_log_save_evaluate

(self, tr_loss, model, trial, epoch, ignore_keys_for_eval)

src/mola_trainer_hacked.py:2215

↓ 2 callersMethod_replace_module

(self, parent_module, child_name, new_module, old_module, layer_index)

src/mola_lora_hacked.py:332

↓ 2 callersMethod_set_adapter_layers

(self, enabled=True)

src/mola_lora_hacked.py:369

↓ 2 callersMethod_set_signature_columns_if_needed

src/mola_trainer_hacked.py:718

↓ 2 callersMethod_sorted_checkpoints

( self, output_dir=None, checkpoint_prefix=PREFIX_CHECKPOINT_DIR, use_mtime=False )

src/mola_trainer_hacked.py:2860

↓ 2 callersMethodadd_adapter

(self, adapter_name: str, peft_config: PeftConfig)

src/mola_peft_model_hacked.py:517

↓ 2 callersMethodautocast_smart_context_manager

A helper wrapper that creates an appropriate context manager for `autocast` while feeding it the desired arguments, depending on the

src/mola_trainer_hacked.py:2614

↓ 2 callersMethodcall_model_init

(self, trial=None)

src/mola_trainer_hacked.py:1286

↓ 2 callersMethodcompute_loss

How the loss is computed by Trainer. By default, all models return the loss in the first element. Subclass and override for custom b

src/mola_trainer_hacked.py:2681

↓ 2 callersMethodcreate_optimizer_and_scheduler

Setup the optimizer and the learning rate scheduler. We provide a reasonable default that works well. If you want to use something e

src/mola_trainer_hacked.py:1018

↓ 2 callersFunctiondetect_nan_inf

src/mola_modeling_llama_hacked.py:126

↓ 2 callersMethodevaluate

Run evaluation and returns metrics. The calling script will be responsible for providing a method to compute metrics, as they are ta

src/mola_trainer_hacked.py:2909

↓ 2 callersMethodgenerate_prompt

( self, instruction: str, input: Union[None, str] = None, label: Union[None, s

utils/prompter.py:28

↓ 2 callersMethodgenerate_prompt_meta

( self, instruction: str, input: Union[None, str] = None, label: Union[None, s

utils/prompter.py:50

↓ 2 callersMethodget_base_model

Returns the base model.

src/mola_peft_model_hacked.py:511

↓ 2 callersMethodget_input_embeddings

src/mola_modeling_llama_hacked.py:684

↓ 2 callersFunctionget_peft_model

Returns a Peft model object from a model and a config. Args: model ([`transformers.PreTrainedModel`]): Model to be wrapped.

src/mola_mapping_hacked.py:114

↓ 2 callersMethodinit_git_repo

Initializes a git repo in `self.args.hub_model_id`. Args: at_init (`bool`, *optional*, defaults to `False`):

src/mola_trainer_hacked.py:3428

↓ 2 callersFunctionmark_only_lora_as_trainable

(model: nn.Module, bias: str = "none")

src/mola_lora_hacked.py:507

↓ 2 callersMethodmerge

src/mola_lora_hacked.py:799

↓ 2 callersMethodprediction_step

Perform an evaluation step on `model` using `inputs`. Subclass and override to inject custom behavior. Args: mo

src/mola_trainer_hacked.py:3305

↓ 2 callersMethodpush_to_hub

Upload *self.model* and *self.tokenizer* to the 🤗 model hub on the repo *self.args.hub_model_id*. Parameters: commit_mes

src/mola_trainer_hacked.py:3573

↓ 2 callersFunctionrotate_half

Rotates half the hidden dims of the input.

src/mola_modeling_llama_hacked.py:216

↓ 2 callersMethodset_additional_trainable_modules

(self, peft_config, adapter_name)

src/mola_peft_model_hacked.py:531

↓ 2 callersFunctiontokenize

(prompt, add_eos_token=True)

mola_training.py:153

↓ 2 callersFunctiontokenize

(prompt, add_eos_token=True)

mola_training_instruction.py:148

↓ 2 callersMethodtraining_step

Perform a training step on a batch of inputs. Subclass and override to inject custom behavior. Args: model (`nn

src/mola_trainer_hacked.py:2633

↓ 2 callersMethodunmerge

src/mola_lora_hacked.py:815

↓ 2 callersMethodupdate_layer

(self, adapter_name, r, lora_alpha, lora_dropout, init_lora_weights, layer_index)

src/mola_lora_hacked.py:561

↓ 1 callersMethod_add_sm_patterns_to_gitignore

Add SageMaker Checkpointing patterns to .gitignore file.

src/mola_trainer_hacked.py:3795

↓ 1 callersMethod_check_quantization_dependency

src/mola_lora_hacked.py:181

↓ 1 callersMethod_check_target_module_exists

(self, lora_config, key)

src/mola_lora_hacked.py:190

↓ 1 callersMethod_create_new_module

(self, lora_config, adapter_name, target, layer_index, number_experts, top_k)

src/mola_lora_hacked.py:216

↓ 1 callersFunction_expand_mask

Expands attention_mask from `[bsz, seq_len]` to `[bsz, 1, tgt_seq_len, src_seq_len]`.

src/mola_modeling_llama_hacked.py:150

↓ 1 callersMethod_get_train_sampler

src/mola_trainer_hacked.py:770

↓ 1 callersMethod_hp_search_setup

HP search setup code

src/mola_trainer_hacked.py:1214

↓ 1 callersMethod_load_best_model

src/mola_trainer_hacked.py:2142

↓ 1 callersFunction_make_causal_mask

Make causal mask used for bi-directional self-attention.

src/mola_modeling_llama_hacked.py:135

↓ 1 callersMethod_prefix_tuning_forward

( self, input_ids=None, attention_mask=None, inputs_embeds=None, label

src/mola_peft_model_hacked.py:839

↓ 1 callersMethod_prefix_tuning_forward

( self, input_ids=None, attention_mask=None, inputs_embeds=None, label

src/mola_peft_model_hacked.py:1427

↓ 1 callersMethod_prefix_tuning_forward

( self, input_ids=None, attention_mask=None, inputs_embeds=None, start

src/mola_peft_model_hacked.py:1602

↓ 1 callersMethod_prepare_decoder_attention_mask

(self, attention_mask, input_shape, inputs_embeds, past_key_values_length)

src/mola_modeling_llama_hacked.py:691

↓ 1 callersMethod_prepare_lora_config

(peft_config, model_config)

src/mola_lora_hacked.py:399

↓ 1 callersMethod_prepare_model_for_gradient_checkpointing

r""" Prepares the model for gradient checkpointing if necessary

src/mola_peft_model_hacked.py:389

↓ 1 callersFunction_prepare_prompt_learning_config

(peft_config: PeftConfig, model_config: Dict[str, Any])

src/mola_mapping_hacked.py:72

↓ 1 callersMethod_push_from_checkpoint

(self, checkpoint_folder)

src/mola_trainer_hacked.py:3531

↓ 1 callersMethod_report_to_hp_search

(self, trial: Union["optuna.Trial", Dict[str, Any]], step: int, metrics: Dict[str, float])

src/mola_trainer_hacked.py:1255

↓ 1 callersMethod_rotate_checkpoints

(self, use_mtime=False, output_dir=None)

src/mola_trainer_hacked.py:2884

↓ 1 callersMethod_save_checkpoint

(self, model, trial, metrics=None)

src/mola_trainer_hacked.py:2295

↓ 1 callersMethod_save_tpu

(self, output_dir: Optional[str] = None)

src/mola_trainer_hacked.py:2796

↓ 1 callersMethod_setup_prompt_encoder

(self, adapter_name: str)

src/mola_peft_model_hacked.py:354

↓ 1 callersMethod_split_kwargs

(cls, kwargs: Dict[str, Any])

src/mola_peft_model_hacked.py:540

↓ 1 callersMethod_tune_save_checkpoint

src/mola_trainer_hacked.py:1273

↓ 1 callersMethodadd_adapter

(self, adapter_name, number_experts, top_k, config=None)

src/mola_lora_hacked.py:164

↓ 1 callersMethodadd_callback

Add a callback to the current list of [`~transformer.TrainerCallback`]. Args: callback (`type` or [`~transformer.TrainerC

src/mola_trainer_hacked.py:674

↓ 1 callersFunctionapply_rotary_pos_emb

(q, k, cos, sin, offset: int = 0)

src/mola_modeling_llama_hacked.py:223

↓ 1 callersMethodcreate_model_card

Creates a draft of a model card using the information available to the `Trainer`. Args: language (`str`, *optional*):

src/mola_trainer_hacked.py:3475

↓ 1 callersFunctioncreate_one_example

(format, question, context, choice, answer, lecture, solution, test_example=True)

preparation_scienceqa_data.py:45

↓ 1 callersMethodcreate_optimizer

Setup the optimizer. We provide a reasonable default that works well. If you want to use something else, you can pass a tuple in the

src/mola_trainer_hacked.py:1034

↓ 1 callersMethodcreate_or_update_model_card

Updates or create model card to include information about peft: 1. Adds `peft` library tag 2. Adds peft version 3. Ad

src/mola_peft_model_hacked.py:682

↓ 1 callersMethodcreate_scheduler

Setup the scheduler. The optimizer of the trainer must have been set up either before this method is called or passed as an argument.

src/mola_trainer_hacked.py:1183

↓ 1 callersMethoddisable_adapter_layers

src/mola_lora_hacked.py:377

↓ 1 callersMethodenable_adapter_layers

src/mola_lora_hacked.py:374

↓ 1 callersMethodfloating_point_ops

For models that inherit from [`PreTrainedModel`], uses that method to compute the number of floating point operations for every backw

src/mola_trainer_hacked.py:3410

next →1–100 of 226, ranked by callers