MCPcopy Create free account

hub / github.com/McGill-NLP/length-generalization / functions

Functions942 in github.com/McGill-NLP/length-generalization

↓ 117 callersMethodpop
(self, key: str, default: Any = DEFAULT, keep_as_dict: bool = False)
src/common/params.py:197
↓ 91 callersMethodget
Performs the functionality associated with dict.get(key) but also checks for returned dicts and returns a Params object in their plac
src/common/params.py:262
↓ 45 callersMethodlog
(self, obj)
src/analyzers/analyzer.py:44
↓ 35 callersFunctionnumber_to_string
Converts a number to a string such that digits are separated by spaces. Also, it handles both negative and positive numbers.
src/data/data_instance_processor/data_instance_processor.py:726
↓ 30 callersMethodget_dataset
( self, stage: Optional[ExperimentStage] = None, path: Optional[str] = None, f
src/data/base_dl_factory.py:152
↓ 30 callersMethodshuffle
Shuffles two sequences "seq1" and "seq2". In other words, given two sequences, returns an interleaved sequence of those strings, respe
scripts/dataset_builders/make_dyck_pred_dataset.py:120
↓ 29 callersMethodget_ds_file_path
( self, stage: ExperimentStage = None, path: str = None, no_exception: bool =
src/data/base_dl_factory.py:74
↓ 23 callersMethodformat
(self, record)
src/common/py_utils.py:17
↓ 23 callersMethodfrom_params
This is the automatic implementation of `from_params`. Any class that subclasses `FromParams` (or `Registrable`, which itself subclas
src/common/from_params.py:692
↓ 22 callersFunctionis_world_process_zero
()
src/common/torch_utils.py:39
↓ 22 callersMethodwrite
(self, data)
src/runtime/model_inference_shell.py:20
↓ 19 callersMethodsave
(self, obj, save_persistent_id=True)
src/common/from_params.py:109
↓ 15 callersMethod_load_last_checkpoint
(self, trainer: Seq2SeqTrainer)
src/runtime/seq2seq_runtime.py:511
↓ 15 callersMethodconstruct
Call the constructor to create an instance of `T`.
src/common/lazy.py:74
↓ 15 callersMethodfrom_split
(split: str)
src/common/__init__.py:24
↓ 14 callersFunctionset_seed
# Taken from https://huggingface.co/transformers/v3.0.2/_modules/transformers/trainer_utils.html Helper function for reproducible behavior to
scripts/dataset_builders/make_sum_dataset.py:11
↓ 13 callersMethodtokenize
(example: Dict[str, Any])
src/data/s2s_dl_factory.py:291
↓ 10 callersMethodcreate_trainer
(self, stage: ExperimentStage, **kwargs)
src/runtime/seq2seq_runtime.py:393
↓ 10 callersMethodget_collate_fn
(self, state: ExperimentStage)
src/data/base_dl_factory.py:118
↓ 9 callersMethod__init__
(self, config, has_relative_attention_bias=False)
src/models/custom_t5_decoder_only.py:537
↓ 9 callersFunctionget_cluster_name
(host: str)
notebooks/plotting/attention_analysis_utils.py:34
↓ 9 callersFunctionget_true_remote_path
(cluster: str, remote_path: str)
notebooks/plotting/attention_analysis_utils.py:26
↓ 8 callersMethodcreate_human_readable_scratchpad
( self, scratchpad_steps: List[UnifedScratchpadStep] )
src/data/data_instance_processor/data_instance_processor.py:656
↓ 7 callersMethodinit
(self, runtime, eval_dataset: Dataset, eval_split: str, **kwargs)
src/callbacks/base_callback.py:12
↓ 7 callersFunctionsave_and_make_executable
(job_path, script)
scripts/launch_experiment.py:320
↓ 6 callersMethod_load_best_checkpoint
(self, trainer)
src/runtime/seq2seq_runtime.py:522
↓ 6 callersFunctionchunks
(l, n)
src/common/py_utils.py:179
↓ 6 callersFunctionget_cluster_name
(host: str)
notebooks/plotting/attention_analysis.py:32
↓ 6 callersMethodset_tokenizer
(self, tokenizer)
src/data/base_dl_factory.py:71
↓ 6 callersFunctionuse_torch_distributed
(args: argparse.Namespace = None)
scripts/upload_experiment.py:42
↓ 5 callersFunction_construct_key
Returns the new_key if no previous key exists, otherwise concatenates previous key, separator, and new_key :param previous_key: :para
src/common/nest.py:32
↓ 5 callersFunctioncommand_to_bash_str
( cmd: str, configs_str: str, prefix: str = "", args: argparse.Namespace = None )
scripts/upload_experiment.py:56
↓ 5 callersFunctiongarbage_collection_cuda
Garbage collection Torch (CUDA) memory.
src/common/torch_utils.py:82
↓ 5 callersMethodgenerate
(self, current_size, max_size)
scripts/dataset_builders/make_dyck_pred_dataset2.py:35
↓ 5 callersFunctionget_entity_name
()
notebooks/plotting/utils.py:14
↓ 5 callersFunctionget_project_name
()
notebooks/plotting/utils.py:20
↓ 5 callersFunctionget_tempfile_path
()
scripts/launch_experiment.py:129
↓ 5 callersFunctionget_true_remote_path
(cluster: str, remote_path: str)
notebooks/plotting/attention_analysis.py:24
↓ 5 callersFunctionget_unique_tokens
()
scripts/dataset_builders/make_copy_dataset.py:39
↓ 5 callersMethodis_prediction_correct
( self, prediction: str, data_instance: Dict[str, Any] )
src/data/data_instance_processor/data_instance_processor.py:35
↓ 5 callersMethodresolve_class_name
Returns the subclass that corresponds to the given `name`, along with the name of the method that was registered as a constructor for
src/common/registrable.py:162
↓ 5 callersFunctionsave_and_make_executable
(job_path, script)
scripts/launcher.py:41
↓ 5 callersMethodtrain
(self, eval_split: str = "valid", train_split: str = "train")
src/runtime/seq2seq_runtime.py:549
↓ 4 callersMethod_create_answer
(self, example: Dict[str, Any])
src/data/data_instance_processor/sum_instance_processor.py:132
↓ 4 callersFunctionadd_python_paths
(script)
scripts/upload_experiment.py:122
↓ 4 callersFunctionapply_rotary_pos_emb
(x, sincos, offset=0)
src/models/custom_t5_decoder_only.py:71
↓ 4 callersFunctioncan_construct_from_params
(type_: Type)
src/common/from_params.py:193
↓ 4 callersMethodevaluate_scratchpad
( self, prediction: str, data_instance: Dict[str, Any], )
src/data/data_instance_processor/data_instance_processor.py:572
↓ 4 callersFunctionget_entity_name
()
src/common/wandb_utils.py:106
↓ 4 callersMethodget_last_checkpoint_path
(self)
src/runtime/seq2seq_runtime.py:345
↓ 4 callersMethodget_problem_type
(self)
src/data/base_dl_factory.py:149
↓ 4 callersFunctionget_project_name
()
src/common/wandb_utils.py:112
↓ 4 callersFunctionget_repo_dir
()
src/common/wandb_utils.py:118
↓ 4 callersFunctionis_oom_error
(exception: BaseException)
src/common/torch_utils.py:44
↓ 4 callersMethodlog_metrics_to_console
( self, split: str = "None", metrics: Dict[str, Any] = None )
src/runtime/seq2seq_runtime.py:493
↓ 4 callersFunctionmake_executable
(script_path)
scripts/launch_experiment.py:123
↓ 4 callersMethodpredict
( self, split: str = "test", enable_metrics: bool = False, load_best: bool = T
src/runtime/seq2seq_runtime.py:731
↓ 4 callersMethodregister
Register a class under a particular name. # Parameters name : `str` The name to register the class under.
src/common/registrable.py:61
↓ 4 callersFunctionrun_on_slurm
(args: argparse.Namespace)
scripts/launch_experiment.py:177
↓ 4 callersFunctionsubstr_to_token_ids
( substr: str, encoding: BatchEncoding, orig_seq: str )
src/tokenization_utils/__init__.py:23
↓ 4 callersFunctionunflatten
Creates a hierarchical dictionary from a flattened dictionary Assumes no lists are present :param flat_dict: a dictionary with no hierarc
src/common/nest.py:364
↓ 3 callersMethod__init__
Construct
src/common/plotnine_utils.py:96
↓ 3 callersMethod__init__
(self, num_submission_to_queue: int = 1, **kwargs)
scripts/launcher.py:545
↓ 3 callersFunction_all_color_pals
()
src/common/plotnine_utils.py:25
↓ 3 callersMethod_check_is_dict
(self, new_history, value)
src/common/params.py:418
↓ 3 callersMethod_collect_metrics
( self, create_compute_fn: ComputeFuncCreator, prefix: str )
src/analyzers/runtime_analyzer.py:63
↓ 3 callersMethod_create_answer
(self, example: Dict[str, Any])
src/data/data_instance_processor/sort_instance_processor.py:59
↓ 3 callersMethod_get_inputs
( batch_size: int, length: int, device: str, include_labels: bool = True )
src/analyzers/runtime_analyzer.py:257
↓ 3 callersMethodanalyze_all
(self, load_best: bool = True, split: str = "test")
src/runtime/seq2seq_runtime.py:1069
↓ 3 callersMethodas_dict
Sometimes we need to just represent the parameters as a dict, for instance when we pass them to PyTorch code. # Parameters
src/common/params.py:328
↓ 3 callersMethodcombine_pred
(self, split: str = "test", force: bool = False)
src/runtime/seq2seq_runtime.py:898
↓ 3 callersFunctioncreate_extras
Given a dictionary of extra arguments, returns a dictionary of kwargs that actually are a part of the signature of the cls.from_params (o
src/common/from_params.py:349
↓ 3 callersFunctioncreate_kwargs
Given some class, a `Params` object, and potentially other keyword arguments, create a dict of keyword args suitable for passing to the class
src/common/from_params.py:284
↓ 3 callersFunctioncreate_md5_hash
(inp: str)
scripts/launcher.py:17
↓ 3 callersFunctiondownload_trained_model
(args: argparse.Namespace)
scripts/launch_experiment.py:506
↓ 3 callersFunctionflatten
Flattens a dictionary with nested structure to a dictionary with no hierarchy Consider ignoring keys that you are not interested in to pr
src/common/nest.py:51
↓ 3 callersFunctionget_entity_name
()
scripts/fail_if_sweep_not_complete.py:12
↓ 3 callersFunctionget_exp_metadata
(exp: Path)
scripts/launch_experiment.py:383
↓ 3 callersFunctionget_human_readable_count
Abbreviates an integer number with K, M, B, T for thousands, millions, billions and trillions, respectively. Examples: >>> get_h
src/common/py_utils.py:139
↓ 3 callersFunctionget_run_name_from_config_obj
(config_obj: Dict[str, Any], sep: str = ".")
src/common/py_utils.py:213
↓ 3 callersMethodget_tokenization_info
( self, data_instance: Dict[str, Any], encoding: BatchEncoding, orig_seq: str )
src/data/data_instance_processor/data_instance_processor.py:41
↓ 3 callersFunctionget_unique_tokens
()
scripts/dataset_builders/make_reverse_dataset.py:37
↓ 3 callersMethodhandle_tokenizer
(self, tokenizer: Optional[Tokenizer] = None)
src/models/t5.py:33
↓ 3 callersFunctionis_hp_run_complete
(exp_dir: Path, metric_name)
scripts/manual_sweep.py:106
↓ 3 callersFunctioniter_sweep_params
(sweep_config)
scripts/manual_sweep.py:110
↓ 3 callersFunctionlist_metadata
(download_dir, run)
notebooks/plotting/attention_analysis_utils.py:301
↓ 3 callersFunctionmaybe_add_post_script
(args)
scripts/upload_experiment.py:33
↓ 3 callersFunctionmaybe_set_master_ip_and_address
(args: argparse.Namespace = None)
scripts/upload_experiment.py:50
↓ 3 callersFunctionreplace_env_vars
(target_str: str)
scripts/launch_experiment.py:499
↓ 3 callersFunctionreplace_env_vars
(target_str: str)
scripts/launcher.py:47
↓ 3 callersFunctionupload
(args)
scripts/launch_experiment.py:432
↓ 2 callersMethod__init__
( self, source_seq_key: Optional[str] = "source", target_seq_key: Optional[str] = "tar
src/data/data_instance_processor/data_instance_processor.py:14
↓ 2 callersMethod__init__
( self, include_scratchpad: Optional[bool] = None, **kwargs, )
src/data/data_instance_processor/sum_instance_processor.py:18
↓ 2 callersFunction_all_fill_pals
()
src/common/plotnine_utils.py:31
↓ 2 callersFunction_all_shape_pals
()
src/common/plotnine_utils.py:37
↓ 2 callersMethod_analyze_prediction
(self, predictions_path: Path)
src/analyzers/seq_cls_analyzer.py:80
↓ 2 callersMethod_analyze_prediction
(self, predictions_path: Path)
src/analyzers/seq2seq_analyzer.py:91
↓ 2 callersMethod_compute_metrics_during_training
(self, logits, labels)
src/trainers/trainer_with_metrics.py:63
↓ 2 callersMethod_create_answer
(self, example: Dict[str, Any])
src/data/data_instance_processor/lego_instance_processor.py:62
↓ 2 callersMethod_create_answer
(self, example: Dict[str, Any])
src/data/data_instance_processor/addition_instance_processor.py:56
next →1–100 of 942, ranked by callers