Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/McGill-NLP/length-generalization
/ functions
Functions
942 in github.com/McGill-NLP/length-generalization
⨍
Functions
942
◇
Types & classes
118
↓ 117 callers
Method
pop
(self, key: str, default: Any = DEFAULT, keep_as_dict: bool = False)
src/common/params.py:197
↓ 91 callers
Method
get
Performs the functionality associated with dict.get(key) but also checks for returned dicts and returns a Params object in their plac
src/common/params.py:262
↓ 45 callers
Method
log
(self, obj)
src/analyzers/analyzer.py:44
↓ 35 callers
Function
number_to_string
Converts a number to a string such that digits are separated by spaces. Also, it handles both negative and positive numbers.
src/data/data_instance_processor/data_instance_processor.py:726
↓ 30 callers
Method
get_dataset
( self, stage: Optional[ExperimentStage] = None, path: Optional[str] = None, f
src/data/base_dl_factory.py:152
↓ 30 callers
Method
shuffle
Shuffles two sequences "seq1" and "seq2". In other words, given two sequences, returns an interleaved sequence of those strings, respe
scripts/dataset_builders/make_dyck_pred_dataset.py:120
↓ 29 callers
Method
get_ds_file_path
( self, stage: ExperimentStage = None, path: str = None, no_exception: bool =
src/data/base_dl_factory.py:74
↓ 23 callers
Method
format
(self, record)
src/common/py_utils.py:17
↓ 23 callers
Method
from_params
This is the automatic implementation of `from_params`. Any class that subclasses `FromParams` (or `Registrable`, which itself subclas
src/common/from_params.py:692
↓ 22 callers
Function
is_world_process_zero
()
src/common/torch_utils.py:39
↓ 22 callers
Method
write
(self, data)
src/runtime/model_inference_shell.py:20
↓ 19 callers
Method
save
(self, obj, save_persistent_id=True)
src/common/from_params.py:109
↓ 15 callers
Method
_load_last_checkpoint
(self, trainer: Seq2SeqTrainer)
src/runtime/seq2seq_runtime.py:511
↓ 15 callers
Method
construct
Call the constructor to create an instance of `T`.
src/common/lazy.py:74
↓ 15 callers
Method
from_split
(split: str)
src/common/__init__.py:24
↓ 14 callers
Function
set_seed
# Taken from https://huggingface.co/transformers/v3.0.2/_modules/transformers/trainer_utils.html Helper function for reproducible behavior to
scripts/dataset_builders/make_sum_dataset.py:11
↓ 13 callers
Method
tokenize
(example: Dict[str, Any])
src/data/s2s_dl_factory.py:291
↓ 10 callers
Method
create_trainer
(self, stage: ExperimentStage, **kwargs)
src/runtime/seq2seq_runtime.py:393
↓ 10 callers
Method
get_collate_fn
(self, state: ExperimentStage)
src/data/base_dl_factory.py:118
↓ 9 callers
Method
__init__
(self, config, has_relative_attention_bias=False)
src/models/custom_t5_decoder_only.py:537
↓ 9 callers
Function
get_cluster_name
(host: str)
notebooks/plotting/attention_analysis_utils.py:34
↓ 9 callers
Function
get_true_remote_path
(cluster: str, remote_path: str)
notebooks/plotting/attention_analysis_utils.py:26
↓ 8 callers
Method
create_human_readable_scratchpad
( self, scratchpad_steps: List[UnifedScratchpadStep] )
src/data/data_instance_processor/data_instance_processor.py:656
↓ 7 callers
Method
init
(self, runtime, eval_dataset: Dataset, eval_split: str, **kwargs)
src/callbacks/base_callback.py:12
↓ 7 callers
Function
save_and_make_executable
(job_path, script)
scripts/launch_experiment.py:320
↓ 6 callers
Method
_load_best_checkpoint
(self, trainer)
src/runtime/seq2seq_runtime.py:522
↓ 6 callers
Function
chunks
(l, n)
src/common/py_utils.py:179
↓ 6 callers
Function
get_cluster_name
(host: str)
notebooks/plotting/attention_analysis.py:32
↓ 6 callers
Method
set_tokenizer
(self, tokenizer)
src/data/base_dl_factory.py:71
↓ 6 callers
Function
use_torch_distributed
(args: argparse.Namespace = None)
scripts/upload_experiment.py:42
↓ 5 callers
Function
_construct_key
Returns the new_key if no previous key exists, otherwise concatenates previous key, separator, and new_key :param previous_key: :para
src/common/nest.py:32
↓ 5 callers
Function
command_to_bash_str
( cmd: str, configs_str: str, prefix: str = "", args: argparse.Namespace = None )
scripts/upload_experiment.py:56
↓ 5 callers
Function
garbage_collection_cuda
Garbage collection Torch (CUDA) memory.
src/common/torch_utils.py:82
↓ 5 callers
Method
generate
(self, current_size, max_size)
scripts/dataset_builders/make_dyck_pred_dataset2.py:35
↓ 5 callers
Function
get_entity_name
()
notebooks/plotting/utils.py:14
↓ 5 callers
Function
get_project_name
()
notebooks/plotting/utils.py:20
↓ 5 callers
Function
get_tempfile_path
()
scripts/launch_experiment.py:129
↓ 5 callers
Function
get_true_remote_path
(cluster: str, remote_path: str)
notebooks/plotting/attention_analysis.py:24
↓ 5 callers
Function
get_unique_tokens
()
scripts/dataset_builders/make_copy_dataset.py:39
↓ 5 callers
Method
is_prediction_correct
( self, prediction: str, data_instance: Dict[str, Any] )
src/data/data_instance_processor/data_instance_processor.py:35
↓ 5 callers
Method
resolve_class_name
Returns the subclass that corresponds to the given `name`, along with the name of the method that was registered as a constructor for
src/common/registrable.py:162
↓ 5 callers
Function
save_and_make_executable
(job_path, script)
scripts/launcher.py:41
↓ 5 callers
Method
train
(self, eval_split: str = "valid", train_split: str = "train")
src/runtime/seq2seq_runtime.py:549
↓ 4 callers
Method
_create_answer
(self, example: Dict[str, Any])
src/data/data_instance_processor/sum_instance_processor.py:132
↓ 4 callers
Function
add_python_paths
(script)
scripts/upload_experiment.py:122
↓ 4 callers
Function
apply_rotary_pos_emb
(x, sincos, offset=0)
src/models/custom_t5_decoder_only.py:71
↓ 4 callers
Function
can_construct_from_params
(type_: Type)
src/common/from_params.py:193
↓ 4 callers
Method
evaluate_scratchpad
( self, prediction: str, data_instance: Dict[str, Any], )
src/data/data_instance_processor/data_instance_processor.py:572
↓ 4 callers
Function
get_entity_name
()
src/common/wandb_utils.py:106
↓ 4 callers
Method
get_last_checkpoint_path
(self)
src/runtime/seq2seq_runtime.py:345
↓ 4 callers
Method
get_problem_type
(self)
src/data/base_dl_factory.py:149
↓ 4 callers
Function
get_project_name
()
src/common/wandb_utils.py:112
↓ 4 callers
Function
get_repo_dir
()
src/common/wandb_utils.py:118
↓ 4 callers
Function
is_oom_error
(exception: BaseException)
src/common/torch_utils.py:44
↓ 4 callers
Method
log_metrics_to_console
( self, split: str = "None", metrics: Dict[str, Any] = None )
src/runtime/seq2seq_runtime.py:493
↓ 4 callers
Function
make_executable
(script_path)
scripts/launch_experiment.py:123
↓ 4 callers
Method
predict
( self, split: str = "test", enable_metrics: bool = False, load_best: bool = T
src/runtime/seq2seq_runtime.py:731
↓ 4 callers
Method
register
Register a class under a particular name. # Parameters name : `str` The name to register the class under.
src/common/registrable.py:61
↓ 4 callers
Function
run_on_slurm
(args: argparse.Namespace)
scripts/launch_experiment.py:177
↓ 4 callers
Function
substr_to_token_ids
( substr: str, encoding: BatchEncoding, orig_seq: str )
src/tokenization_utils/__init__.py:23
↓ 4 callers
Function
unflatten
Creates a hierarchical dictionary from a flattened dictionary Assumes no lists are present :param flat_dict: a dictionary with no hierarc
src/common/nest.py:364
↓ 3 callers
Method
__init__
Construct
src/common/plotnine_utils.py:96
↓ 3 callers
Method
__init__
(self, num_submission_to_queue: int = 1, **kwargs)
scripts/launcher.py:545
↓ 3 callers
Function
_all_color_pals
()
src/common/plotnine_utils.py:25
↓ 3 callers
Method
_check_is_dict
(self, new_history, value)
src/common/params.py:418
↓ 3 callers
Method
_collect_metrics
( self, create_compute_fn: ComputeFuncCreator, prefix: str )
src/analyzers/runtime_analyzer.py:63
↓ 3 callers
Method
_create_answer
(self, example: Dict[str, Any])
src/data/data_instance_processor/sort_instance_processor.py:59
↓ 3 callers
Method
_get_inputs
( batch_size: int, length: int, device: str, include_labels: bool = True )
src/analyzers/runtime_analyzer.py:257
↓ 3 callers
Method
analyze_all
(self, load_best: bool = True, split: str = "test")
src/runtime/seq2seq_runtime.py:1069
↓ 3 callers
Method
as_dict
Sometimes we need to just represent the parameters as a dict, for instance when we pass them to PyTorch code. # Parameters
src/common/params.py:328
↓ 3 callers
Method
combine_pred
(self, split: str = "test", force: bool = False)
src/runtime/seq2seq_runtime.py:898
↓ 3 callers
Function
create_extras
Given a dictionary of extra arguments, returns a dictionary of kwargs that actually are a part of the signature of the cls.from_params (o
src/common/from_params.py:349
↓ 3 callers
Function
create_kwargs
Given some class, a `Params` object, and potentially other keyword arguments, create a dict of keyword args suitable for passing to the class
src/common/from_params.py:284
↓ 3 callers
Function
create_md5_hash
(inp: str)
scripts/launcher.py:17
↓ 3 callers
Function
download_trained_model
(args: argparse.Namespace)
scripts/launch_experiment.py:506
↓ 3 callers
Function
flatten
Flattens a dictionary with nested structure to a dictionary with no hierarchy Consider ignoring keys that you are not interested in to pr
src/common/nest.py:51
↓ 3 callers
Function
get_entity_name
()
scripts/fail_if_sweep_not_complete.py:12
↓ 3 callers
Function
get_exp_metadata
(exp: Path)
scripts/launch_experiment.py:383
↓ 3 callers
Function
get_human_readable_count
Abbreviates an integer number with K, M, B, T for thousands, millions, billions and trillions, respectively. Examples: >>> get_h
src/common/py_utils.py:139
↓ 3 callers
Function
get_run_name_from_config_obj
(config_obj: Dict[str, Any], sep: str = ".")
src/common/py_utils.py:213
↓ 3 callers
Method
get_tokenization_info
( self, data_instance: Dict[str, Any], encoding: BatchEncoding, orig_seq: str )
src/data/data_instance_processor/data_instance_processor.py:41
↓ 3 callers
Function
get_unique_tokens
()
scripts/dataset_builders/make_reverse_dataset.py:37
↓ 3 callers
Method
handle_tokenizer
(self, tokenizer: Optional[Tokenizer] = None)
src/models/t5.py:33
↓ 3 callers
Function
is_hp_run_complete
(exp_dir: Path, metric_name)
scripts/manual_sweep.py:106
↓ 3 callers
Function
iter_sweep_params
(sweep_config)
scripts/manual_sweep.py:110
↓ 3 callers
Function
list_metadata
(download_dir, run)
notebooks/plotting/attention_analysis_utils.py:301
↓ 3 callers
Function
maybe_add_post_script
(args)
scripts/upload_experiment.py:33
↓ 3 callers
Function
maybe_set_master_ip_and_address
(args: argparse.Namespace = None)
scripts/upload_experiment.py:50
↓ 3 callers
Function
replace_env_vars
(target_str: str)
scripts/launch_experiment.py:499
↓ 3 callers
Function
replace_env_vars
(target_str: str)
scripts/launcher.py:47
↓ 3 callers
Function
upload
(args)
scripts/launch_experiment.py:432
↓ 2 callers
Method
__init__
( self, source_seq_key: Optional[str] = "source", target_seq_key: Optional[str] = "tar
src/data/data_instance_processor/data_instance_processor.py:14
↓ 2 callers
Method
__init__
( self, include_scratchpad: Optional[bool] = None, **kwargs, )
src/data/data_instance_processor/sum_instance_processor.py:18
↓ 2 callers
Function
_all_fill_pals
()
src/common/plotnine_utils.py:31
↓ 2 callers
Function
_all_shape_pals
()
src/common/plotnine_utils.py:37
↓ 2 callers
Method
_analyze_prediction
(self, predictions_path: Path)
src/analyzers/seq_cls_analyzer.py:80
↓ 2 callers
Method
_analyze_prediction
(self, predictions_path: Path)
src/analyzers/seq2seq_analyzer.py:91
↓ 2 callers
Method
_compute_metrics_during_training
(self, logits, labels)
src/trainers/trainer_with_metrics.py:63
↓ 2 callers
Method
_create_answer
(self, example: Dict[str, Any])
src/data/data_instance_processor/lego_instance_processor.py:62
↓ 2 callers
Method
_create_answer
(self, example: Dict[str, Any])
src/data/data_instance_processor/addition_instance_processor.py:56
next →
1–100 of 942, ranked by callers