MCPcopy Create free account

hub / github.com/Fugaku-LLM/DeepSpeedFugaku / functions

Functions1,158 in github.com/Fugaku-LLM/DeepSpeedFugaku

⨍ Functions 1,158◇ Types & classes 129
↓ 2 callersMethoddecode
(self, ids)
megatron/tokenizer/tokenizer.py:196
↓ 2 callersFunctionensure_divisibility
Ensure that numerator is divisible by the denominator.
megatron/mpu/utils.py:20
↓ 2 callersMethodevaluate
(self, qa_data, split)
tasks/orqa/evaluate_utils.py:118
↓ 2 callersFunctionfinetune
Main finetune function used across all tasks.
tasks/vision/finetune_utils.py:222
↓ 2 callersMethodforward
Run a forward pass for each of the models and return the respective embeddings.
megatron/model/realm_model.py:66
↓ 2 callersMethodforward
(self, hidden_states)
megatron/model/transformer.py:101
↓ 2 callersFunctionforward_step
(model, tokens, position_ids, attention_mask, tokentype_ids, layer_past=None, get_key_value=N
megatron/text_generation_utils.py:443
↓ 2 callersFunctionforward_step_helper
Helper method to run forward step with model split into chunks (run set_virtual_pipeline_model_parallel_rank() before calling forward_
megatron/schedules.py:255
↓ 2 callersFunctionfree_param
Free underlying storage of a parameter.
DeepSpeed/deepspeed/runtime/zero/partition_parameters.py:231
↓ 2 callersFunctiongather_RACE_results
(result_path, task)
examples/data_efficiency/bert/finetune/ds_finetune_gather_result.py:73
↓ 2 callersFunctiongenerate_result_table
(tab_header, configs, task_order, caption, avg_range, avg_tag, avg_only=False, fontsize="\\footnotesize",
examples/data_efficiency/gpt/eval/ds_evalharness_gather_result.py:167
↓ 2 callersFunctiongenerate_samples_eval
(model, context, max_gen_length, eos_token_id)
megatron/text_generation_utils.py:196
↓ 2 callersFunctionget_adlr_autoresume
ADLR autoresume object. It can be None so no need to check if it is initialized.
megatron/global_vars.py:86
↓ 2 callersFunctionget_all_subclasses
(cls)
DeepSpeed/deepspeed/runtime/zero/partition_parameters.py:217
↓ 2 callersMethodget_args
(self)
tools/convert_checkpoint/deepspeed_checkpoint.py:86
↓ 2 callersFunctionget_data_parallel_group
Get the data parallel group the caller rank belongs to.
megatron/mpu/initialize.py:202
↓ 2 callersFunctionget_datasets_weights_and_num_samples
(data_prefix, train_valid_test_num_samples)
megatron/data/dataset_utils.py:45
↓ 2 callersMethodget_embedding_state
(self, tp_index: int)
tools/convert_checkpoint/deepspeed_checkpoint.py:80
↓ 2 callersMethodget_final_norm_state
(self, tp_index:int)
tools/convert_checkpoint/deepspeed_checkpoint.py:104
↓ 2 callersFunctionget_group_world_size_rank
()
pretrain_ict.py:43
↓ 2 callersMethodget_loss_scale
FP32 optimizer does not do any scaling.
megatron/optimizer/optimizer.py:499
↓ 2 callersFunctionget_metric_function
(metric_name)
examples/data_efficiency/analyze_data.py:145
↓ 2 callersFunctionget_metric_type
(metric_name)
examples/data_efficiency/analyze_data.py:155
↓ 2 callersFunctionget_open_retrieval_wiki_dataset
()
megatron/data/orqa_wiki_dataset.py:28
↓ 2 callersFunctionget_pairs
Return set of symbol pairs in a word. Word is represented as tuple of symbols (symbols being variable-length strings).
megatron/tokenizer/gpt2_tokenization.py:79
↓ 2 callersMethodget_parameters
(self)
megatron/optimizer/optimizer.py:86
↓ 2 callersFunctionget_parameters_in_billions
(model)
megatron/utils.py:237
↓ 2 callersFunctionget_pipeline_model_parallel_group
Get the pipeline model parallel group the caller rank belongs to.
megatron/mpu/initialize.py:195
↓ 2 callersFunctionget_samples_mapping
Get a list that maps a sample index to a starting sentence index, end sentence index, and length
megatron/data/dataset_utils.py:634
↓ 2 callersMethodget_start_end_idx
(self)
megatron/data/data_samplers.py:85
↓ 2 callersFunctionget_target_sample_len
megatron/data/helpers.cpp:188
↓ 2 callersFunctionget_tensor_model_parallel_src_rank
Calculate the global rank corresponding to the first local rank in the tensor model parallel group.
megatron/mpu/initialize.py:322
↓ 2 callersFunctionget_train_valid_test_split_
Get dataset splits from comma or '/' separated string list.
megatron/data/dataset_utils.py:607
↓ 2 callersMethodget_transformer_state
(self, tp_index: int, pp_index: int)
tools/convert_checkpoint/deepspeed_checkpoint.py:94
↓ 2 callersFunctionget_virtual_pipeline_model_parallel_rank
Return the virtual pipeline-parallel rank.
megatron/mpu/initialize.py:304
↓ 2 callersFunctionget_virtual_pipeline_model_parallel_world_size
Return the virtual pipeline-parallel world size.
megatron/mpu/initialize.py:316
↓ 2 callersFunctionget_wandb_writer
Return wandb writer. It can be None so no need to check if it is initialized.
megatron/global_vars.py:78
↓ 2 callersFunctionget_words
(text)
tools/openwebtext/filter_ngrams.py:33
↓ 2 callersMethodinitialize_word_embeddings
(self, init_method_normal)
megatron/model/module.py:65
↓ 2 callersFunctioninsert_dict
(words, ngrams, pos)
tools/openwebtext/filter_ngrams.py:187
↓ 2 callersFunctionis_rank_0
Check whether it is rank 0. For AML, check if it is rank 0 of a node
megatron/__init__.py:65
↓ 2 callersFunctionis_start_piece
Check if the current word piece is the starting piece (BERT).
megatron/data/dataset_utils.py:173
↓ 2 callersFunctionload_biencoder_checkpoint
selectively load retrieval models for indexing/retrieving from saved checkpoints
megatron/checkpointing.py:458
↓ 2 callersMethodload_from_file
Populate members from instance saved to file
megatron/data/realm_index.py:50
↓ 2 callersFunctionlog2_ceil
megatron/fused_kernels/scaled_upper_triang_masked_softmax.h:65
↓ 2 callersFunctionlog2_ceil
megatron/fused_kernels/scaled_masked_softmax.h:50
↓ 2 callersFunctionmake_attention_mask
Returns a 2-dimensional (2-D) attention mask :param source_block: 1-D array :param target_block: 1-D array
megatron/data/ict_dataset.py:12
↓ 2 callersFunctionmake_attention_mask
Returns a 2-dimensional (2-D) attention mask :param source_block: 1-D array :param target_block: 1-D array
megatron/data/biencoder_dataset_utils.py:12
↓ 2 callersMethodmerge_file_
(self, another_file)
megatron/data/indexed_dataset.py:299
↓ 2 callersFunctionmerge_partitions
(merged, partitions, partition_dim, stride)
tools/merge_mp_partitions.py:55
↓ 2 callersFunctionparallel_self_attention
(tensor_model_parallel_size, num_att_heads_per_partition, hidden_size_per_att_head
megatron/mpu/tests/test_layers.py:328
↓ 2 callersFunctionparallel_transformer
(tensor_model_parallel_size, num_att_heads_per_partition, hidden_size_per_att_head, b
megatron/mpu/tests/test_layers.py:413
↓ 2 callersFunctionparse_arguments
()
tools/convert_checkpoint/deepspeed_to_megatron.py:21
↓ 2 callersFunctionprint_progress
(prefix, start_time, urls_counter, domain_blacklist_counter, extention_b
tools/openwebtext/blacklist_urls.py:233
↓ 2 callersFunctionprint_progress
(prefix, start_time, num_docs, num_fixed_text, num_non_english_docs, chars_non_english_docs
tools/openwebtext/cleanup_dataset.py:30
↓ 2 callersFunctionprocess_batch
Process batch and produce inputs for the model.
tasks/vision/finetune_utils.py:34
↓ 2 callersMethodread_data
(self, path)
megatron/data/indexed_dataset.py:160
↓ 2 callersFunctionrecv_forward
Receive tensor from previous rank in pipeline (forward receive).
megatron/p2p_communication.py:131
↓ 2 callersFunctionreduce_from_tensor_model_parallel_region
(input_)
megatron/mpu/mappings.py:148
↓ 2 callersMethodreload_model_params
(self)
megatron/optimizer/optimizer.py:548
↓ 2 callersFunctionround_sig
(x, sig=3)
examples/data_efficiency/gpt/eval/ds_evalharness_gather_result.py:162
↓ 2 callersFunctionsend_forward
Send tensor to next rank in pipeline (forward send).
megatron/p2p_communication.py:165
↓ 2 callersMethodset_doc_idx
(self, doc_idx_)
megatron/data/indexed_dataset.py:574
↓ 2 callersFunctionset_global_variables
Set args, tokenizer, tensorboard-writer, adlr-autoresume, and timers.
megatron/global_vars.py:98
↓ 2 callersFunctionsetup_model_and_optimizer
Setup model and optimizer.
megatron/training.py:493
↓ 2 callersFunctionshingles
(text, char_ngram=5)
tools/openwebtext/find_duplicates.py:30
↓ 2 callersMethodstate
(self)
megatron/data/realm_index.py:37
↓ 2 callersMethodstate_dict
(self)
megatron/learning_rates.py:130
↓ 2 callersMethodstate_dict_for_save_checkpoint
Save dict with state dicts of each of the models.
megatron/model/realm_model.py:90
↓ 2 callersMethodstate_dict_for_save_checkpoint
For easy load when model is combined with other heads, add an extra key.
megatron/model/t5_model.py:154
↓ 2 callersMethodstate_dict_for_save_checkpoint
Save dict with state dicts of each of the models.
megatron/model/biencoder_model.py:110
↓ 2 callersFunctiont5_position_ids
(token_ids)
megatron/model/t5_model.py:46
↓ 2 callersFunctiontrain_step
Single training step.
megatron/training.py:636
↓ 2 callersFunctiontrain_valid_test_datasets_provider_bert
Build train, valid, and test datasets.
examples/data_efficiency/analyze_data.py:86
↓ 2 callersFunctiontrain_valid_test_datasets_provider_gpt
Build train, valid, and test datasets.
examples/data_efficiency/analyze_data.py:67
↓ 2 callersFunctiontraining_log
Log training information such as losses, timing, ....
megatron/training.py:773
↓ 2 callersMethodupdate
(self, found_inf)
megatron/optimizer/grad_scaler.py:100
↓ 2 callersFunctionurl_pairs_to_remove
(args, bucket_urls, url_doc)
tools/openwebtext/find_duplicates.py:63
↓ 2 callersMethodvocab_size
(self)
megatron/tokenizer/tokenizer.py:85
↓ 2 callersFunctionwrite_remove_urls_list
(remove_urls_list, f_out)
tools/openwebtext/find_duplicates.py:99
↓ 2 callersMethodwriter
(cls, path, dtype)
megatron/data/indexed_dataset.py:380
↓ 1 callersFunctionWARP_SHFL_XOR_NATIVE
megatron/fused_kernels/scaled_upper_triang_masked_softmax.h:86
↓ 1 callersFunctionWARP_SHFL_XOR_NATIVE
megatron/fused_kernels/scaled_masked_softmax.h:71
↓ 1 callersFunction__best_fitting_dtype
(vocab_size=None)
megatron/data/indexed_dataset.py:29
↓ 1 callersMethod__init__
(self, num_classes, finetune=False)
megatron/model/vit_model.py:126
↓ 1 callersMethod__init__
(self, ict_head_size, num_tokentypes=1, parallel_output=Tru
megatron/model/realm_model.py:40
↓ 1 callersMethod__init__
(self, num_tokentypes=0, parallel_output=True)
megatron/model/t5_model.py:89
↓ 1 callersMethod__init__
(self, num_tokentypes=0, parallel_output=True, pre_process=
megatron/model/gpt_model.py:70
↓ 1 callersMethod__init__
(self, num_tokentypes=1, parallel_output=True, only_query_m
megatron/model/biencoder_model.py:46
↓ 1 callersMethod__init__
(self, module, args)
megatron/model/module.py:152
↓ 1 callersMethod__init__
(self, num_tokentypes=2, add_binary_head=True, parallel_out
megatron/model/bert_model.py:128
↓ 1 callersMethod__init__
(self, m, n)
megatron/mpu/tests/test_layers.py:177
↓ 1 callersMethod__init__
(self, optimizer, clip_grad, log_num_zeros_in_grad, params_have_main_grad)
megatron/optimizer/optimizer.py:482
↓ 1 callersMethod__init__
(self, task_name, dataset_name, datapath, tokenizer, max_seq_length)
tasks/orqa/natural_questions/nq.py:179
↓ 1 callersFunction_add_activation_checkpoint_args
(parser: argparse.ArgumentParser)
megatron/arguments.py:923
↓ 1 callersFunction_add_autoresume_args
(parser: argparse.ArgumentParser)
megatron/arguments.py:801
↓ 1 callersFunction_add_biencoder_args
(parser: argparse.ArgumentParser)
megatron/arguments.py:813
↓ 1 callersFunction_add_checkpointing_args
(parser: argparse.ArgumentParser)
megatron/arguments.py:598
↓ 1 callersFunction_add_data_args
(parser: argparse.ArgumentParser)
megatron/arguments.py:723
↓ 1 callersFunction_add_distillation_args
(parser: argparse.ArgumentParser)
megatron/arguments.py:941
← previousnext →201–300 of 1,158, ranked by callers