MCPcopy Create free account

hub / github.com/AIS-SNU/Smart-Infinity / functions

Functions4,989 in github.com/AIS-SNU/Smart-Infinity

⨍ Functions 4,989◇ Types & classes 624
↓ 5 callersMethod_start_timers
(self, timer_names)
deepspeed/runtime/engine.py:2128
↓ 5 callersMethod_stop_timers
(self, timer_names)
deepspeed/runtime/engine.py:2132
↓ 5 callersMethodadd
Allocate a chunk of memory from the buffer to tensor and copy the values.
DeepSpeedExample/megatron/memory.py:89
↓ 5 callersMethodadd
Track the rng state.
DeepSpeedExample/megatron/core/tensor_parallel/random.py:131
↓ 5 callersFunctionbackward_step
Backward step through passed-in output tensor. If last stage, output_tensor_grad is None, otherwise gradient of loss with respect to stage's
DeepSpeedExample/megatron/core/pipeline_parallel/schedules.py:231
↓ 5 callersFunctionbuild_tokenizer
Initialize tokenizer.
DeepSpeedExample/megatron/tokenizer/tokenizer.py:25
↓ 5 callersMethodbyte2readable
(self, num)
setup/nvme_speed_test.py:70
↓ 5 callersFunctioncached_path
Given something that might be a URL (or might be a local path), determine which. If it's a URL, download the file and cache it, and retur
DeepSpeedExample/megatron/deprecated_data_utils/file_utils.py:87
↓ 5 callersMethodcheck_and_propagate_first_head_layout
If all heads require same sparsity layout, it propagate first head layout to all heads Arguments: layout: required: a tensor of
deepspeed/ops/sparse_attention/sparsity_config.py:48
↓ 5 callersMethodcommunication_backend_name
(self)
deepspeed/accelerator/cpu_accelerator.py:176
↓ 5 callersMethodcreate_job
(self, rnd_type, queue_size, thread_num)
setup/nvme_speed_test.py:83
↓ 5 callersFunctiondebug_param2name
(param)
deepspeed/utils/debug.py:38
↓ 5 callersFunctiondebug_rank0
(message: str)
deepspeed/runtime/zero/partitioned_param_coordinator.py:20
↓ 5 callersMethoddecode
(self, tokens)
DeepSpeedExample/megatron/tokenizer/gpt2_tokenization.py:283
↓ 5 callersFunctiondetach_variable
(inputs, device=None)
deepspeed/runtime/activation_checkpointing/checkpointing.py:65
↓ 5 callersMethoddetokenize
(self, token_ids)
DeepSpeedExample/megatron/tokenizer/tokenizer.py:97
↓ 5 callersFunctiondivide
Ensure that numerator is divisible by the denominator and return the division value.
DeepSpeedExample/megatron/mpu/utils.py:26
↓ 5 callersFunctiondo_aio_operation_overlap
deepspeed/ops/csrc/aio/common/deepspeed_aio_common.cpp:190
↓ 5 callersFunctiondo_aio_operation_sequential
deepspeed/ops/csrc/aio/common/deepspeed_aio_common.cpp:126
↓ 5 callersMethodelasticity_enabled
(self)
deepspeed/runtime/engine.py:470
↓ 5 callersFunctionenable_grad_sync
Enable asynchronous grad reductions
DeepSpeedExample/megatron/core/pipeline_parallel/schedules.py:411
↓ 5 callersFunctionflops_to_string
(flops, units=None, precision=2)
deepspeed/profiling/flops_profiler/profiler.py:1028
↓ 5 callersMethodfork
Fork the cuda rng state, perform operations, and exit with the original state.
DeepSpeedExample/megatron/mpu/random.py:178
↓ 5 callersFunctionforward_step
Forward step for passed-in model. If first stage, input tensor is obtained from data_iterator, otherwise passed-in input_tensor is used.
DeepSpeedExample/megatron/core/pipeline_parallel/schedules.py:166
↓ 5 callersFunctionget_bias_dropout_add
(training)
DeepSpeedExample/megatron/model/transformer.py:428
↓ 5 callersFunctionget_checkpoint_tracker_filename
Tracker file rescords the latest chckpoint during training to restart from.
DeepSpeedExample/megatron/checkpointing.py:86
↓ 5 callersFunctionget_cuda_rng_tracker
Get cuda rng tracker.
DeepSpeedExample/megatron/core/tensor_parallel/random.py:173
↓ 5 callersFunctionget_cuda_rng_tracker
Get cuda rng tracker.
DeepSpeedExample/megatron/mpu/random.py:202
↓ 5 callersFunctionget_global_norm
Compute total from a list of norms
deepspeed/runtime/utils.py:300
↓ 5 callersMethodget_instantiation_memory_required_per_gpu
(self, zero_stage)
deepspeed/autotuning/autotuner.py:278
↓ 5 callersFunctionget_language_model
Build language model and return along with the key to save.
DeepSpeedExample/megatron/model/language_model.py:47
↓ 5 callersFunctionget_sequence_parallel_group
Get the sequence parallel group the caller rank belongs to.
DeepSpeedExample/megatron/core/parallel_state.py:360
↓ 5 callersFunctionget_sized_buffers
(buffer_list, num_elems_list)
deepspeed/runtime/swap_tensor/utils.py:234
↓ 5 callersMethodget_slice_parallel_world_size
(self)
deepspeed/runtime/pipe/topology.py:452
↓ 5 callersMethodget_src_pp_degree
(self)
deepspeed/checkpoint/zero_checkpoint.py:38
↓ 5 callersMethodget_src_tp_degree
(self)
deepspeed/checkpoint/zero_checkpoint.py:35
↓ 5 callersMethodget_stage_id
(self)
deepspeed/runtime/pipe/topology.py:367
↓ 5 callersMethodhas_space
(self, numel)
deepspeed/runtime/swap_tensor/utils.py:71
↓ 5 callersFunctioninitialize_distributed
Initialize torch.distributed.
DeepSpeedExample/megatron/mpu/tests/commons.py:42
↓ 5 callersMethodis_record_trace
(self)
deepspeed/runtime/zero/partitioned_param_coordinator.py:131
↓ 5 callersMethodload_state_dict
Load the state dicts of each of the models
DeepSpeedExample/megatron/model/realm_model.py:105
↓ 5 callersMethodload_universal_checkpoint
(self)
deepspeed/runtime/engine.py:783
↓ 5 callersFunctionmacs_to_string
(macs, units=None, precision=2)
deepspeed/profiling/flops_profiler/profiler.py:986
↓ 5 callersMethodmaybe_print
(self, msg)
DeepSpeedExample/megatron/fp16/fp16.py:258
↓ 5 callersMethodpad
(self)
DeepSpeedExample/megatron/tokenizer/tokenizer.py:112
↓ 5 callersFunctionpartition_data
(data_list, num_partitions)
deepspeed/checkpoint/reshape_utils.py:51
↓ 5 callersMethodpostscale_gradients
(self)
deepspeed/runtime/engine.py:800
↓ 5 callersMethodquantize
deepspeed/ops/csrc/includes/quantization_utils.h:64
↓ 5 callersMethodremove_partition_and_release_buffers
(self, params)
deepspeed/runtime/swap_tensor/partitioned_param_swapper.py:225
↓ 5 callersMethodreport_ipg_memory_usage
(self, tag, param_elems)
deepspeed/runtime/zero/stage_1_and_2.py:821
↓ 5 callersMethodrun
(self)
deepspeed/autotuning/scheduler.py:159
↓ 5 callersFunctionsave_checkpoint
Save a model checkpoint.
DeepSpeedExample/megatron/checkpointing.py:119
↓ 5 callersMethodstart_timers
(self, timer_names)
deepspeed/runtime/zero/stage_1_and_2.py:1579
↓ 5 callersMethodstart_timers
(self, name_list)
deepspeed/runtime/fp16/fused_optimizer.py:187
↓ 5 callersMethodstate_dict
(self, destination=None, prefix='', keep_vars=False)
DeepSpeedExample/megatron/fp16/fp16.py:76
↓ 5 callersMethodstep
r"""Execute the weight update step after forward and backward propagation on effective_train_batch.
deepspeed/runtime/engine.py:2023
↓ 5 callersMethodstop_timers
(self, name_list)
deepspeed/runtime/fp16/fused_optimizer.py:192
↓ 5 callersMethodtokenize
(self, text)
DeepSpeedExample/megatron/tokenizer/tokenizer.py:94
↓ 5 callersFunctiontranspose
(data)
deepspeed/module_inject/replace_module.py:208
↓ 5 callersFunctionvalidate_aio_operation
deepspeed/ops/csrc/aio/common/deepspeed_aio_common.cpp:319
↓ 4 callersMethodBackwardFusedAdd
deepspeed/ops/csrc/includes/normalize_layer.h:138
↓ 4 callersMethodForwardCheckpoint
deepspeed/ops/csrc/includes/normalize_layer.h:49
↓ 4 callersMethodForwardWithBias
deepspeed/ops/csrc/includes/dropout.h:36
↓ 4 callersMethodGetHiddenSize
deepspeed/ops/csrc/includes/ds_transformer_cuda.h:142
↓ 4 callersMethodGetSeqLength
deepspeed/ops/csrc/includes/softmax.h:59
↓ 4 callersMethodGetTokenizer
(self)
DeepSpeedExample/megatron/deprecated_data_utils/datasets.py:272
↓ 4 callersMethod__init__
(self, input_size: int, output_size: int, config: Transform
DeepSpeedExample/megatron/core/transformer/custom_layers/transformer_engine.py:33
↓ 4 callersMethod_align_qkv
Internal helper for accepting the head-contiguous weight matrix and chunking the query, key, and value components.
deepspeed/module_inject/containers/features/hybrid_megatron.py:14
↓ 4 callersMethod_backup_cpuinfo
(self)
deepspeed/ops/op_builder/builder.py:359
↓ 4 callersMethod_dump_mapping
(self, data_map, map_tag=None)
deepspeed/checkpoint/deepspeed_checkpoint.py:215
↓ 4 callersFunction_gather_along_first_dim
Gather tensors and concatinate along the first dimension.
DeepSpeedExample/megatron/core/tensor_parallel/mappings.py:90
↓ 4 callersFunction_get_aio_latencies
deepspeed/ops/csrc/aio/common/deepspeed_aio_common.cpp:60
↓ 4 callersMethod_get_current_buffer
(self)
deepspeed/runtime/swap_tensor/utils.py:173
↓ 4 callersMethod_get_optimizer_param
(self, param_name)
deepspeed/runtime/engine.py:2190
↓ 4 callersMethod_get_param_swap_info
(self, parameter)
deepspeed/runtime/swap_tensor/optimizer_utils.py:659
↓ 4 callersMethod_io_aligned_numel
(self, numel)
deepspeed/runtime/swap_tensor/partitioned_optimizer_swapper.py:249
↓ 4 callersFunction_is_odd
(x)
deepspeed/runtime/pipe/schedule.py:493
↓ 4 callersMethod_model_parallel_all_reduce
Perform all reduce within model parallel group, if any.
deepspeed/runtime/zero/stage3.py:1527
↓ 4 callersMethod_model_parallel_all_reduce
Perform all reduce within model parallel group, if any.
deepspeed/runtime/zero/stage_1_and_2.py:1466
↓ 4 callersMethod_partition_qkv
Internal helper for taking contiguous QKV and partitioning it for contiguous heads.
deepspeed/module_inject/containers/features/hybrid_megatron.py:53
↓ 4 callersFunction_report_aio_statistics
deepspeed/ops/csrc/aio/common/deepspeed_aio_common.cpp:47
↓ 4 callersMethodabsolute_name
Returns absolute build path for cases where the op is pre-installed, e.g., deepspeed.ops.adam.cpu_adam will be installed as something
deepspeed/ops/op_builder/builder.py:114
↓ 4 callersMethodadd_data
(self, pp_index, tp_index, data)
deepspeed/checkpoint/reshape_meg_2d.py:22
↓ 4 callersMethodall_gather_coalesced
(params: Iterable[Parameter], safe_mode: bool = False)
deepspeed/runtime/zero/partition_parameters.py:881
↓ 4 callersMethodall_to_all_single
(self, output, input, output_spl
deepspeed/comm/torch.py:171
↓ 4 callersMethodautotuning_enabled
(self)
deepspeed/runtime/engine.py:600
↓ 4 callersMethodbackward
(ctx, output_grads)
DeepSpeedExample/megatron/core/fusions/fused_softmax.py:91
↓ 4 callersFunctionbert_extended_attention_mask
(attention_mask)
DeepSpeedExample/megatron/model/bert_model.py:35
↓ 4 callersFunctionbert_position_ids
(token_ids)
DeepSpeedExample/megatron/model/bert_model.py:51
↓ 4 callersFunctionbuild_data_loader
Data loader. Note that batch-size is the local (per GPU) batch-size.
DeepSpeedExample/tasks/finetune_utils.py:74
↓ 4 callersMethodconcat_and_pad_tokens
Concat with special tokens and pad sequence to self.max_seq_length
DeepSpeedExample/megatron/data/ict_dataset.py:126
↓ 4 callersFunctionconvert_by_vocab
Converts a sequence of [tokens|ids] using the vocab.
DeepSpeedExample/megatron/tokenizer/bert_tokenization.py:136
↓ 4 callersMethodcreate_op_builder
(self, op_name)
deepspeed/accelerator/cpu_accelerator.py:236
↓ 4 callersMethodcurriculum_enabled_legacy
(self)
deepspeed/runtime/engine.py:517
↓ 4 callersFunctiondebug_param2name_id_shape_status
(param)
deepspeed/utils/debug.py:61
↓ 4 callersMethoddecode
(self, tokens)
DeepSpeedExample/megatron/deprecated_data_utils/tokenization_gpt2.py:281
↓ 4 callersMethoddynamic_loss_scale_args
(self)
deepspeed/runtime/engine.py:827
↓ 4 callersMethodeigenvalue_gas_boundary_resolution
(self)
deepspeed/runtime/engine.py:508
↓ 4 callersFunctionfind_fit_int_dtype
(min_value, max_value)
deepspeed/runtime/data_pipeline/data_sampling/utils.py:13
↓ 4 callersMethodflatten_dense_tensors_aligned
(self, tensor_list, alignment)
deepspeed/runtime/zero/stage_1_and_2.py:829
← previousnext →301–400 of 4,989, ranked by callers