MCPcopy Create free account

hub / github.com/AIS-SNU/Smart-Infinity / functions

Functions4,989 in github.com/AIS-SNU/Smart-Infinity

⨍ Functions 4,989◇ Types & classes 624
↓ 8 callersMethodrandom_ltd_enabled
(self)
deepspeed/runtime/engine.py:541
↓ 8 callersMethodstage_to_global
(self, stage_id, **kwargs)
deepspeed/runtime/pipe/topology.py:398
↓ 8 callersMethodstate_dict
(self, destination=None, prefix='', keep_vars=False)
DeepSpeedExample/megatron/model/module.py:181
↓ 8 callersMethodstore
deepspeed/ops/csrc/includes/quantization_utils.h:82
↓ 8 callersFunctionswap_in_tensors
(swap_handle, tensor_buffers, swap_paths)
deepspeed/runtime/swap_tensor/utils.py:19
↓ 8 callersFunctiontranspose
(data)
deepspeed/module_inject/policy.py:113
↓ 8 callersMethodupdate_state
deepspeed/ops/csrc/includes/cpu_adam.h:227
↓ 7 callersMethodQuantize
(self, value_list, quantize_bits, groups, key, merge_dim=0)
deepspeed/runtime/weight_quantizer.py:42
↓ 7 callersMethod__init__
(self, init_method, output_layer_init_method)
DeepSpeedExample/megatron/model/transformer.py:69
↓ 7 callersMethod_log_timers
(self, name_list, force=False)
deepspeed/runtime/swap_tensor/optimizer_utils.py:676
↓ 7 callersMethodadd_item
(self, tensor)
deepspeed/runtime/data_pipeline/data_sampling/indexed_dataset.py:283
↓ 7 callersMethodall_gather
(self, tensor_list, tensor, group=None, async_op=False)
deepspeed/comm/torch.py:120
↓ 7 callersFunctionalloc_mat
deepspeed/ops/csrc/adam/opencl/adam/util.cpp:25
↓ 7 callersMethodallocate_all
(self, num_elems, dtype)
deepspeed/runtime/swap_tensor/utils.py:213
↓ 7 callersMethodattention
Returns attention qkv and dense parameters weight: (3*hidden, hidden) and (hidden, hidden) bias: (3*hidden) and (hidden)
deepspeed/module_inject/policy.py:33
↓ 7 callersFunctiondata_file_path
(prefix_path)
DeepSpeedExample/megatron/data/indexed_dataset.py:115
↓ 7 callersFunctiondata_file_path
(prefix_path)
deepspeed/runtime/data_pipeline/data_sampling/indexed_dataset.py:126
↓ 7 callersFunctionduration_to_string
(duration, units=None, precision=2)
deepspeed/profiling/flops_profiler/profiler.py:1070
↓ 7 callersMethodencode
(self, *inputs, **kwargs)
deepspeed/model_implementations/diffusers/vae.py:98
↓ 7 callersMethodflops_profiler_enabled
(self)
deepspeed/runtime/engine.py:571
↓ 7 callersMethodget_command
get command token corresponding to `name`
DeepSpeedExample/megatron/deprecated_data_utils/tokenization.py:271
↓ 7 callersMethodget_data
(self, pp_index=None, tp_index=None)
deepspeed/checkpoint/reshape_meg_2d.py:31
↓ 7 callersMethodget_global_rank
(self)
deepspeed/runtime/pipe/topology.py:407
↓ 7 callersMethodget_param_coordinator
(self, training)
deepspeed/runtime/zero/parameter_offload.py:270
↓ 7 callersMethodget_partition_rank
subclass can overload to specify different relative rank in parameter partition group
deepspeed/runtime/zero/partition_parameters.py:1518
↓ 7 callersMethodget_rss
(self)
deepspeed/accelerator/cpu_accelerator.py:106
↓ 7 callersMethodget_world_group
(self)
deepspeed/comm/torch.py:243
↓ 7 callersFunctioninit_method_normal
Init method based on N(0, sigma).
DeepSpeedExample/megatron/model/utils.py:25
↓ 7 callersFunctioninstalled_cuda_version
(name="")
deepspeed/ops/op_builder/builder.py:38
↓ 7 callersMethodload
(self, path: str, map_location=None)
deepspeed/runtime/checkpoint_engine/checkpoint_engine.py:25
↓ 7 callersMethodmemory_allocated
(self, device_index=None)
deepspeed/accelerator/cpu_accelerator.py:117
↓ 7 callersMethodmlp
Returns mlp intermediate and output weight: (intermediate, hidden) and (hidden, intermediate) bias: (intermediate) and (hidde
deepspeed/module_inject/policy.py:94
↓ 7 callersMethodon_accelerator
(self, tensor)
deepspeed/accelerator/cpu_accelerator.py:228
↓ 7 callersFunctionprint_mat
deepspeed/ops/csrc/adam/opencl/adam/util.cpp:45
↓ 7 callersMethodtrain
r
deepspeed/runtime/engine.py:1650
↓ 7 callersMethodtrain_micro_batch_size_per_gpu
(self)
deepspeed/runtime/engine.py:635
↓ 6 callersMethodDropout
deepspeed/ops/csrc/includes/dropout.h:26
↓ 6 callersMethodIntTensor
(self)
deepspeed/accelerator/cpu_accelerator.py:209
↓ 6 callersMethodRATIO
deepspeed/ops/csrc/includes/dropout.h:22
↓ 6 callersMethodSetTokenizer
(self, tokenizer)
DeepSpeedExample/megatron/deprecated_data_utils/datasets.py:263
↓ 6 callersMethod__init__
Initialize the Sparsity Pattern Config. For usage example please see, TODO DeepSpeed Sparse Transformer Tutorial Arguments:
deepspeed/ops/sparse_attention/sparsity_config.py:15
↓ 6 callersFunction_ensure_divisibility
Ensure that numerator is divisible by the denominator.
deepspeed/utils/groups.py:53
↓ 6 callersMethod_flush_gradient_swapper
(self, gradient_swapper, use_fpga=False)
deepspeed/runtime/swap_tensor/optimizer_utils.py:215
↓ 6 callersFunction_is_even
(x)
deepspeed/runtime/pipe/schedule.py:489
↓ 6 callersMethodamp_enabled
(self)
deepspeed/runtime/engine.py:765
↓ 6 callersFunctionassert_ints_same_as_other_ranks
NOTE: creates both communication and synchronization overhead so should be used sparingly takes a list of ints from each rank and ensure
deepspeed/runtime/zero/utils.py:76
↓ 6 callersMethodcompress_by_chunk
(self, cupy_bool_tensor, num_chunks)
deepspeed/runtime/compression/cupy.py:22
↓ 6 callersMethodcompressed_allreduce
(self, buffer_m: torch.tensor, worker_error, server_error, local_rank)
deepspeed/runtime/comm/mpi.py:132
↓ 6 callersMethodcurriculum_learning_enabled
(self)
deepspeed/runtime/engine.py:535
↓ 6 callersFunctiondeallocate_output_tensor
Pseudo-deallocate (i.e., set to scalar) the output tensor's '.data' field. This method should be called right after the output tensor has been
DeepSpeedExample/megatron/core/pipeline_parallel/schedules.py:108
↓ 6 callersMethoddynamic_loss_scale
(self)
deepspeed/runtime/engine.py:821
↓ 6 callersMethodencode
(self, text)
DeepSpeedExample/megatron/deprecated_data_utils/tokenization_gpt2.py:278
↓ 6 callersFunctionget_checkpoint_name
A unified checkpoint name.
DeepSpeedExample/megatron/checkpointing.py:72
↓ 6 callersFunctionget_dict_param
(param_dict, param_name, param_default_value)
deepspeed/runtime/config_utils.py:194
↓ 6 callersFunctionget_fp16_enabled
(param_dict)
deepspeed/runtime/config.py:152
↓ 6 callersFunctionget_linear_layer
Simple linear layer with weight initialization.
DeepSpeedExample/megatron/model/utils.py:43
↓ 6 callersMethodget_mask
(self, pruning_type='row')
deepspeed/compression/basic_layer.py:296
↓ 6 callersFunctionget_params
(name, group_dict)
deepspeed/compression/config.py:134
↓ 6 callersMethodget_slice_parallel_group
(self)
deepspeed/runtime/pipe/topology.py:455
↓ 6 callersFunctionindex_file_path
(prefix_path)
DeepSpeedExample/megatron/data/indexed_dataset.py:111
↓ 6 callersFunctionindex_file_path
(prefix_path)
deepspeed/runtime/data_pipeline/data_sampling/indexed_dataset.py:122
↓ 6 callersMethodirecv
(self, tensor, src=None, group=None, tag=0)
deepspeed/comm/torch.py:194
↓ 6 callersMethodis_complete_trace
(self)
deepspeed/runtime/zero/partitioned_param_coordinator.py:125
↓ 6 callersMethodis_first_stage
True if this process is in the first stage in the pipeline.
deepspeed/runtime/pipe/engine.py:458
↓ 6 callersMethodisend
(self, tensor, dst, group=None, tag=0)
deepspeed/comm/torch.py:191
↓ 6 callersMethodloss_scale
(self)
deepspeed/runtime/engine.py:774
↓ 6 callersMethodmake_dxx_lut
(layout, block, step, trans, device, transform=lambda idx: idx)
deepspeed/ops/sparse_attention/matmul.py:385
↓ 6 callersMethodmax_memory_allocated
(self, device_index=None)
deepspeed/accelerator/cpu_accelerator.py:120
↓ 6 callersMethodmax_train_batch_size
(self)
deepspeed/autotuning/autotuner.py:244
↓ 6 callersFunctionmaybe_get_lora
(p)
deepspeed/module_inject/policy.py:219
↓ 6 callersMethodmem_status
(self, msg, print_rank=-1, reset_max=False)
deepspeed/runtime/pipe/engine.py:1193
↓ 6 callersFunctionopen_file
deepspeed/ops/csrc/aio/common/deepspeed_aio_common.cpp:265
↓ 6 callersFunctionparams_to_string
(params_num, units=None, precision=2)
deepspeed/profiling/flops_profiler/profiler.py:1053
↓ 6 callersFunctionprint_rank_0
(message)
deepspeed/runtime/zero/contiguous_memory_allocator.py:11
↓ 6 callersFunctionprint_rank_0
(message, debug=False, force=False)
deepspeed/runtime/swap_tensor/partitioned_param_swapper.py:20
↓ 6 callersMethodreduce
(self, tensor, dst, op=ReduceOp.SUM, group=None, async_op=False)
deepspeed/comm/torch.py:107
↓ 6 callersMethodreset
(self)
deepspeed/runtime/swap_tensor/utils.py:43
↓ 6 callersMethodrun_ds_config
(self, ds_config, exp_name)
deepspeed/autotuning/autotuner.py:1052
↓ 6 callersFunctionscaled_init_method_normal
Init method based on N(0, sigma/sqrt(2*num_layers).
DeepSpeedExample/megatron/model/utils.py:33
↓ 6 callersMethodset_device
(self, device_index)
deepspeed/accelerator/cpu_accelerator.py:31
↓ 6 callersMethodsetup_layout
Create layout tensor for the given sequence length Arguments: seq_len: required: an integer determining number of attention head
deepspeed/ops/sparse_attention/sparsity_config.py:31
↓ 6 callersMethodstop_timers
(self, timer_names)
deepspeed/runtime/zero/stage_1_and_2.py:1586
↓ 6 callersFunctionswap_out_tensors
(swap_handle, tensor_buffers, swap_paths)
deepspeed/runtime/swap_tensor/utils.py:24
↓ 5 callersMethodGetNumHeads
deepspeed/ops/csrc/includes/softmax.h:57
↓ 5 callersMethodIncrementStep
deepspeed/ops/csrc/includes/cpu_adam.h:207
↓ 5 callersMethodSynchronizeStreams
deepspeed/ops/csrc/includes/cpu_adam.h:189
↓ 5 callersMethod__init__
(self, args, world_info_base64)
deepspeed/launcher/multinode_runner.py:53
↓ 5 callersMethod__init__
(self, act_range_momentum=0.95, quant_mode='symmetric')
deepspeed/compression/basic_layer.py:28
↓ 5 callersMethod__init__
(self, dim=None, dtype=torch.float, eps=1e-5, weight=None, bias=None)
deepspeed/module_inject/layers.py:58
↓ 5 callersMethod_check_and_set
Auxiliary function for checking the values in the checkpoint and setting them.
DeepSpeedExample/megatron/learning_rates.py:120
↓ 5 callersFunction_ensure_var_is_not_initialized
Make sure the input variable is not None.
DeepSpeedExample/megatron/global_vars.py:150
↓ 5 callersMethod_get_swap_paths
(self, params, must_exist=False)
deepspeed/runtime/swap_tensor/partitioned_param_swapper.py:137
↓ 5 callersMethod_io_aligned_numel
(self, numel)
deepspeed/runtime/swap_tensor/partitioned_param_swapper.py:374
↓ 5 callersMethod_jobname_templ
(self, job, rw)
setup/nvme_speed_test.py:77
↓ 5 callersFunction_log_rank0
(msg)
deepspeed/runtime/zero/mics_utils.py:22
↓ 5 callersMethod_reduce_op
Helper function. If the op provided is not a torch.dist.ReduceOp, convert it and return
deepspeed/comm/torch.py:249
↓ 5 callersMethod_release_params
Helper for `release_[component]` methods. Accepts a list of tuples where the first element is the module param that needs to be delet
deepspeed/module_inject/containers/features/hybrid_engine.py:77
↓ 5 callersFunction_set_cuda_rng_state
Sets the random number generator state of the current GPU. Argumentss: new_state (torch.ByteTensor): The desired state This function
DeepSpeedExample/megatron/core/tensor_parallel/random.py:64
↓ 5 callersFunction_set_cuda_rng_state
Sets the random number generator state of the current GPU. Argumentss: new_state (torch.ByteTensor): The desired state This function
DeepSpeedExample/megatron/mpu/random.py:71
↓ 5 callersFunction_set_cuda_rng_state
Sets the random number generator state of the current GPU. Arguments: new_state (torch.ByteTensor): The desired state This function i
deepspeed/runtime/activation_checkpointing/checkpointing.py:88
← previousnext →201–300 of 4,989, ranked by callers