MCPcopy Create free account

hub / github.com/bytedance/ABQ-LLM / functions

Functions14,558 in github.com/bytedance/ABQ-LLM

⨍ Functions 14,558◇ Types & classes 4,979
↓ 19 callersMethodgetWorkspaceSize
fastertransformer/src/fastertransformer/tensorrt_plugin/vit/ViTPlugin.cpp:226
↓ 19 callersMethodhalf
(self)
fastertransformer/examples/pytorch/baichuan2/baichuanv2.py:258
↓ 19 callersFunctionload_with_pointer_offset
Loads a fragment from memory at the location pointed to by the iterator. (scalar loads)
fastertransformer/3rdparty/cutlass/include/cutlass/gemm/warp/mma_simt_tile_iterator.h:472
↓ 19 callersFunctionsetScope
fastertransformer/src/fastertransformer/utils/nvtx_utils.cc:34
↓ 18 callersFunctioncutlass_get_smem_pointer
CUTLASS helper to get SMEM pointer
fastertransformer/3rdparty/cutlass/include/cutlass/arch/memory_sm75.h:96
↓ 18 callersMethoddivmod
Computes integer division and modulus using precomputed values. This is computationally inexpensive. Simply returns the quotient
fastertransformer/3rdparty/cutlass/include/cutlass/fast_math.h:307
↓ 18 callersMethodget
Returns a pointer to the managed object
fastertransformer/3rdparty/cutlass/tools/util/include/cutlass/util/device_memory.h:214
↓ 18 callersFunctionnorm
fastertransformer/3rdparty/cutlass/include/cutlass/complex.h:393
↓ 18 callersMethodrank
Returns the number of dimensions of the problem space
fastertransformer/3rdparty/cutlass/tools/profiler/src/problem_space.h:839
↓ 18 callersMethodsetLevel
fastertransformer/src/fastertransformer/utils/logger.h:68
↓ 18 callersMethodset_k_partition
Functionally required for serial reduction in the epilogue
fastertransformer/3rdparty/cutlass/include/cutlass/epilogue/thread/conversion_op.h:96
↓ 18 callersMethodslice
fastertransformer/src/fastertransformer/utils/Tensor.cc:382
↓ 18 callersFunctiontoString
fastertransformer/examples/cpp/multi_gpu_gpt/multi_gpu_gpt_async_example.cc:60
↓ 17 callersMethodadd
Elementwise add operator (1-by-2)
fastertransformer/3rdparty/cutlass/include/cutlass/matrix.h:276
↓ 17 callersFunctionadd_to_logging
(name)
fastertransformer/3rdparty/Megatron-LM/megatron/training.py:518
↓ 17 callersMethodcopy
(self)
fastertransformer/examples/pytorch/gptj/utils/reference_gptj.py:43
↓ 17 callersFunctionfloat2_to_half2
fastertransformer/src/fastertransformer/kernels/decoder_masked_multihead_attention_utils.h:305
↓ 17 callersFunctioninvokeGeneralAddBiasResidualPreLayerNorm
fastertransformer/src/fastertransformer/kernels/layernorm_kernels.h:85
↓ 17 callersMethodrelease
Releases ownership of the managed object, if any
fastertransformer/3rdparty/cutlass/include/cutlass/platform/platform.h:780
↓ 17 callersFunctionsave_val
(val, key, tp_num=None)
fastertransformer/examples/pytorch/gpt/utils/huggingface_opt_convert.py:65
↓ 17 callersMethodsizeBytes
fastertransformer/src/fastertransformer/utils/Tensor.cc:220
↓ 17 callersMethodwrite
(self, sizes, doc_idx)
fastertransformer/3rdparty/Megatron-LM/megatron/data/indexed_dataset.py:363
↓ 16 callersFunctionadd_pointer_offset
fastertransformer/3rdparty/cutlass/include/cutlass/epilogue/threadblock/shared_load_iterator.h:67
↓ 16 callersMethodadd_tile_offset
fastertransformer/src/fastertransformer/cutlass_extensions/include/cutlass_extensions/epilogue/threadblock/epilogue_tensor_op_int32.h:216
↓ 16 callersFunctionbF
(b0,b1)
fastertransformer/3rdparty/cutlass/docs/jquery.js:16
↓ 16 callersFunctioncopy
fastertransformer/src/fastertransformer/kernels/llama/flash_attention2/utils.h:138
↓ 16 callersFunctionfind_divisor
* Find divisor, using find_log2 */
fastertransformer/3rdparty/cutlass/include/cutlass/fast_math.h:212
↓ 16 callersMethodget
Returns a pointer
fastertransformer/3rdparty/cutlass/include/cutlass/transform/threadblock/regular_scale_bias_vector_access_iterator.h:129
↓ 16 callersFunctionhandleOptArg
fastertransformer/src/fastertransformer/kernels/gpt_kernels.h:146
↓ 16 callersMethodinverse
fastertransformer/3rdparty/cutlass/include/cutlass/layout/matrix.h:113
↓ 16 callersMethodload
fastertransformer/src/fastertransformer/cutlass_extensions/include/cutlass_extensions/gemm/warp/mma_tensorop_dequantizer.h:249
↓ 16 callersFunctionload_with_byte_offset
Loads a fragment from memory with additional logical offset
fastertransformer/3rdparty/cutlass/include/cutlass/gemm/warp/mma_complex_tensor_op_tile_iterator_sm80.h:253
↓ 16 callersMethodpad
(self)
fastertransformer/3rdparty/Megatron-LM/megatron/tokenizer/tokenizer.py:114
↓ 16 callersMethodreset
Updates only the pointer
fastertransformer/3rdparty/cutlass/include/cutlass/tensor_ref.h:249
↓ 16 callersMethodstop
(self, tag='__default')
fastertransformer/examples/pytorch/gpt/bloom_lambada.py:101
↓ 16 callersMethodstride
Returns the stride of the layout
fastertransformer/3rdparty/cutlass/include/cutlass/layout/tensor_op_multiplicand_sm80.h:127
↓ 16 callersMethodtokenize
(self, text)
fastertransformer/3rdparty/Megatron-LM/tasks/orqa/unsupervised/tokenizers.py:146
↓ 16 callersMethodvalid
Returns true if the current coordinate is within the filter tensor W
fastertransformer/3rdparty/cutlass/include/cutlass/conv/threadblock/conv3d_fprop_filter_tile_access_iterator_optimized.h:225
↓ 15 callersMethodat
Check if the kernel has already been compiled
fastertransformer/3rdparty/cutlass/tools/library/scripts/pycutlass/src/cpp/compiler.h:52
↓ 15 callersMethodemit
(self, operation)
fastertransformer/3rdparty/cutlass/tools/library/scripts/pycutlass/src/pycutlass/epilogue.py:775
↓ 15 callersMethodequal
(self, tensor_D, tensor_D_ref, problem_size)
fastertransformer/3rdparty/cutlass/tools/library/scripts/pycutlass/src/pycutlass/test/conv2d_testbed.py:341
↓ 15 callersFunctionfinalize
(input_ids, beam_width, parent_ids, sequence_lengths, outputs, end_id, max_seq_len=None)
fastertransformer/examples/tensorflow/gpt/gpt_example.py:176
↓ 15 callersFunctiongetCommWorldRank
fastertransformer/src/fastertransformer/utils/mpi_utils.cc:82
↓ 15 callersFunctiongetCommWorldSize
fastertransformer/src/fastertransformer/utils/mpi_utils.cc:91
↓ 15 callersFunctiongetLayerNormType
fastertransformer/src/fastertransformer/kernels/layernorm_kernels.h:33
↓ 15 callersFunctioninvokeAddBiasResidual
fastertransformer/src/fastertransformer/kernels/add_residual_kernels.h:44
↓ 15 callersFunctionload_with_pointer_offset
Loads a fragment from memory with additional logical offset
fastertransformer/3rdparty/cutlass/include/cutlass/gemm/warp/mma_tensor_op_tile_iterator.h:419
↓ 15 callersMethodmalloc
fastertransformer/src/fastertransformer/utils/allocator.h:318
↓ 15 callersFunctionset_random_seed
Set random seed for reproducability.
fastertransformer/3rdparty/Megatron-LM/megatron/mpu/tests/commons.py:34
↓ 15 callersMethodsize
Returns the number of elements stored in the host tensor
fastertransformer/3rdparty/cutlass/tools/util/include/cutlass/util/host_tensor.h:234
↓ 15 callersMethodstep
Performs a single optimization step. Arguments: closure (callable, optional): A closure that reevaluates the model
fastertransformer/examples/pytorch/bert/bert-quantization-sparsity/optimization.py:113
↓ 15 callersFunctionstore_with_pointer_offset
Stores a fragment to memory with additional pointer offset
fastertransformer/3rdparty/cutlass/include/cutlass/gemm/warp/mma_tensor_op_tile_iterator.h:3815
↓ 15 callersFunctionx
(bx)
fastertransformer/3rdparty/cutlass/docs/jquery.js:23
↓ 14 callersFunctionRematerializeBlockIdxZ
Helper to rematerialize block Idx. Reduces register liveness.
fastertransformer/3rdparty/cutlass/include/cutlass/gemm/threadblock/threadblock_swizzle.h:84
↓ 14 callersMethodcontains
Determines whether a location is within a tensor
fastertransformer/3rdparty/cutlass/include/cutlass/tensor_view.h:190
↓ 14 callersMethoddecode
(self, tokens)
fastertransformer/examples/tensorflow/gpt/utils/gpt_token_encoder.py:141
↓ 14 callersFunctionf
(e)
fastertransformer/3rdparty/cutlass/docs/jquery.js:23
↓ 14 callersFunctionfast_cos
fastertransformer/3rdparty/cutlass/include/cutlass/fast_math.h:623
↓ 14 callersMethodmin
fastertransformer/src/fastertransformer/utils/Tensor.h:235
↓ 14 callersMethodround
fastertransformer/3rdparty/cutlass/tools/profiler/src/problem_space.h:360
↓ 14 callersMethodset_residual_tile
fastertransformer/3rdparty/cutlass/examples/42_fused_multi_head_attention/iterators/predicated_tile_iterator_residual_last.h:851
↓ 14 callersFunctiontodevice
Pass the host_data to device memory
fastertransformer/3rdparty/cutlass/tools/library/scripts/pycutlass/src/pycutlass/memory_manager.py:54
↓ 14 callersFunctionzero
fastertransformer/src/fastertransformer/kernels/decoder_masked_multihead_attention_utils.h:1301
↓ 13 callersFunctionRematerializeBlockIdxY
Helper to rematerialize block Idx. Reduces register liveness.
fastertransformer/3rdparty/cutlass/include/cutlass/gemm/threadblock/threadblock_swizzle.h:78
↓ 13 callersMethodSequence
Helper to construct a sequence range
fastertransformer/3rdparty/cutlass/tools/profiler/src/problem_space.h:461
↓ 13 callersFunctionTestGemmOperator
Test GEMM Operation based on configuration
fastertransformer/3rdparty/cutlass/tools/library/scripts/pycutlass/test/unit/test_sm80.py:44
↓ 13 callersMethodbegin
fastertransformer/3rdparty/cutlass/tools/profiler/src/problem_space.cpp:145
↓ 13 callersFunctioncreate_initializer
Creates a `truncated_normal_initializer` with the given range.
fastertransformer/examples/tensorflow/bert/tensorflow_bert/my_modeling.py:381
↓ 13 callersMethodidentity
Constructs an identity matrix
fastertransformer/3rdparty/cutlass/include/cutlass/matrix.h:2574
↓ 13 callersMethodinsert
Insert a new compiled kernel for new configuration
fastertransformer/3rdparty/cutlass/tools/library/scripts/pycutlass/src/cpp/compiler.h:62
↓ 13 callersFunctionload_with_byte_offset
Loads a fragment from memory with additional logical offset
fastertransformer/3rdparty/cutlass/include/cutlass/gemm/warp/scale_bias_tile_iterator.h:258
↓ 13 callersFunctionload_with_pointer_offset
Loads a fragment from memory with additional logical offset
fastertransformer/3rdparty/cutlass/include/cutlass/gemm/warp/mma_tensor_op_tile_iterator_sm70.h:330
↓ 13 callersMethodmoe_fc1_test_helper
(self, compute_type, weight_dtype, rtol, atol, activation_str)
fastertransformer/tests/moe/th_moe_unit_tests.py:190
↓ 13 callersFunctionmul
fastertransformer/tests/unittests/gtest_utils.h:148
↓ 13 callersFunctionprint_separator
(message)
fastertransformer/3rdparty/Megatron-LM/megatron/mpu/tests/commons.py:76
↓ 13 callersFunctionread_word_list
fastertransformer/src/fastertransformer/utils/word_list.cc:24
↓ 13 callersMethodsafe_to_vec
(batch_size, value, dtype, is_cpu=False)
fastertransformer/tests/unittests/th_op/test_th_decode_op.py:79
↓ 13 callersMethodtokenize
(self, text)
fastertransformer/examples/pytorch/bert/bert-quantization-sparsity/tokenization.py:112
↓ 13 callersMethodupdate
Lightweight update given a subset of arguments
fastertransformer/3rdparty/cutlass/include/cutlass/gemm/device/trmm.h:459
↓ 12 callersMethodLogger
fastertransformer/src/fastertransformer/utils/logger.cc:22
↓ 12 callersMethodRandom
Helper to construct a range that is a random distribution
fastertransformer/3rdparty/cutlass/tools/profiler/src/problem_space.h:466
↓ 12 callersMethodargument_index
Returns the index of an argument by name
fastertransformer/3rdparty/cutlass/tools/profiler/src/problem_space.cpp:510
↓ 12 callersMethodbegin
(self)
fastertransformer/examples/tensorflow/bert/bert-quantization/run_squad.py:210
↓ 12 callersMethodcheck_result
(self, beam_width, datatype, test_time, topk=4, topp=0.0, batch_size=-1, decoder_bleu_sco
fastertransformer/tests/decoding/tf_decoding_unit_test.py:50
↓ 12 callersMethodcompress_weights
fastertransformer/src/fastertransformer/models/llama/LlamaWeight.cc:315
↓ 12 callersMethodcompute
(self, input_tokens, beam_width, max_seq_len, top_k=1, top_p = 0.0, beam_search_diversity_rate = 0.0,
fastertransformer/examples/tensorflow/t5/utils/ft_decoding.py:252
↓ 12 callersFunctioncreate_initializer
(initializer_range=0.02, data_type=tf.float32)
fastertransformer/examples/tensorflow/bert/utils/common.py:72
↓ 12 callersFunctioncreate_int_feature
(values)
fastertransformer/examples/tensorflow/bert/tensorflow_bert/bert/run_classifier.py:492
↓ 12 callersFunctioncreate_int_feature
(values)
fastertransformer/examples/tensorflow/bert/bert-quantization/utils/create_glue_data.py:463
↓ 12 callersMethodcuda
(self)
fastertransformer/examples/pytorch/llama/llamav1.py:254
↓ 12 callersFunctionfast_sin
fastertransformer/3rdparty/cutlass/include/cutlass/fast_math.h:641
↓ 12 callersFunctionfill
(x)
fastertransformer/examples/pytorch/bert/bert-quantization-sparsity/apex_sparsity/sparse_masklib.py:9
↓ 12 callersMethodgetAlgo
fastertransformer/src/fastertransformer/utils/cublasAlgoMap.cc:140
↓ 12 callersMethodgetUseOrderCol322R4R4
fastertransformer/src/fastertransformer/utils/cublasINT8MMWrapper.cc:456
↓ 12 callersFunctionget_allocated_size
()
fastertransformer/3rdparty/cutlass/tools/library/scripts/pycutlass/src/pycutlass/memory_manager.py:72
↓ 12 callersFunctionget_num_microbatches
()
fastertransformer/3rdparty/Megatron-LM/megatron/global_vars.py:44
↓ 12 callersFunctionget_world_size
()
fastertransformer/examples/pytorch/bert/bert-quantization-sparsity/utils.py:28
↓ 12 callersMethodgrid_shape
Computes the grid size given a chosen threadblock shape
fastertransformer/3rdparty/cutlass/include/cutlass/reduction/kernel/reduce_split_k.h:134
↓ 12 callersFunctionh0_h0
fastertransformer/src/fastertransformer/kernels/decoder_masked_multihead_attention_utils.h:395
↓ 12 callersFunctionhead_projection
Project hidden states to a specific head with a 4D-shape.
fastertransformer/examples/tensorflow/xlnet/modeling.py:97
← previousnext →301–400 of 14,558, ranked by callers