Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/bytedance/ABQ-LLM
/ functions
Functions
14,558 in github.com/bytedance/ABQ-LLM
⨍
Functions
14,558
◇
Types & classes
4,979
↓ 33 callers
Function
load_with_pointer_offset
Loads a fragment from memory
fastertransformer/3rdparty/cutlass/include/cutlass/transform/threadblock/regular_tile_iterator_tensor_op_sm70.h:201
↓ 32 callers
Function
TensorRef_aligned
fastertransformer/3rdparty/cutlass/include/cutlass/tensor_ref.h:397
↓ 32 callers
Function
a
(e)
fastertransformer/3rdparty/cutlass/docs/jquery.js:23
↓ 32 callers
Function
diffTime
fastertransformer/src/fastertransformer/utils/cuda_utils.h:264
↓ 32 callers
Function
get_workspace_size
fastertransformer/3rdparty/cutlass/include/cutlass/gemm/kernel/grouped_problem_visitor.h:417
↓ 32 callers
Function
is_main_process
()
fastertransformer/examples/pytorch/bert/bert-quantization-sparsity/utils.py:36
↓ 32 callers
Method
push
fastertransformer/3rdparty/cutlass/examples/43_dual_gemm/threadblock/dual_epilogue.h:375
↓ 31 callers
Method
at
Returns a reference to the element at a given Coord
fastertransformer/3rdparty/cutlass/include/cutlass/tensor_ref.h:321
↓ 31 callers
Method
decode
(self, tokens)
fastertransformer/examples/pytorch/gpt/utils/gpt_token_encoder.py:141
↓ 31 callers
Function
ftNcclAllGather
fastertransformer/src/fastertransformer/utils/nccl_utils.cc:70
↓ 31 callers
Method
reset
Replaces the managed object, deleting the old object.
fastertransformer/3rdparty/cutlass/include/cutlass/platform/platform.h:787
↓ 31 callers
Function
store_with_pointer_offset
Store a fragment to memory
fastertransformer/3rdparty/cutlass/include/cutlass/transform/threadblock/regular_tile_iterator_tensor_op_sm70.h:237
↓ 30 callers
Function
SubstituteTemplate
(template, values)
fastertransformer/3rdparty/cutlass/tools/library/scripts/pycutlass/src/pycutlass/__init__.py:4
↓ 30 callers
Method
get
Returns a pointer to the vector starting at the current coordinate
fastertransformer/3rdparty/cutlass/include/cutlass/conv/threadblock/conv3d_fprop_filter_tile_access_iterator_optimized.h:231
↓ 30 callers
Function
getDevice
fastertransformer/src/fastertransformer/utils/cuda_utils.h:317
↓ 30 callers
Function
safe_transpose
(param: torch.nn.Parameter)
fastertransformer/examples/pytorch/gpt/utils/huggingface_bloom_convert.py:126
↓ 30 callers
Method
valid
Returns whether access is valid or not
fastertransformer/3rdparty/cutlass/examples/42_fused_multi_head_attention/iterators/predicated_tile_access_iterator_residual_last.h:877
↓ 29 callers
Method
begin
Returns a const iterator
fastertransformer/3rdparty/cutlass/tools/library/src/manifest.cpp:91
↓ 29 callers
Method
from_pretrained
Instantiate a PreTrainedBertModel from a pre-trained model file. Download and cache the pre-trained model file if needed.
fastertransformer/examples/tensorflow/bert/bert-quantization/tokenization.py:207
↓ 29 callers
Method
uniform
Constructs a matrix from a uniform element
fastertransformer/3rdparty/cutlass/include/cutlass/matrix.h:114
↓ 29 callers
Method
update
(self, update_params: dict)
fastertransformer/examples/pytorch/gpt/utils/gpt.py:742
↓ 28 callers
Method
getPtrWithOffset
fastertransformer/src/fastertransformer/utils/Tensor.h:180
↓ 28 callers
Method
isUseSparse
fastertransformer/src/fastertransformer/utils/cublasAlgoMap.cc:217
↓ 28 callers
Method
loadModel
fastertransformer/src/fastertransformer/models/bert/BertWeight.cc:120
↓ 28 callers
Method
norm
Returns the sum of squared elements
fastertransformer/3rdparty/cutlass/include/cutlass/matrix.h:571
↓ 28 callers
Function
np2th
Possibly convert HWIO to OIHW.
fastertransformer/examples/pytorch/vit/ViT-quantization/vit_int8.py:57
↓ 28 callers
Function
np2th
Possibly convert HWIO to OIHW.
fastertransformer/examples/pytorch/vit/ViT-quantization/ViT-pytorch/models/modeling.py:38
↓ 28 callers
Method
stride
Returns the layout object's stride vector
fastertransformer/3rdparty/cutlass/include/cutlass/tensor_ref.h:291
↓ 27 callers
Function
CreateConv2dOperator
(manifest, layout, tile_descriptions, data_type, alignment_constraints, \ conv_kinds = [ConvKind.Fprop, Conv
fastertransformer/3rdparty/cutlass/tools/library/scripts/generator.py:338
↓ 27 callers
Method
Tensor
fastertransformer/src/fastertransformer/utils/Tensor.cc:84
↓ 27 callers
Function
TensorFillRandomGaussian
< Layout function
fastertransformer/3rdparty/cutlass/tools/util/include/cutlass/util/reference/host/tensor_fill.h:387
↓ 27 callers
Function
get_cutlass_status
//////////////////////////////////////////////////////////////////////////////////////////// Converts a cuDNN status to cutlass::Status
fastertransformer/3rdparty/cutlass/tools/profiler/src/cudnn_helpers.cpp:45
↓ 27 callers
Function
pad_to_multiple_of_16
fastertransformer/src/fastertransformer/kernels/moe_kernels.h:26
↓ 27 callers
Method
setGemmConfig
fastertransformer/src/fastertransformer/utils/cublasMMWrapper.cc:356
↓ 26 callers
Method
backward
(ctx, grad_in)
fastertransformer/examples/pytorch/swin/Swin-Transformer-Quantization/SwinTransformer/kernels/window_process/unit_test.py:28
↓ 26 callers
Function
checkCudaStatus
fastertransformer/tests/unittests/fp8_gemm_test/2022_03_21__fp8_stride_batch_example/worker.cpp:15
↓ 26 callers
Method
compressMatrix
fastertransformer/src/fastertransformer/utils/cublasMMWrapper.cc:770
↓ 26 callers
Function
create_initializer
(initializer_range=0.02, data_type=tf.float32)
fastertransformer/examples/tensorflow/decoder/utils/common.py:247
↓ 25 callers
Function
TensorForEach
< rank of index space
fastertransformer/3rdparty/cutlass/tools/util/include/cutlass/util/reference/host/tensor_foreach.h:93
↓ 25 callers
Method
__init__
(self, config)
fastertransformer/examples/pytorch/bert/bert-quantization-sparsity/modeling.py:534
↓ 25 callers
Method
add_pointer_offset
Adds a pointer offset in units of Element
fastertransformer/3rdparty/cutlass/include/cutlass/epilogue/threadblock/predicated_tile_iterator_v2.h:247
↓ 25 callers
Method
all
fastertransformer/src/fastertransformer/utils/Tensor.h:277
↓ 25 callers
Function
bert_example
(args)
fastertransformer/examples/pytorch/bert/bert_example.py:90
↓ 25 callers
Function
checkCublasStatus
fastertransformer/tests/unittests/fp8_gemm_test/worker.cpp:89
↓ 25 callers
Method
device_view
Accesses the tensor reference pointing to data
fastertransformer/3rdparty/cutlass/tools/util/include/cutlass/util/host_tensor.h:298
↓ 25 callers
Function
encoder_example
(args)
fastertransformer/examples/pytorch/encoder/encoder_example.py:60
↓ 25 callers
Function
ftNcclInitialize
fastertransformer/src/fastertransformer/utils/nccl_utils.cc:308
↓ 25 callers
Method
mk
Obtains a Coord<2> from GemmCoord
fastertransformer/3rdparty/cutlass/include/cutlass/gemm/gemm.h:181
↓ 25 callers
Function
swap
fastertransformer/3rdparty/cutlass/include/cutlass/fast_math.h:58
↓ 24 callers
Method
add
Allocate a chunk of memory from the buffer to tensor and copy the values.
fastertransformer/3rdparty/Megatron-LM/megatron/memory.py:89
↓ 24 callers
Function
const_max
fastertransformer/3rdparty/cutlass/include/cutlass/fast_math.h:594
↓ 24 callers
Function
ftNcclAllReduceSum
fastertransformer/src/fastertransformer/utils/nccl_utils.cc:56
↓ 24 callers
Function
ftNcclStreamSynchronize
fastertransformer/src/fastertransformer/utils/nccl_utils.cc:215
↓ 24 callers
Method
get
Return a tensor with the input `shape` as a view into the 1-D data starting at `start_index`.
fastertransformer/3rdparty/Megatron-LM/megatron/model/distributed.py:44
↓ 24 callers
Method
raw_data
fastertransformer/3rdparty/cutlass/include/cutlass/array.h:416
↓ 24 callers
Function
store_with_pointer_offset
Stores a fragment to memory at the location pointed to by the iterator
fastertransformer/3rdparty/cutlass/include/cutlass/gemm/warp/mma_simt_tile_iterator.h:507
↓ 24 callers
Function
test_all_gemm
(operation: 'GemmOperationUniversal', testcase="universal")
fastertransformer/3rdparty/cutlass/tools/library/scripts/pycutlass/src/pycutlass/test/gemm_testbed.py:459
↓ 24 callers
Method
to_host_vec
fastertransformer/src/fastertransformer/utils/gpu_buf.h:65
↓ 23 callers
Method
end
(self, session)
fastertransformer/examples/tensorflow/bert/bert-quantization/run_squad.py:243
↓ 23 callers
Function
fast_sqrt
fastertransformer/3rdparty/cutlass/include/cutlass/fast_math.h:695
↓ 23 callers
Function
get_tokenizer
Return tokenizer.
fastertransformer/3rdparty/Megatron-LM/megatron/global_vars.py:57
↓ 23 callers
Function
half2_to_float2
fastertransformer/src/fastertransformer/kernels/decoder_masked_multihead_attention_utils.h:331
↓ 23 callers
Function
k
(q)
fastertransformer/3rdparty/cutlass/docs/jquery.js:61
↓ 23 callers
Method
load
(module, prefix='')
fastertransformer/examples/pytorch/bert/bert-quantization-sparsity/modeling.py:832
↓ 23 callers
Method
set_kgroup_index
Notify the iterator which k-group it is currently pointing to. This does not advance the iterator. Rather, it overrides its internal tracking with co
fastertransformer/3rdparty/cutlass/include/cutlass/conv/warp/mma_depthwise_simt_tile_iterator.h:242
↓ 22 callers
Method
GemmCoord
Default ctor
fastertransformer/3rdparty/cutlass/include/cutlass/gemm/gemm.h:121
↓ 22 callers
Function
add
fastertransformer/src/fastertransformer/kernels/decoder_masked_multihead_attention_utils.h:181
↓ 22 callers
Method
copy_from_host
fastertransformer/3rdparty/cutlass/tools/util/include/cutlass/util/device_memory.h:307
↓ 22 callers
Function
fma
fastertransformer/src/fastertransformer/kernels/decoder_masked_multihead_attention_utils.h:404
↓ 22 callers
Function
ftNcclGroupEnd
fastertransformer/src/fastertransformer/utils/nccl_utils.cc:208
↓ 22 callers
Function
ftNcclGroupStart
fastertransformer/src/fastertransformer/utils/nccl_utils.cc:201
↓ 22 callers
Function
getCublasDataType
fastertransformer/src/fastertransformer/utils/gemm.cc:1024
↓ 22 callers
Function
get_timers
Return timers.
fastertransformer/3rdparty/Megatron-LM/megatron/global_vars.py:75
↓ 22 callers
Method
insertIfValid
fastertransformer/src/fastertransformer/utils/Tensor.h:345
↓ 22 callers
Method
layout
Returns the layout object
fastertransformer/3rdparty/cutlass/include/cutlass/thread/matrix.h:125
↓ 22 callers
Function
print_mem_usage
fastertransformer/src/fastertransformer/utils/cuda_utils.h:271
↓ 22 callers
Method
set_iteration_index
Overrides the internal iteration index
fastertransformer/3rdparty/cutlass/include/cutlass/transform/threadblock/predicated_tile_access_iterator.h:234
↓ 21 callers
Function
TensorHash
fastertransformer/3rdparty/cutlass/test/unit/conv/device/cache_testbed_output.h:505
↓ 21 callers
Method
argument
(cupy_ndarray: 'cp.ndarray')
fastertransformer/3rdparty/cutlass/tools/library/scripts/pycutlass/src/pycutlass/frontend.py:103
↓ 21 callers
Method
generate
# Args. input_token_ids: IntTensor, (batch_size, max_input_length), input hidden state to decoder. i
fastertransformer/examples/pytorch/gpt/utils/gpt_decoder.py:1013
↓ 21 callers
Method
get_grid_shape
Computes the grid shape
fastertransformer/3rdparty/cutlass/include/cutlass/gemm/device/gemv.h:109
↓ 21 callers
Function
reduction_op
(tensor, direction, math, factor)
fastertransformer/3rdparty/cutlass/tools/library/scripts/pycutlass/src/pycutlass/epilogue.py:422
↓ 21 callers
Function
resetScope
fastertransformer/src/fastertransformer/utils/nvtx_utils.cc:39
↓ 21 callers
Method
setStream
fastertransformer/src/fastertransformer/models/t5/T5Encoder.cc:248
↓ 21 callers
Method
zero_grad
Copied from torch.optim.optimizer
fastertransformer/3rdparty/Megatron-LM/megatron/optimizer/optimizer.py:503
↓ 20 callers
Function
TensorReLu
< Layout function
fastertransformer/3rdparty/cutlass/tools/util/include/cutlass/util/reference/device/tensor_relu.h:123
↓ 20 callers
Method
add_pointer_offset
Adds a pointer offset in units of Element
fastertransformer/3rdparty/cutlass/examples/42_fused_multi_head_attention/iterators/predicated_tile_iterator_residual_last.h:815
↓ 20 callers
Function
cp_async_wait<0>
fastertransformer/3rdparty/cutlass/include/cutlass/arch/memory_sm80.h:454
↓ 20 callers
Method
multiply
Elementwise multiply operator (1-by-2)
fastertransformer/3rdparty/cutlass/include/cutlass/matrix.h:332
↓ 20 callers
Method
start
(self, tag='__default')
fastertransformer/examples/pytorch/gpt/bloom_lambada.py:98
↓ 20 callers
Method
update
fastertransformer/3rdparty/cutlass/tools/library/scripts/pycutlass/src/cpp/include/gemm/gemm_universal_with_visitor.h:344
↓ 19 callers
Function
add
fastertransformer/tests/unittests/gtest_utils.h:142
↓ 19 callers
Function
copy_to_device
fastertransformer/3rdparty/cutlass/tools/util/include/cutlass/util/device_memory.h:97
↓ 19 callers
Function
copy_to_host
fastertransformer/3rdparty/cutlass/tools/util/include/cutlass/util/device_memory.h:102
↓ 19 callers
Method
encode
(self, text)
fastertransformer/examples/pytorch/gpt/utils/gpt_token_encoder.py:134
↓ 19 callers
Function
fast_divmod
* Find quotient and remainder using device-side intrinsics */
fastertransformer/3rdparty/cutlass/include/cutlass/fast_math.h:229
↓ 19 callers
Method
forward
fastertransformer/src/fastertransformer/th_op/vit/ViTOp.cc:99
↓ 19 callers
Function
getDeviceCount
fastertransformer/src/fastertransformer/utils/cuda_utils.h:324
↓ 19 callers
Function
getLocalBatchSize
fastertransformer/src/fastertransformer/utils/nccl_utils.cc:420
↓ 19 callers
Function
getTensorRef
(tensor: np.ndarray, problem_size: cutlass.gemm.GemmCoord, operand: str, layout: cutlass.layout)
fastertransformer/3rdparty/cutlass/tools/library/scripts/pycutlass/src/pycutlass/test/gemm_testbed.py:55
← previous
next →
201–300 of 14,558, ranked by callers