Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/bytedance/ABQ-LLM
/ functions
Functions
14,558 in github.com/bytedance/ABQ-LLM
⨍
Functions
14,558
◇
Types & classes
4,979
↓ 73 callers
Method
non_const_ref
fastertransformer/3rdparty/cutlass/include/cutlass/tensor_ref.h:244
↓ 72 callers
Method
get
Returns a pointer to the managed object or nullptr if no object is owned.
fastertransformer/3rdparty/cutlass/include/cutlass/platform/platform.h:777
↓ 72 callers
Function
getSMVersion
fastertransformer/src/fastertransformer/utils/cuda_utils.h:281
↓ 72 callers
Method
log
fastertransformer/src/fastertransformer/utils/logger.h:47
↓ 71 callers
Method
call
Add position encodings to :obj:`inputs`. Args: inputs: The inputs to encode. position: The single position to encode, to u
fastertransformer/examples/tensorflow/bert/utils/position.py:34
↓ 71 callers
Method
run
* \brief Forward a model and asynchronously check whether to stop. * * The device having the last rank of a pipeline parallel group checks a
fastertransformer/examples/cpp/multi_gpu_gpt/multi_gpu_gpt_async_example.cc:338
↓ 70 callers
Method
loglikelihood
(self, requests)
algorithm/lm_eval/models/dummy.py:13
↓ 68 callers
Method
clear
< Efficiently disables all accesses guarded by mask
fastertransformer/3rdparty/cutlass/examples/42_fused_multi_head_attention/iterators/epilogue_predicated_tile_iterator.h:154
↓ 68 callers
Method
size
Size of the packed Params structure
fastertransformer/3rdparty/cutlass/tools/library/scripts/rt.py:329
↓ 68 callers
Method
write
Writes the entire listing to a file
fastertransformer/3rdparty/cutlass/test/unit/conv/device/cache_testbed_output.h:195
↓ 67 callers
Function
abs
fastertransformer/3rdparty/cutlass/include/cutlass/complex.h:381
↓ 67 callers
Method
dtype
(self)
fastertransformer/examples/pytorch/gpt/utils/gpt_decoder.py:939
↓ 67 callers
Method
setFP32GemmConfig
fastertransformer/src/fastertransformer/utils/cublasMMWrapper.cc:330
↓ 66 callers
Method
load
Loads a fragment from memory at the location pointed to by the iterator.
fastertransformer/3rdparty/cutlass/include/cutlass/gemm/warp/tile_iterator_planar_complex.h:156
↓ 65 callers
Method
keys
fastertransformer/src/fastertransformer/utils/Tensor.cc:432
↓ 65 callers
Method
transform
Transform the mma operands to the required types
fastertransformer/3rdparty/cutlass/include/cutlass/gemm/warp/mma_simt.h:251
↓ 64 callers
Method
at
Returns a reference to the element at a given Coord
fastertransformer/3rdparty/cutlass/include/cutlass/thread/matrix.h:165
↓ 64 callers
Function
cp_async_fence
Establishes an ordering w.r.t previously issued cp.async instructions. Does not block.
fastertransformer/3rdparty/cutlass/include/cutlass/arch/memory_sm80.h:435
↓ 63 callers
Method
clear_mask
Clears the predicate set efficiently
fastertransformer/3rdparty/cutlass/examples/42_fused_multi_head_attention/iterators/predicated_tile_iterator_residual_last.h:846
↓ 62 callers
Method
max
fastertransformer/src/fastertransformer/utils/Tensor.h:212
↓ 62 callers
Method
state_dict
(self)
fastertransformer/examples/pytorch/swin/Swin-Transformer-Quantization/SwinTransformer/utils.py:217
↓ 62 callers
Function
to_string
Converts a ExecutionMode enumerant to a string
fastertransformer/3rdparty/cutlass/tools/profiler/src/enumerated_types.cpp:56
↓ 61 callers
Function
SubstituteTemplate
(template, values)
fastertransformer/3rdparty/cutlass/tools/library/scripts/library.py:482
↓ 61 callers
Method
get_tile_offset
Obtains the threadblock offset (in units of threadblock-scoped tiles)
fastertransformer/3rdparty/cutlass/include/cutlass/gemm/threadblock/threadblock_swizzle.h:185
↓ 61 callers
Method
valid
Returns whether access is valid or not
fastertransformer/3rdparty/cutlass/include/cutlass/transform/threadblock/predicated_tile_access_iterator.h:291
↓ 60 callers
Function
TensorFillRandomUniform
< Layout function
fastertransformer/3rdparty/cutlass/tools/util/include/cutlass/util/reference/host/tensor_fill.h:774
↓ 58 callers
Function
as_GPU_tensor
fastertransformer/src/fastertransformer/triton_backend/triton_utils.hpp:45
↓ 58 callers
Function
c
(g,e)
fastertransformer/3rdparty/cutlass/docs/jquery.js:32
↓ 58 callers
Method
forward
(self, start_ids: torch.IntTensor, start_lengths: torch.IntTensor,
fastertransformer/examples/pytorch/gpt/utils/gpt.py:580
↓ 58 callers
Method
reserve
Used for initialization
fastertransformer/3rdparty/cutlass/tools/library/src/manifest.cpp:70
↓ 57 callers
Function
BlockFillSequential
fastertransformer/3rdparty/cutlass/tools/util/include/cutlass/util/reference/host/tensor_fill.h:1196
↓ 57 callers
Method
from_pretrained
Instantiate a PreTrainedBertModel from a pre-trained model file. Download and cache the pre-trained model file if needed.
fastertransformer/examples/pytorch/bert/bert-quantization-sparsity/tokenization.py:140
↓ 57 callers
Function
move_tensor_H2D
fastertransformer/src/fastertransformer/triton_backend/triton_utils.hpp:25
↓ 54 callers
Function
barrier
()
fastertransformer/examples/pytorch/bert/bert-quantization-sparsity/utils.py:40
↓ 54 callers
Method
store
Stores a fragment to memory
fastertransformer/3rdparty/cutlass/include/cutlass/epilogue/threadblock/predicated_tile_iterator_v2.h:379
↓ 53 callers
Method
clone
fastertransformer/src/fastertransformer/tensorrt_plugin/vit/ViTPlugin.cpp:169
↓ 52 callers
Function
rotary_embedding_transform
fastertransformer/src/fastertransformer/kernels/decoder_masked_multihead_attention_utils.h:1333
↓ 52 callers
Function
torch2np
(tensor: torch.Tensor, np_data_type: typing.Optional[np.dtype] = None)
fastertransformer/examples/pytorch/utils.py:28
↓ 51 callers
Function
can_implement
fastertransformer/3rdparty/cutlass/include/cutlass/gemm/kernel/gemm_with_k_reduction.h:334
↓ 51 callers
Function
enum_auto
()
fastertransformer/3rdparty/cutlass/tools/library/scripts/pycutlass/src/pycutlass/library.py:50
↓ 51 callers
Function
get
Unpacks an element from memory
fastertransformer/3rdparty/cutlass/include/cutlass/subbyte_reference.h:140
↓ 51 callers
Function
swap
fastertransformer/3rdparty/cutlass/include/cutlass/platform/platform.h:819
↓ 49 callers
Function
TensorFillIdentity
< Layout function
fastertransformer/3rdparty/cutlass/tools/util/include/cutlass/util/reference/host/tensor_fill.h:999
↓ 48 callers
Function
const_min
fastertransformer/3rdparty/cutlass/include/cutlass/fast_math.h:589
↓ 48 callers
Function
free
fastertransformer/3rdparty/cutlass/tools/util/include/cutlass/util/device_memory.h:72
↓ 47 callers
Function
TensorEquals
< Layout function
fastertransformer/3rdparty/cutlass/tools/util/include/cutlass/util/reference/host/tensor_compare.h:106
↓ 47 callers
Method
saveNpy
fastertransformer/src/fastertransformer/utils/Tensor.cc:330
↓ 46 callers
Method
encode
(self, json_line)
fastertransformer/3rdparty/Megatron-LM/tools/preprocess_data.py:78
↓ 46 callers
Function
ftNcclParamDestroy
fastertransformer/src/fastertransformer/utils/nccl_utils.cc:299
↓ 46 callers
Method
load
Loads a fragment from memory
fastertransformer/3rdparty/cutlass/include/cutlass/epilogue/threadblock/predicated_tile_iterator_v2.h:311
↓ 46 callers
Method
max
Maximum finite value
fastertransformer/3rdparty/cutlass/include/cutlass/half.h:616
↓ 46 callers
Method
update
(self, val, n=1)
fastertransformer/examples/pytorch/vit/ViT-quantization/main.py:57
↓ 45 callers
Method
end
Returns a const iterator
fastertransformer/3rdparty/cutlass/tools/library/src/manifest.cpp:96
↓ 44 callers
Function
CudaToolkitVersionSatisfies
(semantic_ver_string, major, minor, patch = 0)
fastertransformer/3rdparty/cutlass/tools/library/scripts/generator.py:17
↓ 44 callers
Method
load_state_dict
(self, state_dict)
fastertransformer/examples/pytorch/swin/Swin-Transformer-Quantization/SwinTransformer/utils.py:220
↓ 44 callers
Method
setBF16GemmConfig
fastertransformer/src/fastertransformer/utils/cublasMMWrapper.cc:347
↓ 44 callers
Method
setFP16GemmConfig
fastertransformer/src/fastertransformer/utils/cublasMMWrapper.cc:338
↓ 44 callers
Function
test_all_conv2d
(operation: Conv2dOperation, conv_test_sizes = [], interleaved=False)
fastertransformer/3rdparty/cutlass/tools/library/scripts/pycutlass/src/pycutlass/test/conv2d_testbed.py:514
↓ 43 callers
Method
host_data
Gets pointer to host data
fastertransformer/3rdparty/cutlass/tools/util/include/cutlass/util/host_tensor.h:244
↓ 43 callers
Function
invokeGeneralLayerNorm
fastertransformer/src/fastertransformer/kernels/layernorm_kernels.h:138
↓ 43 callers
Method
start
Start the timer.
fastertransformer/3rdparty/Megatron-LM/megatron/global_vars.py:205
↓ 42 callers
Method
get_tiled_shape
Returns the shape of the problem in units of logical tiles Gemm* problem size: gemm(M, N, K)
fastertransformer/3rdparty/cutlass/include/cutlass/gemm/threadblock/threadblock_swizzle.h:118
↓ 41 callers
Function
CreateGemmOperator
(manifest, layouts, tile_descriptions, data_type, \ alignment_constraints, complex_transforms = None, epilog
fastertransformer/3rdparty/cutlass/tools/library/scripts/generator.py:49
↓ 41 callers
Method
set
Set a bit within the predicate vector.
fastertransformer/3rdparty/cutlass/include/cutlass/predicate_vector.h:463
↓ 41 callers
Method
x
Accesses the x element of the imaginary part of the quaternion
fastertransformer/3rdparty/cutlass/include/cutlass/quaternion.h:151
↓ 41 callers
Method
y
Accesses the y element of the imaginary part of the quaternion
fastertransformer/3rdparty/cutlass/include/cutlass/quaternion.h:163
↓ 41 callers
Method
z
Accesses the z element of the imaginary part of the quaternion
fastertransformer/3rdparty/cutlass/include/cutlass/quaternion.h:175
↓ 40 callers
Function
arg
fastertransformer/3rdparty/cutlass/include/cutlass/complex.h:387
↓ 40 callers
Method
to_half
Bitcasts to CUDA's half type
fastertransformer/3rdparty/cutlass/include/cutlass/half.h:450
↓ 39 callers
Function
conj
Returns the complex conjugate
fastertransformer/3rdparty/cutlass/include/cutlass/complex.h:423
↓ 39 callers
Method
stop
Stop the timer.
fastertransformer/3rdparty/Megatron-LM/megatron/global_vars.py:212
↓ 39 callers
Method
store
Store a fragment to memory
fastertransformer/3rdparty/cutlass/examples/42_fused_multi_head_attention/iterators/predicated_tile_iterator_residual_last.h:905
↓ 39 callers
Method
type
Gets the type of the iterator
fastertransformer/3rdparty/cutlass/tools/profiler/src/problem_space.h:172
↓ 38 callers
Method
SpGemm
fastertransformer/src/fastertransformer/utils/gemm.cc:715
↓ 38 callers
Method
clear
Clear the cache
fastertransformer/3rdparty/cutlass/tools/library/scripts/pycutlass/src/cpp/compiler.h:69
↓ 38 callers
Method
stridedBatchedGemm
fastertransformer/src/fastertransformer/utils/gemm.cc:481
↓ 37 callers
Method
GetBoolean
fastertransformer/3rdparty/INIReader.h:468
↓ 37 callers
Method
add_tile_offset
Advances an iterator along logical dimensions of matrix in units of whole threadblock tiles
fastertransformer/3rdparty/cutlass/include/cutlass/conv/threadblock/predicated_scale_bias_vector_iterator.h:347
↓ 37 callers
Function
checkCublasStatus
fastertransformer/tests/unittests/fp8_gemm_test/2022_03_21__fp8_stride_batch_example/worker.cpp:8
↓ 37 callers
Method
valid
Returns true if element is in a defined state
fastertransformer/3rdparty/cutlass/examples/03_visualize_layout/visualize_layout.h:148
↓ 37 callers
Method
what
fastertransformer/src/fastertransformer/utils/gemm.h:601
↓ 36 callers
Function
checkCudaStatus
fastertransformer/tests/unittests/fp8_gemm_test/worker.cpp:98
↓ 36 callers
Function
deserialize_value
fastertransformer/src/fastertransformer/tensorrt_plugin/swin/serialize.hpp:127
↓ 36 callers
Function
divide
Ensure that numerator is divisible by the denominator and return the division value.
fastertransformer/3rdparty/Megatron-LM/megatron/mpu/utils.py:26
↓ 36 callers
Method
filter
Filtering operations based on various criteria
fastertransformer/3rdparty/cutlass/tools/library/scripts/manifest.py:271
↓ 36 callers
Function
serialize_value
fastertransformer/src/fastertransformer/tensorrt_plugin/swin/serialize.hpp:121
↓ 35 callers
Method
bfloat16
(self)
fastertransformer/examples/pytorch/gpt/utils/gpt_decoder.py:969
↓ 35 callers
Function
ftNcclRecv
fastertransformer/src/fastertransformer/utils/nccl_utils.cc:111
↓ 35 callers
Function
ftNcclSend
fastertransformer/src/fastertransformer/utils/nccl_utils.cc:85
↓ 34 callers
Method
ParseError
fastertransformer/3rdparty/INIReader.h:382
↓ 34 callers
Function
bcast
fastertransformer/src/fastertransformer/utils/mpi_utils.cc:114
↓ 34 callers
Method
float
Converts to float
fastertransformer/3rdparty/cutlass/include/cutlass/half.h:426
↓ 34 callers
Function
ftNcclBroadCast
fastertransformer/src/fastertransformer/utils/nccl_utils.cc:136
↓ 34 callers
Function
getActivationType
fastertransformer/src/fastertransformer/utils/activation_types.h:34
↓ 34 callers
Method
kn
Obtains a Coord<2> from GemmCoord
fastertransformer/3rdparty/cutlass/include/cutlass/gemm/gemm.h:199
↓ 34 callers
Function
rotary_embedding_coefficient
fastertransformer/src/fastertransformer/kernels/decoder_masked_multihead_attention_utils.h:1325
↓ 33 callers
Method
advance
fastertransformer/3rdparty/cutlass/include/cutlass/conv/threadblock/conv2d_tile_iterator.h:172
↓ 33 callers
Method
gemm
fastertransformer/src/fastertransformer/utils/gemm.cc:116
↓ 33 callers
Function
get_workspace_size
fastertransformer/3rdparty/cutlass/test/unit/gemm/device/testbed_grouped_scheduler.h:111
↓ 33 callers
Method
layout
Returns the layout object
fastertransformer/3rdparty/cutlass/include/cutlass/tensor_ref.h:285
← previous
next →
101–200 of 14,558, ranked by callers