Functions — github.com/NVIDIA/FasterTransformer

CodeHub MCPcopy Create free account

hub / github.com/NVIDIA/FasterTransformer / functions

Functions4,228 in github.com/NVIDIA/FasterTransformer

⨍ Functions 4,228◇ Types & classes 1,417

↓ 18 callersFunctiontoString

examples/cpp/multi_gpu_gpt/multi_gpu_gpt_async_example.cc:60

↓ 17 callersMethodend

(self, session)

examples/tensorflow/bert/bert-quantization/run_squad.py:243

↓ 17 callersMethodgenerate

# Args. input_token_ids: IntTensor, (batch_size, max_input_length), input hidden state to decoder. i

examples/pytorch/gpt/utils/gpt_decoder.py:1013

↓ 17 callersFunctiongetDeviceCount

src/fastertransformer/utils/cuda_utils.h:324

↓ 17 callersFunctioninvokeGeneralAddBiasResidualPreLayerNorm

src/fastertransformer/kernels/layernorm_kernels.h:85

↓ 17 callersFunctionsave_val

(val, key, tp_num=None)

examples/pytorch/gpt/utils/huggingface_opt_convert.py:65

↓ 17 callersMethodslice

src/fastertransformer/utils/Tensor.cc:334

↓ 16 callersFunctionadd

src/fastertransformer/kernels/decoder_masked_multihead_attention_utils.h:181

↓ 16 callersMethodadd_tile_offset

src/fastertransformer/cutlass_extensions/include/cutlass_extensions/epilogue/threadblock/epilogue_tensor_op_int32.h:216

↓ 16 callersMethodencode

examples/pytorch/gpt/utils/gpt_token_encoder.py:134

↓ 16 callersMethodload

src/fastertransformer/cutlass_extensions/include/cutlass_extensions/gemm/warp/mma_tensorop_dequantizer.h:249

↓ 15 callersFunctiongetLayerNormType

src/fastertransformer/kernels/layernorm_kernels.h:33

↓ 15 callersFunctiongetLocalBatchSize

src/fastertransformer/utils/nccl_utils.cc:420

↓ 15 callersFunctionhalf2_to_float2

src/fastertransformer/kernels/decoder_masked_multihead_attention_utils.h:331

↓ 15 callersMethodmalloc

src/fastertransformer/utils/allocator.h:318

↓ 15 callersFunctionresetScope

src/fastertransformer/utils/nvtx_utils.cc:39

↓ 15 callersMethodstop

(self, tag='__default')

examples/pytorch/gpt/bloom_lambada.py:101

↓ 13 callersMethodcopy

examples/pytorch/gptj/utils/reference_gptj.py:43

↓ 13 callersFunctioncreate_initializer

Creates a `truncated_normal_initializer` with the given range.

examples/tensorflow/bert/tensorflow_bert/my_modeling.py:381

↓ 13 callersMethodmin

src/fastertransformer/utils/Tensor.h:230

↓ 13 callersMethodmoe_fc1_test_helper

(self, compute_type, weight_dtype, rtol, atol, activation_str)

tests/moe/th_moe_unit_tests.py:190

↓ 13 callersMethodsafe_to_vec

(batch_size, value, dtype, is_cpu=False)

tests/unittests/th_op/test_th_decode_op.py:79

↓ 13 callersFunctionsetScope

src/fastertransformer/utils/nvtx_utils.cc:34

↓ 13 callersMethodsizeBytes

src/fastertransformer/utils/Tensor.cc:172

↓ 13 callersMethodtokenize

examples/pytorch/bert/bert-quantization-sparsity/tokenization.py:112

↓ 12 callersMethodLogger

src/fastertransformer/utils/logger.cc:22

↓ 12 callersMethodcheck_result

(self, beam_width, datatype, test_time, topk=4, topp=0.0, batch_size=-1, decoder_bleu_sco

tests/decoding/tf_decoding_unit_test.py:50

↓ 12 callersFunctioncreate_initializer

(initializer_range=0.02, data_type=tf.float32)

examples/tensorflow/bert/utils/common.py:72

↓ 12 callersFunctioncreate_int_feature

examples/tensorflow/bert/bert-quantization/utils/create_glue_data.py:463

↓ 12 callersMethodgetAlgo

src/fastertransformer/utils/cublasAlgoMap.cc:132

↓ 12 callersMethodgetUseOrderCol322R4R4

src/fastertransformer/utils/cublasINT8MMWrapper.cc:456

↓ 12 callersMethodgetWorkspaceSize

src/fastertransformer/tensorrt_plugin/vit/ViTPlugin.cpp:226

↓ 12 callersFunctionget_world_size

examples/pytorch/bert/bert-quantization-sparsity/utils.py:28

↓ 12 callersFunctionh0_h0

src/fastertransformer/kernels/decoder_masked_multihead_attention_utils.h:395

↓ 12 callersFunctionhandleOptArg

src/fastertransformer/kernels/gpt_kernels.h:135

↓ 12 callersFunctionhead_projection

Project hidden states to a specific head with a 4D-shape.

examples/tensorflow/xlnet/modeling.py:97

↓ 12 callersFunctionmake_fn_out

examples/pytorch/gptneox/utils/eleutherai_gpt_neox_convert.py:230

↓ 12 callersMethodsetLevel

src/fastertransformer/utils/logger.h:68

↓ 12 callersMethodstep

Performs a single optimization step. Arguments: closure (callable, optional): A closure that reevaluates the model

examples/pytorch/bert/bert-quantization-sparsity/optimization.py:113

↓ 12 callersMethodtokenize

examples/tensorflow/bert/bert-quantization/tokenization.py:156

↓ 11 callersMethod_read_tsv

Reads a tab separated value file.

examples/tensorflow/bert/bert-quantization/utils/create_glue_data.py:136

↓ 11 callersMethodconvert_tensor

convert TensorFlow Tensor to FasterTransformer Tensor

src/fastertransformer/tf_op/BaseOp.h:99

↓ 11 callersMethoddecode

examples/tensorflow/gpt/utils/gpt_token_encoder.py:141

↓ 11 callersFunctionfinalize

(input_ids, beam_width, parent_ids, sequence_lengths, outputs, end_id, max_seq_len=None)

examples/tensorflow/gpt/gpt_example.py:176

↓ 11 callersFunctionfma

src/fastertransformer/kernels/decoder_masked_multihead_attention_utils.h:404

↓ 11 callersMethodfrom_pretrained

Instantiate a PreTrainedBertModel from a pre-trained model file. Download and cache the pre-trained model file if needed.

examples/tensorflow/bert/bert-quantization/tokenization.py:207

↓ 11 callersMethodgemm

src/fastertransformer/utils/gemm.cc:116

↓ 11 callersFunctiongetCommWorldRank

src/fastertransformer/utils/mpi_utils.cc:82

↓ 11 callersFunctiongetCommWorldSize

src/fastertransformer/utils/mpi_utils.cc:91

↓ 11 callersFunctionget_config

Get a yacs CfgNode object with default values.

examples/pytorch/vit/ViT-quantization/config.py:220

↓ 11 callersFunctionget_shape_list

Returns a list of the shape of tensor, preferring static dimensions. Args: tensor: A tf.Tensor object to find the shape of. expected_rank:

examples/tensorflow/bert/tensorflow_bert/my_modeling.py:905

↓ 11 callersMethodinitialize

examples/cpp/multi_gpu_gpt/multi_gpu_gpt_async_example.cc:301

↓ 11 callersFunctioninvokeAddBiasResidual

src/fastertransformer/kernels/add_residual_kernels.h:44

↓ 11 callersFunctionvec2str

src/fastertransformer/utils/string_utils.h:56

↓ 10 callersMethodasdict

examples/pytorch/gpt/utils/bloom.py:115

↓ 10 callersFunctionbuild_ft_longformer

(hf_model_dir, layer_num, head_num, size_per_head, intermediate_size, local_attn_windo

examples/pytorch/longformer/longformer_qa.py:30

↓ 10 callersMethodcompress_weights

src/fastertransformer/models/bert/BertLayerWeight.cc:149

↓ 10 callersFunctioncreate_initializer

Creates a `truncated_normal_initializer` with the given range.

examples/tensorflow/bert/bert-quantization/modeling.py:408

↓ 10 callersFunctionfill

examples/pytorch/bert/bert-quantization-sparsity/apex_sparsity/sparse_masklib.py:9

↓ 10 callersFunctiongetCublasOperation

src/fastertransformer/utils/gemm.cc:1062

↓ 10 callersMethodinterSize

src/fastertransformer/layers/adapter_layers/LinearAdapterLayer.h:43

↓ 10 callersFunctioninvokeGetPaddingOffset

src/fastertransformer/kernels/bert_preprocess_kernels.h:37

↓ 10 callersFunctionisGatedActivation

src/fastertransformer/utils/activation_types.h:60

↓ 10 callersFunctionparse_from_config

examples/pytorch/longformer/longformer_qa.py:15

↓ 10 callersMethodresizeLayer

src/fastertransformer/models/t5/T5EncoderWeight.cc:379

↓ 9 callersMethodbegin

examples/tensorflow/bert/bert-quantization/run_squad.py:210

↓ 9 callersMethodcompute_sparse_masks

Call this method to enable sparsity. If init(...) was called with allow_recompute_mask=False AND sparsity is disabled, pruned field can be Non

examples/pytorch/bert/bert-quantization-sparsity/apex_sparsity/asp.py:155

↓ 9 callersMethodconvert_tokens_to_ids

Converts a sequence of tokens into ids using the vocab.

examples/pytorch/bert/bert-quantization-sparsity/tokenization.py:119

↓ 9 callersMethodcustomAllReduce

src/fastertransformer/utils/custom_ar_comm.cc:46

↓ 9 callersFunctionfloat2_to_half2

src/fastertransformer/kernels/decoder_masked_multihead_attention_utils.h:305

↓ 9 callersMethodgemm_dequant_test_helper

(self, compute_type, weight_dtype, gemm_ms, gemm_ns, gemm_ks, rtol, atol, act_str="only_gemm", benchmark=False

tests/gemm_dequantize/th_gemm_dequantize.py:65

↓ 9 callersFunctiongetModelFileType

src/fastertransformer/utils/cuda_utils.cc:364

↓ 9 callersFunctionget_bits_in_quant_type

src/fastertransformer/kernels/cutlass_kernels/cutlass_preprocessors.cc:24

↓ 9 callersFunctionget_shape_list

Returns a list of the shape of tensor, preferring static dimensions. Args: tensor: A tf.Tensor object to find the shape of. expected_rank:

examples/tensorflow/bert/bert-quantization/modeling.py:988

↓ 9 callersFunctionhalf_to_float

src/fastertransformer/kernels/decoder_masked_multihead_attention_utils.h:322

↓ 9 callersFunctioninvokeEmbeddingLookupPosEncodingPadCount

src/fastertransformer/kernels/decoding_kernels.h:55

↓ 9 callersFunctionrandom_cuda_tensor

(shape, dtype, mean=0, std=1)

tests/moe/th_moe_unit_tests.py:20

↓ 9 callersFunctionread

src/fastertransformer/tensorrt_plugin/bert_fp8/bertFp8Plugin.h:94

↓ 9 callersMethodrun_all_qa

(self, seq_len, batch_size, ft_longformer, data_type)

tests/longformer/py_longformer_unit_test.py:64

↓ 9 callersMethodsetup

src/fastertransformer/layers/DynamicDecodeLayer.cc:153

↓ 9 callersMethodswapInternalBuffer

src/fastertransformer/utils/custom_ar_comm.cc:109

↓ 9 callersFunctionwrite

src/fastertransformer/tensorrt_plugin/bert_fp8/bertFp8Plugin.h:87

↓ 8 callersFunction__internal_pack_u16x2_to_u32

tests/unittests/fp8_gemm_test/2022_03_21__fp8_stride_batch_example/include/cuda_fp8.hpp:625

↓ 8 callersMethod_read_tsv

Reads a tab separated value file.

examples/pytorch/bert/bert-quantization-sparsity/processors/glue.py:70

↓ 8 callersFunctioncheckIfFileExist

src/fastertransformer/utils/memory_utils.h:124

↓ 8 callersMethodcompute

(self, input_tokens, beam_width, max_seq_len, top_k=1, top_p = 0.0, beam_search_diversity_rate = 0.0,

examples/tensorflow/t5/utils/ft_decoding.py:252

↓ 8 callersMethodinit_model_for_pruning

Call this method to modify your model to take advantage of sparse matrix multiplication. Note that this call alone only augments the model wit

examples/pytorch/bert/bert-quantization-sparsity/apex_sparsity/asp.py:29

↓ 8 callersFunctionmegatron2hf_name

(saved_key, name_mapping)

examples/pytorch/t5/utils/nemo_t5_ckpt_convert.py:112

↓ 8 callersMethodrun_attn

(self, batch_size, seq_len, head_num, size_per_head, data_type)

tests/decoding/tf_fused_self_multihead_attention_unit_test.py:68

↓ 8 callersFunctiontime_test

(sess, tensor, iterations=100, warmup=True)

examples/tensorflow/bert/utils/common.py:75

↓ 7 callersMethod__init__

examples/pytorch/vit/ViT-quantization/vit_int8.py:212

↓ 7 callersFunctionbuild_dataset

(is_train, config)

examples/pytorch/vit/ViT-quantization/data.py:111

↓ 7 callersFunctionbuild_model

examples/pytorch/swin/Swin-Transformer-Quantization/models/build.py:12

↓ 7 callersMethodconvertTritonTensorToFt

src/fastertransformer/triton_backend/transformer_triton_backend.hpp:170

↓ 7 callersFunctionconvert_from_float

src/fastertransformer/kernels/decoder_masked_multihead_attention/decoder_masked_multihead_attention_template.hpp:730

↓ 7 callersFunctionconvert_val

examples/pytorch/gpt/utils/megatron_gpt_moe_ckpt_convert.py:191

↓ 7 callersFunctiondiv_up

src/fastertransformer/kernels/decoder_masked_multihead_attention/decoder_masked_multihead_attention_template.hpp:1038

↓ 7 callersMethodforward

src/fastertransformer/models/bert_int8/BertINT8.cc:162

↓ 7 callersFunctiongetAddress

src/fastertransformer/utils/allocator.h:113

↓ 7 callersMethodgetStream

src/fastertransformer/layers/BaseLayer.h:43

← previous next →101–200 of 4,228, ranked by callers