Functions — github.com/NVIDIA/FasterTransformer

CodeHub MCPcopy Create free account

hub / github.com/NVIDIA/FasterTransformer / functions

Functions4,228 in github.com/NVIDIA/FasterTransformer

⨍ Functions 4,228◇ Types & classes 1,417

↓ 4 callersFunctiondropout

Perform dropout. Args: input_tensor: float Tensor. dropout_prob: Python float. The probability of dropping out a value (NOT of *keepi

examples/tensorflow/bert/bert-quantization/modeling.py:367

↓ 4 callersMethodenabled

src/fastertransformer/models/t5/T5AdapterWeight.h:39

↓ 4 callersFunctionfind_chars_or_comment

Return pointer to first char (of chars) or inline comment in given string, or pointer to null at end of string if neither found. Inline comment mus

3rdparty/INIReader.h:137

↓ 4 callersMethodforward

# Args. max_input_length: int, maximum input context length. step: int, the current step index. ite: int

examples/pytorch/gpt/utils/gpt_decoder.py:656

↓ 4 callersMethodforward_features

examples/pytorch/swin/Swin-Transformer-Quantization/models/swin_transformer.py:624

↓ 4 callersMethodfrom_opt

Alternate constructor. Args: model (onmt.modules.NMTModel): See :func:`__init__()`. fields (dict[str, torchtext.data.

examples/pytorch/decoding/utils/translator.py:183

↓ 4 callersFunctionft_bert

Run the bert transformer layer by FasterTransformer. Args: inputs: A tf.Tensor with shape [batch_size, seq_len, hidden_dimension].

examples/tensorflow/bert/utils/bert.py:344

↓ 4 callersFunctionft_decoding

Run the decoding with beam search by TensorFlow. Args: memory_tensor: A tf.tensor with shape [batch_size * beam_width, max(memor

examples/tensorflow/decoding/utils/ft_decoding.py:43

↓ 4 callersFunctionfuse_qkv_weight

examples/pytorch/gpt/utils/huggingface_opt_convert.py:149

↓ 4 callersFunctiongen_relative_pos_bias

src/fastertransformer/th_op/swin/SwinOp.cc:138

↓ 4 callersFunctiongenerateWeightSize

examples/cpp/swin/functions.h:73

↓ 4 callersFunctiongetCusparseOperation

src/fastertransformer/utils/gemm.cc:1086

↓ 4 callersMethodgetDModel

src/fastertransformer/models/t5/T5Encoder.h:157

↓ 4 callersMethodgetMaxSeqLen

src/fastertransformer/models/multi_gpu_gpt/ParallelGptWeight.h:69

↓ 4 callersFunctiongetSetDevice

src/fastertransformer/utils/cuda_utils.cc:333

↓ 4 callersMethodget_checkpoints_paths

Injects tensor/pipeline model parallel ranks into the filepath. Does nothing if not using model parallelism.

examples/pytorch/nemo.py:129

↓ 4 callersFunctionget_ft_quant_type

src/fastertransformer/th_op/common/WeightOnlyQuantOps.cc:40

↓ 4 callersFunctionget_shape_list

(tensor, expected_rank=None, name=None)

examples/tensorflow/bert/utils/bert.py:276

↓ 4 callersFunctiongraph_fn_builder

examples/tensorflow/bert/tensorflow_bert/profile_transformer_inference.py:130

↓ 4 callersMethodhas_adapters

src/fastertransformer/models/t5/T5Decoder.h:125

↓ 4 callersFunctionint_result_cross_check

(name, tf_result, op_result, shape)

examples/tensorflow/decoder/utils/common.py:281

↓ 4 callersFunctioninvokeAddBias

src/fastertransformer/kernels/activation_kernels.h:91

↓ 4 callersFunctioninvokeAddBiasAttentionFfnResidual

src/fastertransformer/kernels/add_residual_kernels.h:67

↓ 4 callersFunctioninvokeMaskPaddingTokens

src/fastertransformer/kernels/gpt_kernels.h:214

↓ 4 callersFunctioninvokeUpdatePaddingCount

src/fastertransformer/kernels/gpt_kernels.h:193

↓ 4 callersMethodisFuseBatchGemm

src/fastertransformer/utils/cublasMMWrapper.cc:625

↓ 4 callersMethodload_from

(self, weights, n_block)

examples/pytorch/vit/ViT-quantization/vit_int8.py:329

↓ 4 callersMethodload_from

(self, weights)

examples/pytorch/vit/ViT-quantization/vit_int8.py:420

↓ 4 callersFunctionnormalize_answer

Lower text and remove punctuation, articles and extra whitespace.

examples/tensorflow/bert/tensorflow_bert/squad_evaluate-v1.1.py:25

↓ 4 callersFunctionnormalize_answer

Lower text and remove punctuation, articles and extra whitespace.

examples/tensorflow/bert/tensorflow_bert/squad_evaluate_v1_1.py:25

↓ 4 callersFunctionnp2th

(weights, is_conv=False)

examples/pytorch/vit/VisionTransformerWeightLoader.py:32

↓ 4 callersMethodnum_bytes

examples/cpp/xlnet/cnpy.h:61

↓ 4 callersFunctionpost_attention

Post-attention processing.

examples/tensorflow/xlnet/modeling.py:106

↓ 4 callersFunctionprintMatrix

tests/unittests/fp8_gemm_test/worker.cpp:191

↓ 4 callersFunctionprint_fraction

(a, b, counters, desc)

examples/pytorch/vit/ViT-quantization/quant_utils.py:243

↓ 4 callersFunctionprint_fraction

(a, b, counters, desc)

examples/pytorch/bert/bert-quantization-sparsity/quant_utils.py:243

↓ 4 callersFunctionprint_fraction

(a, b, counters, desc)

examples/pytorch/swin/Swin-Transformer-Quantization/quant_utils.py:212

↓ 4 callersFunctionread_squad_examples

Read a SQuAD json file into a list of SquadExample.

examples/tensorflow/bert/bert-quantization/utils/create_squad_data.py:152

↓ 4 callersFunctionread_start_ids

examples/cpp/multi_gpu_gpt/gpt_example_utils.cc:27

↓ 4 callersMethodregisterCallback

src/fastertransformer/triton_backend/transformer_triton_backend.hpp:273

↓ 4 callersFunctionrel_attn_core

Core relative positional attention operations.

examples/tensorflow/xlnet/modeling.py:141

↓ 4 callersFunctionreshape_to_matrix

Reshapes a >= rank 2 tensor to a rank 2 tensor (i.e., a matrix).

examples/tensorflow/bert/tensorflow_bert/my_modeling.py:942

↓ 4 callersFunctionrstrip

Strip whitespace chars off end of given string, in place. Return s. */

3rdparty/INIReader.h:118

↓ 4 callersMethodrun_moe_fc

src/fastertransformer/kernels/moe_kernels.h:218

↓ 4 callersFunctionrun_torch

(model, images, mark)

examples/pytorch/swin/infer_swintransformer_op.py:173

↓ 4 callersMethodserialize

examples/pytorch/bert/bert-quantization-sparsity/data/GooglePretrainedWeightDownloader.py:145

↓ 4 callersMethodsetT5StructureDiff

src/fastertransformer/models/t5/T5EncoderWeight.cc:392

↓ 4 callersMethodset_default_quant_desc_input

Args: value: An instance of :func:`QuantDescriptor <quantization.QuantDescriptor>`

examples/tensorflow/bert/bert-quantization/ft-tensorflow-quantization/ft_tensorflow_quantization/python/layers/utils.py:32

↓ 4 callersFunctionsetup

examples/pytorch/vit/ViT-quantization/main.py:96

↓ 4 callersFunctiontile

tests/unittests/gtest_utils.h:122

↓ 4 callersMethodto_bfloat16

examples/pytorch/bert/utils/encoder.py:216

↓ 4 callersMethodto_bfloat16

examples/pytorch/t5/utils/ft_decoding.py:421

↓ 4 callersMethodto_bfloat16

examples/pytorch/t5/utils/ft_encoder.py:355

↓ 4 callersMethodto_cuda

examples/pytorch/bert/utils/encoder.py:196

↓ 4 callersMethodto_dict

Serializes this instance to a Python dictionary.

examples/pytorch/bert/bert-quantization-sparsity/modeling.py:295

↓ 4 callersMethodto_float

examples/pytorch/t5/utils/ft_encoder.py:343

↓ 4 callersMethodto_half

examples/pytorch/bert/utils/encoder.py:210

↓ 4 callersFunctiontrain_loop

(args, model, optimizer, step, num_steps)

examples/pytorch/bert/bert-quantization-sparsity/apex_sparsity/test/toy_problem.py:31

↓ 4 callersMethodunRegisterCallback

src/fastertransformer/triton_backend/transformer_triton_backend.hpp:279

↓ 4 callersMethodwarn

examples/pytorch/vit/ViT-quantization/quant_utils.py:32

↓ 4 callersFunctionweight_quantize

src/fastertransformer/th_op/bert/WeightQuantizeOp.cc:44

↓ 4 callersFunctionwhitespace_tokenize

Runs basic whitespace cleaning and splitting on a piece of text.

examples/pytorch/bert/bert-quantization-sparsity/tokenization.py:86

↓ 3 callersMethodInt8Gemm

src/fastertransformer/utils/cublasMMWrapper.cc:1074

↓ 3 callersMethod__init__

examples/tensorflow/decoder/utils/common.py:298

↓ 3 callersMethod__init__

Constructs a InputExample. Args: guid: Unique id for the example. text_a: string. The untokenized text of the first sequen

examples/tensorflow/xlnet/convertInput.py:73

↓ 3 callersFunction_copy_tokenizer_file_if_defined

(key_name, tokenizer_file_path, saved_dir)

examples/pytorch/gpt/utils/nemo_ckpt_convert.py:469

↓ 3 callersFunction_copy_tokenizer_file_if_defined

(key_name, tokenizer_file_path, saved_dir)

examples/pytorch/t5/utils/nemo_t5_ckpt_convert.py:1165

↓ 3 callersMethod_create_examples

Creates examples for the training and dev sets.

examples/tensorflow/bert/bert-quantization/utils/create_glue_data.py:217

↓ 3 callersMethod_create_examples

Creates examples for the training and dev sets.

examples/tensorflow/bert/bert-quantization/utils/create_glue_data.py:257

↓ 3 callersMethod_create_examples

Creates examples for the training and dev sets.

examples/tensorflow/bert/bert-quantization/utils/create_glue_data.py:297

↓ 3 callersMethod_create_examples

Creates examples for the training and dev sets.

examples/tensorflow/xlnet/convertInput.py:164

↓ 3 callersFunction_cudaGetErrorEnum

debug tools ********************************* */

src/fastertransformer/utils/cuda_utils.h:74

↓ 3 callersMethod_map

examples/pytorch/gptneox/utils/gptneox.py:113

↓ 3 callersMethod_map

examples/pytorch/gpt/utils/gpt.py:220

↓ 3 callersMethod_read_tsv

Reads a tab separated value file.

examples/tensorflow/xlnet/convertInput.py:115

↓ 3 callersFunction_update_config_entry

(key, file_pattern)

examples/pytorch/gpt/utils/nemo_ckpt_convert.py:449

↓ 3 callersFunction_update_config_entry

(key, file_pattern)

examples/pytorch/t5/utils/nemo_t5_ckpt_convert.py:1145

↓ 3 callersFunctionadd_bias_and_interleave_quantized_tensor_inplace

src/fastertransformer/kernels/cutlass_kernels/cutlass_preprocessors.cc:424

↓ 3 callersMethodadd_pointer_offset

Adds a pointer offset in units of Element

src/fastertransformer/cutlass_extensions/include/cutlass_extensions/epilogue/threadblock/epilogue_tensor_op_int32.h:207

↓ 3 callersFunctionadd_special_tokens_to_tokenizer

examples/pytorch/tokenizer.py:16

↓ 3 callersMethodasdict

examples/pytorch/gpt/utils/gpt.py:747

↓ 3 callersFunctionbasic_moe_fc

(activations, expert_for_row, weights, scales, biases)

tests/moe/th_moe_unit_tests.py:23

↓ 3 callersFunctionbatch_igemm_config

src/fastertransformer/utils/gemm_test/encoder_igemm_func.cc:982

↓ 3 callersFunctioncached_path

Given something that might be a URL (or might be a local path), determine which. If it's a URL, download the file and cache it, and retur

examples/pytorch/bert/bert-quantization-sparsity/file_utils.py:97

↓ 3 callersMethodcan_implement

Determines whether kernel satisfies alignment

src/fastertransformer/cutlass_extensions/include/cutlass_extensions/gemm/kernel/fpA_intB_gemm.h:228

↓ 3 callersFunctioncast_to_float

src/fastertransformer/kernels/decoder_masked_multihead_attention/decoder_masked_multihead_attention_template.hpp:877

↓ 3 callersFunctionconvert_examples_to_features

Loads a data file into a list of `InputBatch`s.

examples/tensorflow/bert/bert-quantization/extract_features.py:210

↓ 3 callersFunctionconvert_examples_to_features

Loads a data file into a list of `InputBatch`s.

examples/pytorch/bert/bert-quantization-sparsity/run_swag.py:139

↓ 3 callersFunctionconvert_parameter_name

examples/pytorch/gpt/utils/huggingface_bloom_convert.py:130

↓ 3 callersFunctionconvert_to_unicode

Converts `text` to Unicode (if it's not already), assuming utf-8 input.

examples/tensorflow/bert/bert-quantization/tokenization.py:92

↓ 3 callersFunctioncreate_model

Creates a classification model.

examples/tensorflow/bert/bert-quantization/run_squad.py:308

↓ 3 callersMethodcuda

(self, device=None)

examples/pytorch/gpt/utils/gpt_decoder.py:948

↓ 3 callersFunctioncustomMatmulRun

src/fastertransformer/utils/gemm_test/gemm_func.cc:156

↓ 3 callersFunctiondecode_output

(outputs, model_dir, input_ids, actual_seq_len)

examples/pytorch/longformer/longformer_qa.py:96

↓ 3 callersMethoddownload_glue

(self, task_name)

examples/pytorch/bert/bert-quantization-sparsity/data/Downloader.py:80

↓ 3 callersMethodelapsed_time_in_sec

(self, tag='__default')

examples/pytorch/gpt/bloom_lambada.py:108

↓ 3 callersMethodencode

examples/tensorflow/gpt/utils/gpt_token_encoder.py:134

↓ 3 callersFunctionestimate_best_config_from_occupancies

src/fastertransformer/kernels/cutlass_kernels/cutlass_heuristic.cc:128

↓ 3 callersFunctionexpand_dims

examples/tensorflow/bert/bert-quantization/ft-tensorflow-quantization/ft_tensorflow_quantization/python/utils/utils.py:23

↓ 3 callersFunctionfile_based_convert_examples_to_features

Convert a set of `InputExample`s to a TFRecord file.

examples/tensorflow/bert/bert-quantization/utils/create_glue_data.py:450

← previous next →301–400 of 4,228, ranked by callers