Functions — github.com/huggingface/transformers

hub / github.com/huggingface/transformers / functions

Functions4,119 in github.com/huggingface/transformers

⨍ Functions 4,119◇ Types & classes 939↳ Endpoints 4

↓ 1 callersMethod_from_pretrained

(cls, pretrained_model_name_or_path, *init_inputs, **kwargs)

src/transformers/tokenization_utils_base.py:1143

↓ 1 callersMethod_gather_by_expansion

expand dims of idxs and vectors for all hashes and gather

src/transformers/modeling_reformer.py:730

↓ 1 callersMethod_gather_logprob

(logprob, target)

src/transformers/modeling_tf_transfo_xl_utilities.py:111

↓ 1 callersFunction_gelu_python

Original Implementation of the gelu activation function in Google Bert repo when initially created. For information: OpenAI GPT's gelu is sli

src/transformers/activations.py:15

↓ 1 callersMethod_generate_beam_search

Generate sequences for each example with beam search.

src/transformers/modeling_utils.py:1355

↓ 1 callersMethod_generate_beam_search

Generate sequences for each example with beam search.

src/transformers/modeling_tf_utils.py:1077

↓ 1 callersMethod_generate_no_beam_search

Generate sequences for each example without beam search (num_beams == 1). All returned sequence are generated independantly.

src/transformers/modeling_utils.py:1255

↓ 1 callersMethod_generate_no_beam_search

Generate sequences for each example without beam search (num_beams == 1). All returned sequence are generated independantly.

src/transformers/modeling_tf_utils.py:914

↓ 1 callersMethod_get_attn_subsequent_mask

Get an attention mask to avoid using the subsequent info. Args: size: int Returns: (`LongTensor`):

examples/seq2seq/bertabs/modeling_bertabs.py:347

↓ 1 callersFunction_get_best_indexes

Get the n-best logits from a list.

templates/adding_a_new_example_script/utils_xxx.py:973

↓ 1 callersFunction_get_best_indexes

Get the n-best logits from a list.

src/transformers/data/metrics/squad_metrics.py:336

↓ 1 callersMethod_get_embedding_shapes

(self)

src/transformers/modeling_transfo_xl.py:564

↓ 1 callersMethod_get_embeds

(self, wte, input_ids)

tests/test_modeling_tf_common.py:453

↓ 1 callersMethod_get_example_from_tensor_dict

(self, tensor_dict, evaluate=False)

src/transformers/data/processors/squad.py:458

↓ 1 callersFunction_get_generated_ngrams

(hypo_idx)

src/transformers/modeling_utils.py:1631

↓ 1 callersFunction_get_generated_ngrams

(hypo_idx)

src/transformers/modeling_tf_utils.py:1429

↓ 1 callersMethod_get_new_num_tokens_layer

(self, new_num_tokens, layer)

src/transformers/modeling_transfo_xl.py:551

↓ 1 callersFunction_get_question_end_index

Computes the index of the first occurance of `sep_token_id`.

src/transformers/modeling_longformer.py:46

↓ 1 callersMethod_get_resized_embeddings

Build a resized Embedding Variable from a provided token Embedding Module. Increasing the size will add newly initialized vectors at the

src/transformers/modeling_tf_utils.py:266

↓ 1 callersMethod_get_sorted_bucket_idx_and_undo_sorted_bucket_idx

(self, sequence_length, buckets, num_hashes)

src/transformers/modeling_reformer.py:528

↓ 1 callersFunction_glue_convert_examples_to_features

( examples: List[InputExample], tokenizer: PreTrainedTokenizer, max_length: Optional[int] = None,

src/transformers/data/processors/glue.py:107

↓ 1 callersMethod_hash_vectors

(self, vectors, num_hashes)

src/transformers/modeling_reformer.py:463

↓ 1 callersFunction_improve_answer_span

Returns tokenized answer spans that better match the annotated answer.

templates/adding_a_new_example_script/utils_xxx.py:406

↓ 1 callersFunction_improve_answer_span

Returns tokenized answer spans that better match the annotated answer.

src/transformers/data/processors/squad.py:25

↓ 1 callersMethod_init_attention_seed

This function sets a new seed for the attention layer to make dropout deterministic for both forward calls: 1 nor

src/transformers/modeling_reformer.py:1091

↓ 1 callersMethod_init_bias

(self, bias)

src/transformers/modeling_transfo_xl.py:469

↓ 1 callersMethod_init_cache

(self, memory_bank, num_layers)

examples/seq2seq/bertabs/modeling_bertabs.py:599

↓ 1 callersMethod_init_feed_forward_seed

This function sets a new seed for the feed forward layer to make dropout deterministic for both forward calls: 1

src/transformers/modeling_reformer.py:1111

↓ 1 callersMethod_init_weight

Identical to the XLM create_sinusoidal_embeddings except features are not interleaved. The cos features are in the 2nd half of the vector.

src/transformers/modeling_bart.py:1260

↓ 1 callersMethod_init_weight

(self, weight)

src/transformers/modeling_transfo_xl.py:463

↓ 1 callersMethod_init_weights

Initialize the weights

src/transformers/modeling_t5.py:577

↓ 1 callersMethod_is_chinese_char

Checks whether CP is the codepoint of a CJK character.

src/transformers/tokenization_bert.py:452

↓ 1 callersFunction_is_end_of_word

Checks whether the last character in text is one of a punctuation, control or whitespace character.

src/transformers/tokenization_utils.py:86

↓ 1 callersFunction_is_start_of_word

Checks whether the first character in text is one of a punctuation, control or whitespace character.

src/transformers/tokenization_utils.py:92

↓ 1 callersFunction_is_whitespace

(c)

src/transformers/data/processors/squad.py:80

↓ 1 callersMethod_len_and_dim_norm

length and attention head size dim normalization

src/transformers/modeling_reformer.py:712

↓ 1 callersMethod_len_norm

length normalization

src/transformers/modeling_reformer.py:722

↓ 1 callersMethod_linear

Computes logits by running inputs through a linear layer. Args: inputs: A float32 tensor with shape [batch_size, length, h

src/transformers/modeling_tf_electra.py:121

↓ 1 callersMethod_linear

Computes logits by running inputs through a linear layer. Args: inputs: A float32 tensor with shape [batch_size, length, e

src/transformers/modeling_tf_albert.py:148

↓ 1 callersMethod_linear

Computes logits by running inputs through a linear layer. Args: inputs: A float32 tensor with shape [batch_size, length, h

src/transformers/modeling_tf_mobilebert.py:200

↓ 1 callersMethod_linear

Computes logits by running inputs through a linear layer. Args: inputs: A float32 tensor with shape [..., hidden_size]

src/transformers/modeling_tf_utils.py:1672

↓ 1 callersMethod_linear

Computes logits by running inputs through a linear layer. Args: inputs: A float32 tensor with shape [batch_size, length, h

src/transformers/modeling_tf_bert.py:197

↓ 1 callersMethod_linear

Computes logits by running inputs through a linear layer. Args: inputs: A float32 tensor with shape [batch_size, length, h

src/transformers/modeling_tf_distilbert.py:173

↓ 1 callersMethod_logit

(x, W, b, proj=None)

src/transformers/modeling_tf_transfo_xl_utilities.py:104

↓ 1 callersFunction_long_tensor

(tok_lst)

tests/test_modeling_mobilebert.py:457

↓ 1 callersMethod_mask_invalid_locations

(self, input_tensor, w)

src/transformers/modeling_longformer.py:152

↓ 1 callersMethod_measure_memory

(self, func: Callable[[], None])

src/transformers/benchmark/benchmark_tf.py:167

↓ 1 callersMethod_measure_speed

(self, func)

src/transformers/benchmark/benchmark_tf.py:152

↓ 1 callersFunction_new_check_is_max_context

Check if this is the 'max context' doc span for the token.

src/transformers/data/processors/squad.py:58

↓ 1 callersMethod_pad

Pad encoded inputs (on left/right and up to predefined legnth or max length in the batch) Args: encoded_inputs: Dictionary of to

src/transformers/tokenization_utils_base.py:1963

↓ 1 callersMethod_pad_to_mult_of_chunk_length

( self, input_ids, inputs_embeds=None, attention_mask=None, position_i

src/transformers/modeling_reformer.py:1645

↓ 1 callersMethod_pad_to_window_size

A helper function to pad tokens and mask to work with implementation of Longformer selfattention.

src/transformers/modeling_longformer.py:539

↓ 1 callersFunction_parse_readme

Get link and metadata from opus model card equivalent.

src/transformers/convert_marian_to_pytorch.py:247

↓ 1 callersMethod_prune_heads

Prunes heads of the model. heads_to_prune: dict of {layer_num: list of heads to prune in this layer} See base class PreTraine

src/transformers/modeling_t5.py:883

↓ 1 callersFunction_quantize

(x, bins)

examples/distillation/grouped_batch_sampler.py:27

↓ 1 callersMethod_rel_shift

(self, x)

src/transformers/modeling_tf_transfo_xl.py:168

↓ 1 callersMethod_rel_shift

(self, x)

src/transformers/modeling_transfo_xl.py:270

↓ 1 callersMethod_relative_position_bucket

Adapted from Mesh Tensorflow: https://github.com/tensorflow/mesh/blob/0cb87fe07da627bf0b7e60475d59f95ed6b5be3d/mesh_tensorflow/transf

src/transformers/modeling_t5.py:232

↓ 1 callersMethod_relative_position_bucket

Adapted from Mesh Tensorflow: https://github.com/tensorflow/mesh/blob/0cb87fe07da627bf0b7e60475d59f95ed6b5be3d/mesh_tensorflow/transf

src/transformers/modeling_tf_t5.py:139

↓ 1 callersFunction_reorder_buffer

(attn_cache, new_order)

src/transformers/modeling_bart.py:554

↓ 1 callersMethod_reorder_cache

(past: Tuple, beam_idx: Tensor)

src/transformers/modeling_utils.py:1614

↓ 1 callersMethod_reorder_cache

(past, beam_idx)

src/transformers/modeling_tf_utils.py:1398

↓ 1 callersMethod_resize_cutoffs

(self, new_num_tokens, new_emb_size, new_embedding_shapes, layer)

src/transformers/modeling_transfo_xl.py:579

↓ 1 callersMethod_resize_final_logits_bias

(self, new_num_tokens: int, old_num_tokens: int)

src/transformers/modeling_bart.py:916

↓ 1 callersMethod_resize_token_embeddings

(self, new_num_tokens)

src/transformers/modeling_utils.py:402

↓ 1 callersMethod_resize_token_embeddings

(self, new_num_tokens)

src/transformers/modeling_tf_utils.py:245

↓ 1 callersMethod_resize_token_embeddings

(self, new_num_tokens, layer=-1)

src/transformers/modeling_transfo_xl.py:568

↓ 1 callersMethod_rotate_checkpoints

(self, use_mtime=False)

src/transformers/trainer.py:671

↓ 1 callersMethod_run_model

Computes the loss of the given features and labels pair. Args: features: the batched features. labels: the batche

src/transformers/trainer_tf.py:463

↓ 1 callersMethod_run_split_on_punc

Splits punctuation on a piece of text.

src/transformers/tokenization_bert.py:417

↓ 1 callersMethod_run_strip_accents

Strips accents from a piece of text.

src/transformers/tokenization_bert.py:406

↓ 1 callersMethod_save

(self, output_dir: Optional[str] = None)

src/transformers/trainer.py:641

↓ 1 callersMethod_save_tpu

(self, output_dir: Optional[str] = None)

src/transformers/trainer.py:625

↓ 1 callersMethod_set_num_buckets

(self, sequence_length)

src/transformers/modeling_reformer.py:559

↓ 1 callersMethod_setup_wandb

Setup the optional Weights & Biases (`wandb`) integration. One can override this method to customize the setup if needed. Find more

src/transformers/trainer_tf.py:144

↓ 1 callersMethod_setup_wandb

Setup the optional Weights & Biases (`wandb`) integration. One can override this method to customize the setup if needed. Find more

src/transformers/trainer.py:310

↓ 1 callersMethod_shape

(self, tensor, dim_0, bsz)

src/transformers/modeling_bart.py:587

↓ 1 callersMethod_skew

Convert diagonals into columns (or columns into diagonals depending on `direction`

src/transformers/modeling_longformer.py:119

↓ 1 callersMethod_skew2

shift every row 1 step to right converting columns into diagonals

src/transformers/modeling_longformer.py:126

↓ 1 callersMethod_sliding_chunks_matmul_pv

Same as _sliding_chunks_matmul_qk but for prob and value tensors. It is expecting the same output format from _sliding_chunks_matmul_qk

src/transformers/modeling_longformer.py:211

↓ 1 callersMethod_sliding_chunks_matmul_qk

Matrix multiplicatio of query x key tensors using with a sliding window attention pattern. This implementation splits the input into overlappi

src/transformers/modeling_longformer.py:164

↓ 1 callersMethod_sorted_checkpoints

(self, checkpoint_prefix=PREFIX_CHECKPOINT_DIR, use_mtime=False)

src/transformers/trainer.py:654

↓ 1 callersMethod_special_token_mask

(self, seq)

src/transformers/tokenization_marian.py:210

↓ 1 callersFunction_strip_spaces

(text)

templates/adding_a_new_example_script/utils_xxx.py:907

↓ 1 callersFunction_strip_spaces

(text)

src/transformers/data/metrics/squad_metrics.py:270

↓ 1 callersMethod_tensorize_batch

(self, examples: List[torch.Tensor])

src/transformers/data/data_collator.py:89

↓ 1 callersFunction_tf_glue_convert_examples_to_features

Returns: A ``tf.data.Dataset`` containing the task-specific features.

src/transformers/data/processors/glue.py:70

↓ 1 callersMethod_tokenize

Converts a string in a sequence of tokens (string), using the tokenizer. Split in words for word-based vocabulary or sub-words for sub-wo

src/transformers/tokenization_utils.py:367

↓ 1 callersMethod_tokenize_chinese_chars

Adds whitespace around any CJK character.

src/transformers/tokenization_bert.py:439

↓ 1 callersFunction_tokens_match

(prev_tokens, tokens)

src/transformers/modeling_utils.py:1644

↓ 1 callersFunction_tokens_match

(prev_tokens, tokens)

src/transformers/modeling_tf_utils.py:1442

↓ 1 callersMethod_training_step

( self, model: nn.Module, inputs: Dict[str, Union[torch.Tensor, Any]], optimizer: torch.optim.Optimize

src/transformers/trainer.py:572

↓ 1 callersMethod_training_steps

Returns a generator over training steps (i.e. parameters update).

src/transformers/trainer_tf.py:397

↓ 1 callersFunction_truncate_seq_pair

Truncates a sequence pair in place to the maximum length.

examples/contrib/run_swag.py:190

↓ 1 callersMethod_update_mems

(self, hids, mems, mlen, qlen)

src/transformers/modeling_tf_transfo_xl.py:501

↓ 1 callersMethod_update_mems

(self, hids, mems, mlen, qlen)

src/transformers/modeling_transfo_xl.py:729

↓ 1 callersMethod_update_rate

(self, stack)

examples/seq2seq/bertabs/modeling_bertabs.py:1012

↓ 1 callersMethod_use_saved_state

(self, k, v, saved_state, key_padding_mask, static_kv, bsz)

src/transformers/modeling_bart.py:675

↓ 1 callersMethod_write_logs

( self, trainer: pl.Trainer, pl_module: pl.LightningModule, type_path: str, save_generations=True

examples/seq2seq/callbacks.py:23

↓ 1 callersFunctionacc_and_f1

(preds, labels)

src/transformers/data/metrics/__init__.py:35

← previous next →601–700 of 4,119, ranked by callers