Functions — github.com/huggingface/transformers

hub / github.com/huggingface/transformers / functions

Functions4,119 in github.com/huggingface/transformers

⨍ Functions 4,119◇ Types & classes 939↳ Endpoints 4

↓ 3 callersFunctionread_examples_from_file

(data_dir, mode: Union[Split, str])

examples/token-classification/utils_ner.py:233

↓ 3 callersMethodsave_checkpoint

Save the current state. Only by the master process.

examples/distillation/distiller.py:594

↓ 3 callersFunctionsave_json

(content: Union[Dict, List], path: str)

src/transformers/convert_marian_to_pytorch.py:532

↓ 3 callersMethodsave_vocabulary

Save the sentencepiece vocabulary (copy original file) and special tokens file to a directory.

src/transformers/tokenization_t5.py:196

↓ 3 callersFunctionset_tensor_by_indices_to_value

(tensor, indices, value)

src/transformers/modeling_tf_utils.py:1535

↓ 3 callersMethodshape

projection

src/transformers/modeling_t5.py:330

↓ 3 callersFunctionsimple_accuracy

(preds, labels)

src/transformers/data/metrics/__init__.py:32

↓ 3 callersFunctionsquad_evaluate

(examples, preds, no_answer_probs=None, no_answer_probability_threshold=1.0)

src/transformers/data/metrics/squad_metrics.py:211

↓ 3 callersFunctiontf_top_k_top_p_filtering

Filter a distribution of logits using top-k and/or nucleus (top-p) filtering Args: logits: logits distribution shape (batch size,

src/transformers/modeling_tf_utils.py:1475

↓ 3 callersMethodtie_weights

Tie the weights between the input embeddings and the output embeddings. If the `torchscript` flag is set in the configuration, can't

src/transformers/modeling_utils.py:347

↓ 3 callersMethodto_diff_dict

Removes all attributes from config which correspond to the default config attributes for better readability and serializes to a Pytho

src/transformers/configuration_utils.py:334

↓ 3 callersFunctiontop_k_top_p_filtering

Filter a distribution of logits using top-k and/or nucleus (top-p) filtering Args: logits: logits distribution shape (batch size,

src/transformers/modeling_utils.py:1677

↓ 3 callersFunctiontrim_batch

Remove columns that are populated exclusively by pad_token_id

examples/seq2seq/utils.py:68

↓ 3 callersFunctionuse_task_specific_params

(model, task)

examples/seq2seq/utils.py:175

↓ 3 callersMethodwhoami

Call HF API to know "whoami"

src/transformers/hf_api.py:118

↓ 2 callersFunctionF

(e,t)

docs/source/_static/js/custom.js:246

↓ 2 callersMethod__call__

(self, *args, **kwargs)

src/transformers/pipelines.py:472

↓ 2 callersMethod__init__

(self, config, transformer, encoder)

src/transformers/modeling_mmbt.py:182

↓ 2 callersMethod_check_generated_ids

(self, output_ids)

tests/test_modeling_common.py:775

↓ 2 callersMethod_check_generated_ids

(self, output_ids)

tests/test_modeling_tf_common.py:653

↓ 2 callersMethod_check_match_tokens

(self, generated_ids, bad_words_ids)

tests/test_modeling_common.py:780

↓ 2 callersMethod_check_match_tokens

(self, generated_ids, bad_words_ids)

tests/test_modeling_tf_common.py:658

↓ 2 callersFunction_compute_global_attention_mask

Computes global attention mask by putting attention on all tokens before `sep_token_id` if `before_sep_token is True` else after

src/transformers/modeling_longformer.py:62

↓ 2 callersMethod_compute_logit

(self, hidden, weight, bias, proj)

src/transformers/modeling_transfo_xl_utilities.py:72

↓ 2 callersFunction_compute_softmax

Compute softmax probability over raw logits.

templates/adding_a_new_example_script/utils_xxx.py:985

↓ 2 callersFunction_compute_softmax

Compute softmax probability over raw logits.

src/transformers/data/metrics/squad_metrics.py:348

↓ 2 callersMethod_create_examples

(self, input_data, set_type)

src/transformers/data/processors/squad.py:556

↓ 2 callersMethod_create_examples

Creates examples for the training and dev sets.

examples/adversarial/utils_hans.py:261

↓ 2 callersFunction_create_next_token_logits_penalties

(input_ids, logits, repetition_penalty)

src/transformers/modeling_tf_utils.py:1402

↓ 2 callersMethod_decay_weights_op

(self, var, learning_rate, apply_state)

src/transformers/optimization_tf.py:138

↓ 2 callersMethod_dict_from_json_file

(cls, json_file: str)

src/transformers/configuration_utils.py:323

↓ 2 callersFunction_dump_articles

(path: Path, articles: list)

examples/seq2seq/test_seq2seq_examples.py:80

↓ 2 callersMethod_eval_end

Evaluation called for both Val and Test

examples/token-classification/run_pl_ner.py:108

↓ 2 callersMethod_eval_end

(self, outputs)

examples/text-classification/run_pl_glue.py:111

↓ 2 callersMethod_force_token_ids_generation

force one of token_ids to be generated by setting prob of all other tokens to 0

src/transformers/modeling_bart.py:1035

↓ 2 callersMethod_forward

Internal framework specific forward dispatching. Args: inputs: dict holding all the keyworded arguments for required by t

src/transformers/pipelines.py:476

↓ 2 callersMethod_generate_random_bad_tokens

(self, num_bad_tokens: int, config)

tests/test_modeling_common.py:764

↓ 2 callersMethod_generate_random_bad_tokens

(self, num_bad_tokens, model)

tests/test_modeling_tf_common.py:635

↓ 2 callersMethod_generative_step

(self, batch: dict)

examples/seq2seq/finetune.py:159

↓ 2 callersMethod_get_attn_mask

(self)

tests/test_modeling_reformer.py:697

↓ 2 callersFunction_get_least_common_mult_chunk_len

(config)

src/transformers/modeling_reformer.py:78

↓ 2 callersMethod_get_lr

Retrieves the learning rate with the given state.

src/transformers/optimization_tf.py:151

↓ 2 callersMethod_get_word_embeddings

(self, embeddings)

src/transformers/modeling_tf_utils.py:256

↓ 2 callersMethod_init_weights

(self, module)

src/transformers/modeling_bart.py:140

↓ 2 callersMethod_log

(self, logs: Dict[str, float])

src/transformers/trainer_tf.py:250

↓ 2 callersMethod_measure_memory

(self, func: Callable[[], None])

src/transformers/benchmark/benchmark.py:214

↓ 2 callersMethod_measure_speed

(self, func)

src/transformers/benchmark/benchmark.py:192

↓ 2 callersMethod_merge_hidden_size_dims

merges attn_head_size dim and num_attn_heads dim into hidden_size

src/transformers/modeling_reformer.py:275

↓ 2 callersMethod_prediction_loop

Prediction/evaluation loop, shared by `evaluate()` and `predict()`. Works both with or without labels.

src/transformers/trainer_tf.py:181

↓ 2 callersMethod_prediction_loop

Prediction/evaluation loop, shared by `evaluate()` and `predict()`. Works both with or without labels.

src/transformers/trainer.py:726

↓ 2 callersMethod_prepare_for_model

Prepares a sequence of input id, or a pair of sequences of inputs ids so that it can be used by the model. It adds special tokens, truncates

src/transformers/tokenization_utils.py:624

↓ 2 callersMethod_prepare_inference_func

(self, model_name: str, batch_size: int, sequence_length: int)

src/transformers/benchmark/benchmark.py:85

↓ 2 callersMethod_prepare_inference_func

(self, model_name: str, batch_size: int, sequence_length: int)

src/transformers/benchmark/benchmark_tf.py:116

↓ 2 callersMethod_prepare_train_func

(self, model_name: str, batch_size: int, sequence_length: int)

src/transformers/benchmark/benchmark.py:138

↓ 2 callersMethod_setup_normalizer

(self)

src/transformers/tokenization_marian.py:83

↓ 2 callersMethod_shift_right

(self, input_ids)

src/transformers/modeling_t5.py:609

↓ 2 callersMethod_split_hidden_size_dim

splits hidden_size dim into attn_head_size and num_attn_heads

src/transformers/modeling_reformer.py:267

↓ 2 callersMethod_step

(self, batch: dict)

examples/seq2seq/finetune.py:120

↓ 2 callersMethod_test_infer_dynamic_axis

(self, model, tokenizer, framework)

tests/test_onnx.py:89

↓ 2 callersMethod_test_qa_pipeline

(self, nlp)

tests/test_pipelines.py:338

↓ 2 callersMethod_use_cache

During generation, decide whether to pass the `past` variable to the next forward pass.

src/transformers/modeling_utils.py:810

↓ 2 callersMethod_use_cache

During generation, decide whether to pass the `past` variable to the next forward pass.

src/transformers/modeling_tf_utils.py:544

↓ 2 callersMethodadd_examples

( self, texts_or_text_and_labels, labels=None, ids=None, overwrite_labels=False, overwrite_examples=Fa

src/transformers/data/processors/utils.py:194

↓ 2 callersMethodadd_symbol

(self, sym)

src/transformers/tokenization_transfo_xl.py:270

↓ 2 callersFunctionalign_predictions

(predictions: np.ndarray, label_ids: np.ndarray)

examples/token-classification/run_tf_ner.py:179

↓ 2 callersFunctionalign_predictions

(predictions: np.ndarray, label_ids: np.ndarray)

examples/token-classification/run_ner.py:192

↓ 2 callersFunctionany_requires_grad

(model: nn.Module)

examples/seq2seq/utils.py:248

↓ 2 callersFunctionapply_chunking_to_forward

This function chunks the `input_tensors` into smaller input tensor parts of size `chunk_size` over the dimension `chunk_dim`. It then applies

src/transformers/modeling_utils.py:2159

↓ 2 callersMethodapply_gradients

(self, grads_and_vars, name=None)

src/transformers/optimization_tf.py:147

↓ 2 callersMethodassertListAlmostEqual

(self, list1, list2, tol)

tests/test_optimization.py:90

↓ 2 callersMethodassert_add_special_tokens

(self, tokenizer_r)

tests/test_tokenization_fast.py:682

↓ 2 callersFunctionassert_all_frozen

(model)

examples/seq2seq/utils.py:252

↓ 2 callersMethodassert_embeded_special_tokens

(self, tokenizer_r, tokenizer_p)

tests/test_tokenization_fast.py:663

↓ 2 callersMethodassert_max_length_equal

(self, tokenizer_r, tokenizer_p)

tests/test_tokenization_fast.py:224

↓ 2 callersMethodassert_num_special_tokens_to_add_equal

(self, tokenizer_r, tokenizer_p)

tests/test_tokenization_fast.py:219

↓ 2 callersMethodassert_outputs_same

(self, after_outputs, outputs)

tests/test_modeling_tf_common.py:174

↓ 2 callersMethodassert_pretokenized_inputs

(self, tokenizer_r, tokenizer_p)

tests/test_tokenization_fast.py:339

↓ 2 callersMethodassert_special_tokens_map_equal

(self, tokenizer_r, tokenizer_p)

tests/test_tokenization_fast.py:229

↓ 2 callersMethodassert_tokenization_python_rust_equals

(self, tokenizer_r, tokenizer_p)

tests/test_tokenization_fast.py:187

↓ 2 callersFunctionb

(e)

docs/source/_static/js/custom.js:246

↓ 2 callersMethodbold

(cls, s)

src/transformers/commands/user.py:57

↓ 2 callersMethodbpe

(self, token)

src/transformers/tokenization_xlm.py:705

↓ 2 callersMethodbuild

(self, input_shape)

src/transformers/modeling_tf_mobilebert.py:69

↓ 2 callersMethodbuild

(self, input_shape)

src/transformers/modeling_tf_utils.py:1599

↓ 2 callersMethodbuild_inputs_with_special_tokens

Build model inputs from a sequence or a pair of sequence for sequence classification tasks by concatenating and adding special tokens

src/transformers/tokenization_utils.py:809

↓ 2 callersMethodbuild_vocab

(self)

src/transformers/tokenization_transfo_xl.py:213

↓ 2 callersFunctionbytes_to_mega_bytes

Utility to convert a number of bytes (int) into a number of mega bytes (int)

src/transformers/benchmark/benchmark_utils.py:550

↓ 2 callersFunctioncalc_banned_bad_words_ids

(prev_input_ids, bad_words_ids)

src/transformers/modeling_tf_utils.py:1439

↓ 2 callersFunctioncalc_banned_ngram_tokens

(prev_input_ids, num_hypos, no_repeat_ngram_size, cur_len)

src/transformers/modeling_tf_utils.py:1416

↓ 2 callersMethodcalc_ce_loss

(self, mask, s_logits, t_logits)

examples/seq2seq/distillation.py:134

↓ 2 callersMethodcalc_hidden_loss

(self, attention_mask, hidden_states, hidden_states_T, matches)

examples/seq2seq/distillation.py:251

↓ 2 callersMethodcalc_mse_loss

(self, teacher_outputs: torch.Tensor, student_outputs: torch.Tensor, mask)

examples/seq2seq/distillation.py:123

↓ 2 callersMethodcall

(self, x, training=False)

src/transformers/modeling_tf_gpt2.py:178

↓ 2 callersFunctioncast_marian_config

(raw_cfg: Dict[str, str])

src/transformers/convert_marian_to_pytorch.py:76

↓ 2 callersMethodcheck_loss_output

(self, result)

tests/test_modeling_xlm.py:126

↓ 2 callersMethodclean_up_tokenization

Clean up a list of simple English tokenization artifacts like spaces before punctuations and abreviated forms.

src/transformers/tokenization_utils_base.py:2082

↓ 2 callersFunctioncompute_metrics

(p: EvalPrediction)

examples/text-classification/run_tf_glue.py:186

↓ 2 callersFunctioncompute_token_type_ids

Segment embeddings as described in [1] The values {0,1} were found in the repository [2]. Attributes: batch: torch.Tensor, size [ba

examples/seq2seq/bertabs/utils_summarization.py:143

↓ 2 callersFunctionconvert

( framework: str, model: str, output: str, opset: int, tokenizer: Optional[str] = None,

src/transformers/convert_graph_to_onnx.py:187

↓ 2 callersMethodconvert_batch_encode_plus_format_to_encode_plus

(batch_encode_plus_sequences)

tests/test_tokenization_common.py:116

← previous next →301–400 of 4,119, ranked by callers