Functions — github.com/huggingface/transformers

hub / github.com/huggingface/transformers / functions

Functions4,119 in github.com/huggingface/transformers

⨍ Functions 4,119◇ Types & classes 939↳ Endpoints 4

↓ 1 callersFunctionaccuracy

(out, labels)

examples/contrib/run_swag.py:207

↓ 1 callersFunctionaccuracy

(out, labels)

examples/contrib/run_openai_gpt.py:57

↓ 1 callersMethodadaptive_forward

(self, hidden_states, current_layer, attention_mask=None, head_mask=None)

examples/bert-loses-patience/pabee/modeling_pabee_albert.py:37

↓ 1 callersMethodadaptive_forward

(self, hidden_states, current_layer, attention_mask=None, head_mask=None)

examples/bert-loses-patience/pabee/modeling_pabee_bert.py:39

↓ 1 callersMethodadd

Add a new hypothesis to the list.

src/transformers/modeling_tf_utils.py:1559

↓ 1 callersFunctionaddCustomFooter

()

docs/source/_static/js/custom.js:39

↓ 1 callersFunctionaddGithubButton

()

docs/source/_static/js/custom.js:71

↓ 1 callersFunctionaddHfMenu

()

docs/source/_static/js/custom.js:146

↓ 1 callersFunctionaddIcon

()

docs/source/_static/js/custom.js:24

↓ 1 callersFunctionaddVersionControl

()

docs/source/_static/js/custom.js:84

↓ 1 callersFunctionadd_emb_entries

(wemb, final_bias, n_special_tokens=1)

src/transformers/convert_marian_to_pytorch.py:55

↓ 1 callersMethodadd_examples_from_csv

( self, file_name, split_name="", column_label=0, column_text=1,

src/transformers/data/processors/utils.py:164

↓ 1 callersMethodadd_model_specific_args

(parser, root_dir)

examples/seq2seq/distillation.py:178

↓ 1 callersMethodadd_model_specific_args

(parser, root_dir)

examples/seq2seq/finetune.py:232

↓ 1 callersMethodadd_model_specific_args

(parser, root_dir)

examples/token-classification/run_pl_ner.py:154

↓ 1 callersMethodadd_model_specific_args

(parser, root_dir)

examples/text-classification/run_pl_glue.py:142

↓ 1 callersMethodadd_special

(self, sym)

src/transformers/tokenization_transfo_xl.py:264

↓ 1 callersFunctionadd_special_tokens_to_vocab

(model_dir: Path)

src/transformers/convert_marian_to_pytorch.py:291

↓ 1 callersFunctionadd_to_vocab_

(vocab: Dict[str, int], special_tokens: List[str])

src/transformers/convert_marian_to_pytorch.py:276

↓ 1 callersFunctionadjust_length_to_model

(length, max_sequence_length)

examples/text-generation/run_generation.py:140

↓ 1 callersMethodadjust_logits_during_generation

(self, logits, **kwargs)

src/transformers/modeling_utils.py:807

↓ 1 callersFunctionangle_defn

(pos, i, d_model_size)

src/transformers/modeling_ctrl.py:41

↓ 1 callersFunctionangle_defn

(pos, i, d_model_size)

src/transformers/modeling_tf_ctrl.py:46

↓ 1 callersFunctionanswer_question

( question_doc, s2s_model, s2s_tokenizer, min_len=64, max_len=256, sampling=False, n_beams=2, top_p=0.95,

examples/longform-qa/eli5_app.py:108

↓ 1 callersFunctionapply_no_ans_threshold

(scores, na_probs, qid_to_has_ans, na_prob_thresh)

src/transformers/data/metrics/squad_metrics.py:96

↓ 1 callersMethodassertListAlmostEqual

(self, list1, list2, tol)

tests/test_optimization.py:64

↓ 1 callersMethodassert_add_tokens

(self, tokenizer_r)

tests/test_tokenization_fast.py:235

↓ 1 callersMethodassert_alignement_methods

(self, tokenizer_r)

tests/test_tokenization_fast.py:109

↓ 1 callersMethodassert_batch_encode_dynamic_overflowing

When calling batch_encode with multiple sequence it can returns different number of overflowing encoding for each sequence: [

tests/test_tokenization_fast.py:283

↓ 1 callersMethodassert_create_token_type_ids

(self, tokenizer_r, tokenizer_p)

tests/test_tokenization_fast.py:399

↓ 1 callersMethodassert_offsets_mapping

(self, tokenizer_r)

tests/test_tokenization_fast.py:253

↓ 1 callersMethodassert_offsets_with_special_characters

(self, tokenizer_r)

tests/test_tokenization_fast.py:734

↓ 1 callersMethodassert_padding

(self, tokenizer_r, tokenizer_p, max_length=15)

tests/test_tokenization_fast.py:442

↓ 1 callersMethodassert_padding

(self, tokenizer_r, tokenizer_p, max_length=15)

tests/test_tokenization_fast.py:833

↓ 1 callersMethodbackward_pass

( self, next_attn_output, hidden_states, grad_attn_output, grad_hidden

src/transformers/modeling_reformer.py:1175

↓ 1 callersMethodbpe

(self, token)

src/transformers/tokenization_openai.py:131

↓ 1 callersMethodbpe

(self, token)

src/transformers/tokenization_gpt2.py:180

↓ 1 callersMethodbpe

(self, token)

src/transformers/tokenization_ctrl.py:160

↓ 1 callersMethodbuild

Build shared word embedding layer

src/transformers/modeling_tf_electra.py:62

↓ 1 callersMethodbuild

(self, input_shape)

src/transformers/modeling_tf_albert.py:489

↓ 1 callersMethodbuild

Build shared word embedding layer

src/transformers/modeling_tf_bert.py:138

↓ 1 callersMethodbuild

Build shared word embedding layer

src/transformers/modeling_tf_distilbert.py:106

↓ 1 callersMethodbuild

(self, input_shape)

src/transformers/modeling_tf_xlnet.py:359

↓ 1 callersFunctionbuild_bows_one_hot_vectors

(bow_indices, tokenizer, device="cuda")

examples/text-generation/pplm/run_pplm.py:302

↓ 1 callersFunctionbuild_compute_metrics_fn

(task_name: str)

examples/text-classification/run_glue.py:151

↓ 1 callersFunctionbuild_data_iterator

(args, tokenizer)

examples/seq2seq/bertabs/run_summarization.py:185

↓ 1 callersMethodbuild_inputs_with_special_tokens

Build model inputs from a sequence or a pair of sequence for sequence classification tasks by concatenating and adding special tokens

src/transformers/tokenization_albert.py:235

↓ 1 callersMethodbuild_inputs_with_special_tokens

Build model inputs from a sequence or a pair of sequence for sequence classification tasks by concatenating and adding special tokens

src/transformers/tokenization_xlm.py:852

↓ 1 callersFunctionbuild_predictor

(args, tokenizer, symbols, model, logger=None)

examples/seq2seq/bertabs/modeling_bertabs.py:662

↓ 1 callersFunctionbuild_shape_dict

(name: str, tensor, is_input: bool, seq_len: int)

src/transformers/convert_graph_to_onnx.py:70

↓ 1 callersFunctionbuild_tf_to_pytorch_map

A map of modules from TF to PyTorch. This time I use a map to keep the PyTorch model as identical to the original PyTorch model as possible.

src/transformers/modeling_transfo_xl.py:45

↓ 1 callersFunctionbuild_tf_xlnet_to_pytorch_map

A map of modules from TF to PyTorch. I use a map to keep the PyTorch model as identical to the original PyTorch model as possible.

src/transformers/modeling_xlnet.py:44

↓ 1 callersFunctionbytes_to_unicode

Returns list of utf-8 byte and a mapping to unicode strings. We specifically avoids mapping to whitespace/control characters the bpe code bar

src/transformers/tokenization_gpt2.py:65

↓ 1 callersMethodcache_mem

(self, curr_out, prev_mem)

src/transformers/modeling_xlnet.py:691

↓ 1 callersMethodcache_mem

cache hidden states into memory.

src/transformers/modeling_tf_xlnet.py:443

↓ 1 callersFunctioncalc_banned_bad_words_ids

(prev_input_ids: Iterable[int], bad_words_ids: Iterable[int])

src/transformers/modeling_utils.py:1641

↓ 1 callersFunctioncalc_banned_ngram_tokens

Copied from fairseq for no_repeat_ngram in beam_search

src/transformers/modeling_utils.py:1618

↓ 1 callersMethodcalc_generative_metrics

(self, preds, target)

examples/seq2seq/finetune.py:156

↓ 1 callersFunctioncalculate_bleu_score

Uses sacrebleu's corpus_bleu implementation.

examples/seq2seq/utils.py:63

↓ 1 callersFunctioncalculate_rouge

(output_lns: List[str], reference_lns: List[str])

examples/seq2seq/utils.py:227

↓ 1 callersMethodcall

( self, hidden_states, attention_mask=None, position_bias=None, encode

src/transformers/modeling_tf_t5.py:400

↓ 1 callersFunctioncan_convert_to_float

(string)

examples/benchmarking/plot_csv_file.py:57

↓ 1 callersFunctioncan_convert_to_int

(string)

examples/benchmarking/plot_csv_file.py:49

↓ 1 callersMethodcausal_attention_mask

1's in the lower triangle, counting from the lower right corner. Same as tf.matrix_band_part(tf.ones([nd, ns]), -1, ns-nd), but doesn't produc

src/transformers/modeling_tf_openai.py:95

↓ 1 callersMethodcausal_attention_mask

1's in the lower triangle, counting from the lower right corner. Same as tf.matrix_band_part(tf.ones([nd, ns]), -1, ns-nd), but doesn't produc

src/transformers/modeling_tf_gpt2.py:88

↓ 1 callersMethodchar_to_token

Get the index of the token in the encoded output comprising a character in the original string for a sequence of the batch.

src/transformers/tokenization_utils_base.py:362

↓ 1 callersMethodchar_to_word

Get the word in the original string corresponding to a character in the original string of a sequence of the batch. Can be c

src/transformers/tokenization_utils_base.py:439

↓ 1 callersMethodcheck

Some sanity checks

examples/distillation/lm_seqs_dataset.py:55

↓ 1 callersFunctioncheck_all_links

Check that the provided links are valid. Links are considered valid if a HEAD request to the server returns a 200 status code.

utils/link_tester.py:57

↓ 1 callersMethodcheck_cutoffs_and_n_token

( self, copied_cutoffs, layer, model_embed, model, model_class, resized_value, vocab_size )

tests/test_modeling_transfo_xl.py:172

↓ 1 callersFunctioncheck_equal

(marian_cfg, k1, k2)

src/transformers/convert_marian_to_pytorch.py:309

↓ 1 callersMethodcheck_loss_output

(self, result)

tests/test_modeling_t5.py:96

↓ 1 callersMethodcheck_loss_output

(self, loss)

tests/test_modeling_encoder_decoder.py:256

↓ 1 callersMethodcheck_loss_output

(self, result)

tests/test_modeling_reformer.py:165

↓ 1 callersMethodcheck_loss_output

(self, result)

tests/test_modeling_flaubert.py:123

↓ 1 callersFunctioncheck_marian_cfg_assumptions

(marian_cfg)

src/transformers/convert_marian_to_pytorch.py:314

↓ 1 callersMethodcheck_prepare_lm_labels_via_shift_left

( self, config, input_ids, decoder_input_ids, attention_mask, decoder_attention_mask, lm_labels, )

tests/test_modeling_t5.py:99

↓ 1 callersMethodcheck_tokenizer_from_pretrained

(self, tokenizer_class)

tests/test_tokenization_utils.py:26

↓ 1 callersMethodcheck_transfo_xl_lm_head_output

(self, result)

tests/test_modeling_transfo_xl.py:137

↓ 1 callersMethodcheck_transfo_xl_model_output

(self, result)

tests/test_modeling_transfo_xl.py:101

↓ 1 callersFunctionchunks

Yield successive n-sized chunks from lst.

examples/seq2seq/run_eval.py:19

↓ 1 callersFunctioncollate

Collate formats the data passed to the data loader. In particular we tokenize the data batch after batch to avoid keeping them all in memory

examples/seq2seq/bertabs/run_summarization.py:202

↓ 1 callersMethodcompute_bias

Compute binned relative position bias

src/transformers/modeling_t5.py:279

↓ 1 callersMethodcompute_bias

Compute binned relative position bias

src/transformers/modeling_tf_t5.py:184

↓ 1 callersFunctioncompute_exact

(a_gold, a_pred)

src/transformers/data/metrics/squad_metrics.py:50

↓ 1 callersFunctioncompute_f1

(a_gold, a_pred)

src/transformers/data/metrics/squad_metrics.py:54

↓ 1 callersFunctioncompute_metrics

(p: EvalPrediction)

examples/multiple-choice/run_multiple_choice.py:179

↓ 1 callersFunctioncompute_metrics

(p: EvalPrediction)

examples/token-classification/run_ner.py:208

↓ 1 callersFunctionconvert_all_pt_checkpoints_to_tf

( args_model_type, tf_dump_path, model_shortcut_names_or_path=None, config_shortcut_names_or_p

src/transformers/convert_pytorch_checkpoint_to_tf2.py:233

↓ 1 callersFunctionconvert_bart_checkpoint

Copy/paste/tweak model's weights to our BERT structure.

src/transformers/convert_bart_original_pytorch_checkpoint_to_pytorch.py:95

↓ 1 callersFunctionconvert_bertabs_checkpoints

Copy/paste and tweak the pre-trained weights provided by the creators of BertAbs for the internal architecture.

examples/seq2seq/bertabs/convert_bertabs_original_pytorch_checkpoint.py:64

↓ 1 callersFunctionconvert_dialogpt_checkpoint

(checkpoint_path: str, pytorch_dump_folder_path: str)

src/transformers/convert_dialogpt_original_pytorch_checkpoint_to_pytorch.py:15

↓ 1 callersFunctionconvert_encoder_layer

(opus_dict, layer_prefix: str, converter: dict)

src/transformers/convert_marian_to_pytorch.py:24

↓ 1 callersFunctionconvert_examples_to_features

Loads a data file into a list of `InputBatch`s.

templates/adding_a_new_example_script/utils_xxx.py:187

↓ 1 callersFunctionconvert_longformer_qa_checkpoint_to_pytorch

( longformer_model: str, longformer_question_answering_ckpt_path: str, pytorch_dump_folder_path: str )

src/transformers/convert_longformer_original_pytorch_lightning_to_pytorch.py:38

↓ 1 callersFunctionconvert_pt_checkpoint_to_tf

( model_type, pytorch_checkpoint_path, config_file, tf_dump_path, compare_with_pt_model=False, use_cached_

src/transformers/convert_pytorch_checkpoint_to_tf2.py:187

↓ 1 callersFunctionconvert_pytorch

(nlp: Pipeline, opset: int, output: str, use_external_format: bool)

src/transformers/convert_graph_to_onnx.py:134

↓ 1 callersFunctionconvert_pytorch_checkpoint_to_tf

:param model:BertModel Pytorch model instance to be converted :param ckpt_dir: Tensorflow model directory :param model_name: model name

src/transformers/convert_bert_pytorch_checkpoint_to_original_tf.py:28

↓ 1 callersFunctionconvert_roberta_checkpoint_to_pytorch

Copy/paste/tweak roberta's weights to our BERT structure.

src/transformers/convert_roberta_original_pytorch_checkpoint_to_pytorch.py:42

↓ 1 callersFunctionconvert_tensorflow

(nlp: Pipeline, opset: int, output: str)

src/transformers/convert_graph_to_onnx.py:161

← previous next →701–800 of 4,119, ranked by callers