Functions — github.com/LiteSSLHub/DisCo

CodeHub MCPcopy Create free account

hub / github.com/LiteSSLHub/DisCo / functions

Functions201 in github.com/LiteSSLHub/DisCo

⨍ Functions 201◇ Types & classes 44

↓ 23 callersMethod__init__

src/pytorch_pretrained_bert/modeling.py:346

↓ 20 callersMethodto

src/data.py:498

↓ 8 callersMethodfrom_pretrained

Instantiate a PreTrainedBertModel from a pre-trained model file. Download and cache the pre-trained model file if needed.

src/pytorch_pretrained_bert/tokenization.py:120

↓ 6 callersFunctionevaluate

(model, dataloader, specific_student, extract_nsents, device, num_labels=2, task_name=None, pyrouge=True,

src/evaluate.py:152

↓ 6 callersFunctionload_checkpoints

(filename, device)

src/utils.py:78

↓ 6 callersMethodstep

Performs a single optimization step. Arguments: closure (callable, optional): A closure that reevaluates the model

src/pytorch_pretrained_bert/optimization.py:94

↓ 4 callersFunctionget_logger

Initializes multi-GPU-friendly python command line logger.

src/utils.py:13

↓ 4 callersFunctiontrans_view

src/train.py:109

↓ 3 callersMethod__init__

(self, model_name_or_path, task_name, student_num=2, num_classes=1, aug_num=11)

src/model.py:115

↓ 3 callersMethodconvert_tokens_to_ids

Converts a sequence of tokens into ids using the vocab.

src/pytorch_pretrained_bert/tokenization.py:99

↓ 3 callersMethodfrom_json_file

Constructs a `BertConfig` from a json file of parameters.

src/pytorch_pretrained_bert/modeling.py:163

↓ 3 callersMethodload

(module, prefix='')

src/pytorch_pretrained_bert/modeling.py:595

↓ 3 callersFunctionread_jsonl

src/utils.py:21

↓ 3 callersFunctiontext_clean

src/evaluate.py:20

↓ 3 callersMethodto_json_file

Save this instance to a json file.

src/pytorch_pretrained_bert/modeling.py:181

↓ 3 callersMethodtrain

(self, local_rank, args)

src/general_distill.py:127

↓ 3 callersMethodtranspose_for_scores

src/pytorch_pretrained_bert/modeling.py:258

↓ 3 callersFunctionwhitespace_tokenize

Runs basic whitespace cleaning and splitting on a peice of text.

src/pytorch_pretrained_bert/tokenization.py:66

↓ 2 callersFunctionadversary

(batch, model, student_num, device, label=None, task_name=None)

src/train.py:201

↓ 2 callersFunctioncached_path

Given something that might be a URL (or might be a local path), determine which. If it's a URL, download the file and cache it, and retur

src/pytorch_pretrained_bert/file_utils.py:74

↓ 2 callersFunctionget_decay_parameter

src/train.py:220

↓ 2 callersFunctionget_decay_parameter

src/general_distill.py:58

↓ 2 callersFunctionget_ngrams

src/evaluate.py:24

↓ 2 callersFunctionget_optimizer_and_scheduler

src/train.py:214

↓ 2 callersMethodpredict

(self, batch, specific_student, ensemble_prediction=True)

src/model.py:136

↓ 2 callersMethodprepare

(self, supervised_dataset_size=None)

src/data.py:169

↓ 2 callersFunctionsave_checkpoints

(filename, ckpt)

src/utils.py:69

↓ 2 callersFunctionsplit_s3_path

Split a full s3 path into the bucket name and path.

src/pytorch_pretrained_bert/file_utils.py:104

↓ 2 callersMethodtest_dataloader

src/data.py:321

↓ 2 callersMethodto_json_string

Serializes this instance to a JSON string.

src/pytorch_pretrained_bert/modeling.py:177

↓ 2 callersMethodtokenize

src/pytorch_pretrained_bert/tokenization.py:92

↓ 2 callersMethodtrain_dataloader

src/data.py:288

↓ 2 callersMethodval_dataloader

src/data.py:310

↓ 1 callersMethod_clean_text

Performs invalid character removal and whitespace cleanup on text.

src/pytorch_pretrained_bert/tokenization.py:261

↓ 1 callersMethod_is_chinese_char

Checks whether CP is the codepoint of a CJK character.

src/pytorch_pretrained_bert/tokenization.py:239

↓ 1 callersFunction_is_control

Checks whether `chars` is a control character.

src/pytorch_pretrained_bert/tokenization.py:347

↓ 1 callersFunction_is_punctuation

Checks whether `chars` is a punctuation character.

src/pytorch_pretrained_bert/tokenization.py:359

↓ 1 callersFunction_is_whitespace

Checks whether `chars` is a whitespace character.

src/pytorch_pretrained_bert/tokenization.py:335

↓ 1 callersMethod_precalculate_doc_weights

src/pregenerate_training_data.py:67

↓ 1 callersMethod_run_split_on_punc

Splits punctuation on a piece of text.

src/pytorch_pretrained_bert/tokenization.py:204

↓ 1 callersMethod_run_strip_accents

Strips accents from a piece of text.

src/pytorch_pretrained_bert/tokenization.py:193

↓ 1 callersMethod_tokenize_chinese_chars

Adds whitespace around any CJK character.

src/pytorch_pretrained_bert/tokenization.py:226

↓ 1 callersMethodadd_document

(self, document)

src/pregenerate_training_data.py:57

↓ 1 callersFunctioncalc_rouge_from_pyrouge

(hypothesis, references)

src/metrics.py:19

↓ 1 callersFunctioncalc_rouge_from_python_implementation

(hypothesis, references)

src/metrics.py:65

↓ 1 callersFunctionconsist_train

src/train.py:259

↓ 1 callersFunctionconvert_example_to_features

(example, tokenizer, max_seq_length)

↓ 1 callersFunctionconvert_examples_to_features

(examples, label_list, max_seq_length, tokenizer)

↓ 1 callersFunctionconvert_tf_checkpoint_to_pytorch

(tf_checkpoint_path, bert_config_file, pytorch_dump_path)

src/pytorch_pretrained_bert/convert_tf_checkpoint_to_pytorch.py:30

↓ 1 callersFunctioncreate_instances_from_document

This code is mostly a duplicate of the equivalent function from Google BERT's repo. However, we make some changes and improvements. Sampling is im

src/pregenerate_training_data.py:195

↓ 1 callersFunctioncreate_masked_lm_predictions

Creates the predictions for the masked LM objective. This is mostly copied from the Google BERT repo, but with several refactors to clean it up an

src/pregenerate_training_data.py:131

↓ 1 callersFunctioncreate_training_file

(docs, vocab_list, args, epoch_num, bi_text=True)

src/pregenerate_training_data.py:301

↓ 1 callersFunctionextract_oracle_from_logits

(logits, texts, summaries, extract_n_sents, trigram_block=True)

src/evaluate.py:19

↓ 1 callersMethodfrom_dict

Constructs a `BertConfig` from a Python dictionary of parameters.

src/pytorch_pretrained_bert/modeling.py:155

↓ 1 callersMethodgenerate_data

src/data.py:413

↓ 1 callersFunctionget_aug_method

src/train.py:251

↓ 1 callersFunctionget_dataset_info

src/data.py:123

↓ 1 callersFunctionget_from_cache

Given a URL, look for the corresponding dataset in the local cache. If it's not there, download it. Then return the path to the cached file.

src/pytorch_pretrained_bert/file_utils.py:165

↓ 1 callersFunctionget_model

(model_name_or_path, num, aug_num)

src/model.py:19

↓ 1 callersFunctionget_optimizer_and_scheduler

(model, args, num_train_optimization_steps)

src/general_distill.py:57

↓ 1 callersFunctionhas_same_trigram

src/evaluate.py:38

↓ 1 callersFunctionhttp_get

(url: str, temp_file: IO)

src/pytorch_pretrained_bert/file_utils.py:153

↓ 1 callersFunctionlinear_rampup

(current, args)

src/train.py:102

↓ 1 callersFunctionload_vocab

Loads a vocabulary file into a dictionary.

src/pytorch_pretrained_bert/tokenization.py:51

↓ 1 callersFunctionmain

src/pregenerate_training_data.py:325

↓ 1 callersFunctionpad_1d

(x, pad_len, pad_id)

src/data.py:513

↓ 1 callersFunctionparse_args

src/train.py:27

↓ 1 callersFunctions3_etag

Check ETag on S3 object.

src/pytorch_pretrained_bert/file_utils.py:137

↓ 1 callersFunctions3_get

Pull a file directly from S3.

src/pytorch_pretrained_bert/file_utils.py:146

↓ 1 callersMethodsample_doc

(self, current_idx, sentence_weighted=True)

src/pregenerate_training_data.py:71

↓ 1 callersFunctionseed_everything

src/train.py:93

↓ 1 callersFunctionsimple_accuracy

(preds, labels)

src/metrics.py:78

↓ 1 callersFunctionsumeval

(model, dataloader, specific_student, extract_nsents, device, pyrouge=True, trigram_block=True, en

src/evaluate.py:76

↓ 1 callersMethodto_dict

Serializes this instance to a Python dictionary.

src/pytorch_pretrained_bert/modeling.py:172

↓ 1 callersFunctiontrain

src/train.py:501

↓ 1 callersFunctiontruncate_seq_pair

Truncates a pair of sequences to a maximum sequence length. Lifted from Google's BERT repo.

src/pregenerate_training_data.py:109

↓ 1 callersFunctionudaeval

(model, dataloader, specific_student, device, num_labels=2, ensemble_prediction=True)

src/evaluate.py:117

↓ 1 callersMethodunsupervised_dataloader

src/data.py:299

↓ 1 callersFunctionurl_to_filename

Convert `url` into a hashed filename in a repeatable way. If `etag` is specified, append its hash to the url's, delimited by a period.

src/pytorch_pretrained_bert/file_utils.py:30

↓ 1 callersFunctionvat_generator

(model, x, specific_student, inputs_embeds, target_logits, iter, device=None, loss_func=None,

src/train.py:114

↓ 1 callersFunctionwrite_jsonl

src/utils.py:26

Method__enter__

src/pregenerate_training_data.py:99

(self, exc_type, exc_val, traceback)

src/pregenerate_training_data.py:102

Method__getitem__

(self, index: int)

src/utils.py:53

Method__getitem__

src/data.py:352

Method__getitem__

src/data.py:444

Method__getitem__

src/data.py:468

Method__getitem__

src/pregenerate_training_data.py:93

Method__getitem__

src/pytorch_pretrained_bert/modeling.py:80

src/utils.py:44

src/general_distill.py:91

(self, text=None, label=None)

( self, dataset_name: str, train_batch_size: int, val_batch_si

src/data.py:137

( self, dataset_name: str, task_name: str, train_batch_size: i

src/data.py:242

(self, data_filename, tokenizer, text_max_len: int,

src/data.py:335

(self, training_path, epoch, tokenizer, num_data_epochs, reduce_memory=False)

src/data.py:393

(self, data_filename, tokenizer, max_seq_len)

src/data.py:454

( self, input_ids, attn_mask, pos_ids, cls_mask,

src/data.py:476

(self, reduce_memory=False)

src/pregenerate_training_data.py:39

(self, config, student_num=2, fit_size=768)

src/model.py:39

next →1–100 of 201, ranked by callers