Functions — github.com/beyondguo/genius

CodeHub MCPcopy Create free account

hub / github.com/beyondguo/genius / functions

Functions284 in github.com/beyondguo/genius

⨍ Functions 284◇ Types & classes 37 ↳ Endpoints 1

(self, model, device=DEVICE)

_backup_scripts/run_aug.py:24

global_mention_dict: {'PER': [], 'MISC': [], 'ORG': [], 'LOC': []}

augmentation_ner_qa/genius_ner_aug.py:86

(self, *args, eval_examples=None, post_process_function=None, **kwargs)

augmentation_ner_qa/trainer_qa.py:29

genius_utils.py:191

augmentation_clf/mlm_clf.py:73

augmentation_clf/conditional_clm_clf.py:60

augmentation_clf/backtrans_clf.py:27

augmentation_clf/conditional_mlm_clf.py:85

augmentation_clf/STA/my_dataset.py:40

_backup_scripts/k2t_new_ner_aug.py:203

_backup_scripts/qa_back_trans.py:48

_backup_scripts/k2t_new_sm_aug.py:82

_backup_scripts/s2t_utils.py:43

_backup_scripts/qa_aug_eda.py:39

_backup_scripts/my_dataset.py:20

Method__post_init__

augmentation_ner_qa/run_qa.py:188

Method__setattr__

(self,attr,value)

Method__setattr__

(self,attr,value)

augmentation_clf/STA/utils.py:26

augmentation_ner_qa/run_qa.py:713

Functionadd_kws_to_dataset_longtext

先直接用\n\n给分成很多段落，然后把太短的过滤掉接着抽取关键词(总词数的的1/10) 由于样本增多了，记得在map的时候设置remove_columns=your_dataset['train'].column_names

_backup_scripts/prepare_k2t_data_with_mask.py:70

Functionadd_sketch_to_dataset

pre_training/prepare_genius_pretrain_data.py:49

Functionadd_sketch_to_dataset

pre_training/prepare_genius_pretrain_data_chinese.py:52

Methodbuild_candidate

(self, candidate_string)

yake/datarepresentation.py:32

Methodbuild_features

(self, doc_id=None, keys=None, rel=True, rel_approx=True, isVirtual=False, features=['WFreq', 'WRel', 'tf', 'W

yake/datarepresentation.py:216

Functionbuild_vocab

(content_word_list, vocab_size)

augmentation_clf/STA/utils.py:46

Functioncollate_fn

augmentation_clf/conditional_clm_finetune.py:110

Functioncollate_fn

augmentation_clf/conditional_mlm_finetune.py:112

Functioncompute_metrics

pre_training/genius_pretrain_chinese.py:56

Functioncompute_metrics

pre_training/genius_pretrain.py:54

Functioncompute_metrics

augmentation_clf/genius_finetune.py:142

Functioncompute_metrics

(p: EvalPrediction)

augmentation_ner_qa/run_qa.py:633

Functionevaluate

(logits, labels)

Functionevaluate

(logits, labels)

augmentation_clf/STA/clf.py:97

Methodextract_global_role_kws

(self, labels, sorted_ls_dict, sorted_wllr_dict)

augmentation_clf/STA/keywords_extractor.py:266

Functionextract_keywords

(text,tool='yake',topk=10)

_backup_scripts/run_aug.py:40

Functionextract_keywords_with_given_words

(text,tool='yake',topk=10, given_words=[])

_backup_scripts/run_aug.py:54

Functionextract_mention_spans

把一个句子中

_backup_scripts/k2t_new_ner_aug.py:182

Functionget_dataloader

(file_path, tokenizer, label2idx, maxlen, bsz, collate_fn, shuffle=True)

_backup_scripts/my_dataset.py:24

Functionget_dataloader_from_list

(texts, labels, tokenizer, label2idx, maxlen, bsz, collate_fn, shuffle=True)

Functionget_dataloader_from_list

(texts, labels, tokenizer, label2idx, maxlen, bsz, collate_fn, shuffle=True)

_backup_scripts/my_dataset.py:32

Functionget_idf

ds: dataset, document list

augmentation_clf/STA/keywords_extractor.py:33

Methodget_sketch

(self, s, max_ngram=3, top=10, aspect_keywords=None, use_aspect_as_doc_embedding=False, template=4)

genius_utils.py:132

t: term document_words: split document words term must be in document words

augmentation_clf/STA/keywords_extractor.py:24

Functionget_topk

_backup_scripts/qa_back_trans.py:80

Functionget_topk

_backup_scripts/qa_aug_eda.py:71

Functionget_word2vec_weight

(lang, word_to_id)

augmentation_clf/STA/utils.py:70

Methodhighlight

Returns the highlighted text snippets of matching text in the original data

yake/highlight.py:21

(self, cand1, cand2)

yake/yake.py:49

Functionkeynotes_yake

(text, max_ngram=3, topk=20)

_backup_scripts/k2t_new_sm_aug.py:19

(self, cand1, cand2)

yake/yake.py:52

Functionmakedir

Functionmakedir

augmentation_clf/STA/utils.py:7

Functionpost_processing_function

(examples, features, predictions, stage="eval")

augmentation_ner_qa/run_qa.py:606

Functionpostprocess_qa_predictions_with_beam_search

Post-processes the predictions of a question-answering model with beam search to convert them to answers that are substrings of the original

augmentation_ner_qa/utils_qa.py:252

Functionprepare_train_features

augmentation_ner_qa/run_qa.py:371

Functionprepare_validation_features

augmentation_ner_qa/run_qa.py:505

Functionpreprocess_function

# inputs: the sketch # labels: the original text

pre_training/genius_pretrain_chinese.py:36

Functionpreprocess_function

# inputs: the sketch # labels: the original text

pre_training/genius_pretrain.py:37

Functionpreprocess_function

# inputs: the sketch # labels: the original text

augmentation_clf/genius_finetune.py:125

Functionremove_last_sentence

_backup_scripts/ner_aug.py:166

Functionremove_stopwords

_backup_scripts/run_aug.py:11

Methodselective_text_augmentation

- text, 直接给一段文本（不用分词）， - role_kws_dict: {'CW':list ,'FW_in':list, 'FW_out':list, 'IW':list} CW: class-indicating words, use

augmentation_clf/STA/text_augmenter.py:382

Methodselective_text_augmentation

- text, 直接给一段文本（不用分词）， - role_kws_dict: {'CW':list ,'FW_in':list, 'FW_out':list, 'IW':list} CW: class-indicating words, use

_backup_scripts/easy_text_augmenter.py:358

(self, cand1, cand2)

yake/yake.py:55

Functionsetup_seed

augmentation_clf/STA/utils.py:14

_backup_scripts/ner_aug.py:45

_backup_scripts/k2t_new_ner_aug.py:104

_backup_scripts/prepare_k2t_data_with_mask.py:117

(dataset, i, print_to_file=None)

_backup_scripts/run_ner.py:98

(dataset, i, print_to_file=None)

augmentation_ner_qa/run_ner.py:98

Functionshow_dataset

(dataset, i, print_to_file=None)

_backup_scripts/ner_aug.py:27

Functionshow_dataset

(dataset, i, print_to_file=None)

_backup_scripts/k2t_new_ner_aug.py:86

Functiontext2encoding

(words, word_to_id, maxlen, padding=False)

augmentation_clf/STA/utils.py:62

Functiontext_preprocess

pre_training/prepare_genius_pretrain_data.py:30

Methodthere_is_punc

aspect_keybert.py:72

Functiontokenize_and_align_labels

_backup_scripts/run_ner.py:70

Functiontokenize_and_align_labels

augmentation_ner_qa/run_ner.py:70

Functiontokenize_and_align_labels_block_others

_backup_scripts/run_ner.py:83

Functiontokenize_and_align_labels_fuck_others

augmentation_ner_qa/run_ner.py:83

Functiontokenize_function

augmentation_clf/mlm_clf.py:48

Functiontokenize_function

augmentation_clf/conditional_mlm_clf.py:57

(self, features=None, isVirtual=False)

yake/datarepresentation.py:269

MethodupdateH_old

(self, features=None, isVirtual=False)

yake/datarepresentation.py:306

← previous201–284 of 284, ranked by callers