MCPcopy Create free account

hub / github.com/beyondguo/genius / functions

Functions284 in github.com/beyondguo/genius

⨍ Functions 284◇ Types & classes 37↳ Endpoints 1
↓ 17 callersFunctionget_label2desc
(dataset_name)
augmentation_clf/label_desc.py:1
↓ 16 callersFunctionnormalize
(score, Min, Max)
augmentation_clf/STA/keywords_extractor.py:84
↓ 11 callersMethodextract_keywords
(self, text)
yake/yake.py:58
↓ 11 callersMethodget_kws
(self, s, max_ngram=3, top=10, aspect_keywords=None, use_aspect_as_doc_embedding=False)
genius_utils.py:43
↓ 11 callersMethodtokenizer
(self, text)
augmentation_clf/STA/text_augmenter.py:56
↓ 9 callersMethodaug_by_insertion
(self, text, p, mode='random', selected_words=[], print_info=False)
augmentation_clf/STA/text_augmenter.py:142
↓ 9 callersMethodget_sketch_from_kws
TODO: keywords extracted by YAKE may not always be the same as original, like "IBM's" will be "IBM". for template 3/4, a workar
genius_utils.py:56
↓ 8 callersMethodaug_by_deletion
p：每个词以p的概率被删除
augmentation_clf/STA/text_augmenter.py:257
↓ 8 callersMethodaug_by_replacement
(self, text, p, mode='random', selected_words=[], print_info=False)
augmentation_clf/STA/text_augmenter.py:98
↓ 6 callersMethodaug_by_swap
(self, text, p, mode='random', selected_words=[], print_info=False)
augmentation_clf/STA/text_augmenter.py:204
↓ 6 callersFunctionget_median
(scores)
augmentation_clf/STA/keywords_extractor.py:53
↓ 6 callersMethodreplace_token
(self, text_tokens, y, n_gram_word_list)
yake/highlight.py:145
↓ 5 callersMethodextract_aspect_keywords
Extract keywords/keyphrases for a single document Arguments: doc: The document for which to extract keywords/keyphrases
aspect_keybert.py:23
↓ 5 callersMethodget_text_vec
获取一段文本（一个字，一个词，一个短语，一段话都可以）的w2v向量表示 get the w2v representation of a given string - 如果这个文本直接在词汇表中了，就直接查询w2v - 如果不在，先通过
augmentation_clf/STA/keywords_extractor.py:121
↓ 5 callersFunctionsmall_fix
(text)
augmentation_clf/STA/text_augmenter.py:15
↓ 5 callersMethodtokenizer
(self, text)
_backup_scripts/easy_text_augmenter.py:41
↓ 4 callersMethodadd_word
(self, words, mode, given_word=None)
augmentation_clf/STA/text_augmenter.py:174
↓ 4 callersMethodadd_word
(self, words, mode, given_word=None)
_backup_scripts/easy_text_augmenter.py:157
↓ 4 callersMethodaug_by_replacement
(self, text, p, mode='random', selected_words=[], print_info=False)
_backup_scripts/easy_text_augmenter.py:83
↓ 4 callersMethodaug_by_selection
(self, text, selected_words=[], print_info=False)
augmentation_clf/STA/text_augmenter.py:304
↓ 4 callersFunctioncut_too_long_sequences
(tokens_list, tags_list)
_backup_scripts/ner_aug.py:236
↓ 4 callersFunctioncut_too_long_sequences
(tokens_list, tags_list)
_backup_scripts/k2t_new_ner_aug.py:255
↓ 4 callersFunctionget_dataloader
(file_path, tokenizer, label2idx, maxlen, bsz, collate_fn, shuffle=True)
utils.py:62
↓ 4 callersFunctionget_quartiles
获取分位数。 Q1：上四分位数； Q2：中位数； Q3：下四分位数。
augmentation_clf/STA/keywords_extractor.py:61
↓ 4 callersMethodswap_word
(self, words, mode, selected_word=None)
augmentation_clf/STA/text_augmenter.py:236
↓ 4 callersMethodswap_word
(self, words, mode, selected_word=None)
_backup_scripts/easy_text_augmenter.py:217
↓ 3 callersMethodaug_by_insertion
(self, text, p, mode='random', given_words=[], print_info=False)
_backup_scripts/easy_text_augmenter.py:127
↓ 3 callersFunctionevaluate_from_dataloader
(model, dataloader, disable_tqdm=True)
do_clf.py:99
↓ 3 callersFunctionevaluate_from_dataloader
(model, dataloader)
augmentation_clf/STA/clf.py:102
↓ 3 callersFunctionextract_mentions
return: mentions: [] mention_dict: {'MISC': [], 'PER': [], 'LOC': [], 'ORG': []}
_backup_scripts/k2t_new_ner_aug.py:149
↓ 3 callersMethodget_similar_words
使用预选保存好的相似词典直接查询 若词典中没有该词，则分情况讨论： 1. 对于中文，则查询该词的最后一个字，因为考虑到中文词中多数情况后面的字更能代表该词 2. 对于英文，则直接返回[]
augmentation_clf/STA/text_augmenter.py:62
↓ 3 callersMethodget_similar_words
使用预选保存好的相似词典直接查询 若词典中没有该词，则分情况讨论： 1. 对于中文，则查询该词的最后一个字，因为考虑到中文词中多数情况后面的字更能代表该词 2. 对于英文，则直接返回[]
_backup_scripts/easy_text_augmenter.py:47
↓ 3 callersFunctionget_stopwords
()
genius_utils.py:13
↓ 2 callersMethodaddOrUpdateComposedWord
(self, cand)
yake/datarepresentation.py:176
↓ 2 callersFunctionalign_labels_with_tokens
(labels, word_ids)
_backup_scripts/run_ner.py:45
↓ 2 callersFunctionalign_labels_with_tokens
(labels, word_ids)
augmentation_ner_qa/run_ner.py:45
↓ 2 callersMethodaug_by_deletion
p：每个词以p的概率被删除
_backup_scripts/easy_text_augmenter.py:238
↓ 2 callersMethodaug_by_swap
(self, text, p, mode='random', selected_words=[], print_info=False)
_backup_scripts/easy_text_augmenter.py:187
↓ 2 callersFunctioncontain_alpha
(string)
augmentation_clf/genius_mixup_clf.py:70
↓ 2 callersFunctiondataloader_evaluation
(model, dataloader, return_metric='overall_f1',verbose=False)
_backup_scripts/run_ner.py:237
↓ 2 callersFunctiondataloader_evaluation
(model, dataloader, return_metric='overall_f1',verbose=False)
augmentation_ner_qa/run_ner.py:237
↓ 2 callersFunctionextract_aspect_keywords
aspect_keywords: list, 用于计算aspect embedding aspect_as_doc: bool, 若为True，则将aspect作为抽取关键词的完全参照 add_aspect_keywords: int, 将n个aspect words插入到
_backup_scripts/run_qa_aug.py:31
↓ 2 callersFunctionextract_mentions
return: mentions: [] mention_dict: {'MISC': [], 'PER': [], 'LOC': [], 'ORG': []}
_backup_scripts/ner_aug.py:84
↓ 2 callersFunctionextract_mentions
return: mentions: [] mention_dict: {'MISC': [], 'PER': [], 'LOC': [], 'ORG': []}
augmentation_ner_qa/genius_ner_aug.py:58
↓ 2 callersMethodfind_more_relevant
(self, y, text_tokens, n_gram, relevant_words_array, kw_list, splited_n_gram_word_list)
yake/highlight.py:123
↓ 2 callersMethodgetTag
(self, word, i)
yake/datarepresentation.py:129
↓ 2 callersMethodgetTerm
(self, str_word, save_non_seen=True)
yake/datarepresentation.py:145
↓ 2 callersFunctionget_dataloader
(file_path, tokenizer, label2idx, maxlen, bsz, collate_fn)
augmentation_clf/STA/my_dataset.py:44
↓ 2 callersFunctionget_random_mask_dataset
()
augmentation_clf/conditional_mlm_finetune.py:73
↓ 2 callersFunctionget_topk
(s,max_k=8)
_backup_scripts/run_qa_aug.py:84
↓ 2 callersFunctionget_topk
(s,max_k=8)
augmentation_ner_qa/genius_qa_aug.py:62
↓ 2 callersMethodglobal_role_kws_extraction_one_line
(self, contents, labels, label_desc_dict=None, num_words=None, ou
augmentation_clf/STA/keywords_extractor.py:281
↓ 2 callersMethodisValid
(self)
yake/datarepresentation.py:204
↓ 2 callersMethodk2t
inputs: text or a list of text
_backup_scripts/run_qa_aug.py:13
↓ 2 callersMethodk2t
inputs: text or a list of text
_backup_scripts/run_aug.py:26
↓ 2 callersFunctionkeysents_semantic
(text, candidates=None, aspect_keywords=None, max_ngram=3, topk=20, aspect_only=False)
_backup_scripts/k2t_new_sm_aug.py:29
↓ 2 callersFunctionmain
()
augmentation_ner_qa/run_qa.py:208
↓ 2 callersFunctionmask_unimportant_parts
输出例子： '<mask> Google is acquiring Kaggle <mask> hosts data science and machine learning competitions <mask> Google <mask> hosting its Cl
_backup_scripts/prepare_k2t_data_with_mask.py:23
↓ 2 callersFunctionrun_yake
(text_content)
yake/cli.py:24
↓ 2 callersFunctionsetup_seed
(seed)
genius_utils.py:177
↓ 2 callersMethodtag
(self, sentence)
augmentation_ner_qa/genius_ner_aug.py:106
↓ 2 callersMethodupdateH
if features == None or "WRel" in features: self.PL = self.WDL / maxTF self.PR = self.WDR / maxTF self.WRel = ( (0.
yake/datarepresentation.py:359
↓ 1 callersMethod__init__
(self, tokenizer, texts, labels, label2idx, maxlen)
utils.py:47
↓ 1 callersMethod__ratio
(distance, str_length)
yake/Levenshtein.py:6
↓ 1 callersMethod_build
(self, text, windowsSize, n)
yake/datarepresentation.py:48
↓ 1 callersFunctiona_len
(s)
pre_training/prepare_genius_pretrain_data.py:26
↓ 1 callersMethodaddCooccur
(self, left_term, right_term)
yake/datarepresentation.py:171
↓ 1 callersMethodaddOccur
(self, tag, sent_id, pos_sent, pos_text)
yake/datarepresentation.py:414
↓ 1 callersMethodaug_by_selection
(self, text, selected_words=[], print_info=False)
_backup_scripts/easy_text_augmenter.py:282
↓ 1 callersMethodbuild_mult_terms_features
(self, features=None)
yake/datarepresentation.py:115
↓ 1 callersMethodbuild_single_terms_features
(self, features=None)
yake/datarepresentation.py:103
↓ 1 callersFunctioncal_topk
(text)
_backup_scripts/run_aug.py:161
↓ 1 callersFunctionclean_pipeline
(text)
genius_utils.py:171
↓ 1 callersFunctionclean_pipeline
(text)
_backup_scripts/ner_aug.py:175
↓ 1 callersFunctionclean_pipeline
(text)
_backup_scripts/k2t_new_sm_aug.py:110
↓ 1 callersFunctionclean_pipeline
(text)
_backup_scripts/s2t_utils.py:30
↓ 1 callersFunctionclean_pipeline
(text)
_backup_scripts/prepare_k2t_data_with_mask.py:66
↓ 1 callersFunctionclean_pipeline
(text)
_backup_scripts/run_aug.py:222
↓ 1 callersMethodcompute_label_correlation
给定一批标注的文本，按类别计算每个词跟对应类别的相关性. global_doc_count记录每个类中每个词出现在多少个文章里 对于计算wllr来说，某个词w对于某个类l， 其in-class-count = gdc[l][w]，
augmentation_clf/STA/keywords_extractor.py:215
↓ 1 callersMethodcompute_label_similarity
给定一批标注的文本，按类别计算每个词跟对应类别的语义相似度 num_words: 每个样本最大词数
augmentation_clf/STA/keywords_extractor.py:174
↓ 1 callersMethodcompute_similarity_by_text
计算两段文本（词，句，段）的向量余弦相似度 compute the cosine similarity of two pieces of text
augmentation_clf/STA/keywords_extractor.py:163
↓ 1 callersMethodcompute_similarity_by_vector
(self, v1, v2)
augmentation_clf/STA/keywords_extractor.py:160
↓ 1 callersFunctionconcat_multiple_sequences
(dataset, size=3)
_backup_scripts/ner_aug.py:61
↓ 1 callersFunctionconcat_multiple_sequences
(dataset, size=3, overlap=True)
_backup_scripts/k2t_new_ner_aug.py:120
↓ 1 callersFunctionconcat_multiple_sequences
(dataset, size=3, overlap=True)
augmentation_ner_qa/genius_ner_aug.py:40
↓ 1 callersMethoddistance
(seq1, seq2)
yake/Levenshtein.py:16
↓ 1 callersMethodevaluate
(self, eval_dataset=None, eval_examples=None, ignore_keys=None, metric_key_prefix: str = "eval")
augmentation_ner_qa/trainer_qa.py:34
↓ 1 callersFunctionextract_aspect_keywords
(text, aspect_keywords, aspect_as_doc=True, topk=10, n_gram=2, add_aspect_keywords=0, shuffle=False, return_st
_backup_scripts/run_aug.py:74
↓ 1 callersFunctionextract_keywords
(text,tool='yake',topk=10)
_backup_scripts/ner_aug.py:118
↓ 1 callersFunctionextract_mentions_with_keywords
(tokens, tags, n=1, convert_to_str=False)
_backup_scripts/ner_aug.py:134
↓ 1 callersFunctionfix_seed
(i)
utils.py:37
↓ 1 callersFunctionfix_seed
(i)
augmentation_clf/STA/utils.py:34
↓ 1 callersMethodformat_n_gram_text
(self, text, relevant_words_array, n_gram)
yake/highlight.py:55
↓ 1 callersMethodformat_one_gram_text
(self, text, relevant_words_array)
yake/highlight.py:42
↓ 1 callersFunctionget_begin_words
(text,n=2)
_backup_scripts/sega_yahoo.py:68
↓ 1 callersMethodget_composed_feature
(self, feature_name, discart_stopword=True)
yake/datarepresentation.py:210
↓ 1 callersFunctionget_mention_name
(tag)
_backup_scripts/ner_aug.py:77
↓ 1 callersFunctionget_mention_name
(tag)
_backup_scripts/k2t_new_ner_aug.py:142
↓ 1 callersFunctionget_mention_name
(tag)
augmentation_ner_qa/genius_ner_aug.py:33
↓ 1 callersFunctionget_spans
(tokens,window=3)
_backup_scripts/k2t_new_ner_aug.py:176
next →1–100 of 284, ranked by callers