MCPcopy Create free account

hub / github.com/JackHCC/Chinese-Keyphrase-Extraction / functions

Functions79 in github.com/JackHCC/Chinese-Keyphrase-Extraction

↓ 4 callersMethodget_tokenized_words_embeddings
@param sents_tokened: list of tokenized words string (sentences/phrases) :return: ndarray with shape (len(sents), dimension of embedd
model/SIF_rank/Embedding.py:35
↓ 4 callersFunctionset_graph_edges
(graph, tokens, split_text)
utils.py:72
↓ 3 callersFunctiondata_word_cut
(data_path=os.path.join(DATA_PATH, 'data.xlsx'))
process.py:63
↓ 3 callersFunctionget_oov_weight
(tokenized_sents, word2weight, word, method="max_weight")
model/SIF_rank/Embedding.py:232
↓ 2 callersFunctioncos_sim
计算两个向量之间的余弦相似度 :param vector_a: 向量 a :param vector_b: 向量 b :return: sim
model/SIF_rank/sif_rank.py:97
↓ 2 callersFunctionget_all_dist
:param candidate_embeddings_list: :param text_obj: :param dist_list: :return: dist_all
model/SIF_rank/sif_rank.py:155
↓ 2 callersFunctionget_dist_cosine
(emb1, emb2, sent_emb_method="elmo", elmo_layers_weight=[0.0, 1.0, 0.0])
model/SIF_rank/sif_rank.py:137
↓ 2 callersFunctionget_final_dist
:param dist_all: :param method: "average" :return:
model/SIF_rank/sif_rank.py:177
↓ 2 callersFunctionget_first_window
(split_text)
utils.py:19
↓ 2 callersFunctionget_sent_segmented
(tokens)
model/SIF_rank/Embedding.py:144
↓ 2 callersMethodget_tokenized_sent_embeddings
Based on part of speech return a list of candidate phrases :param text_obj: Input text Representation see @InputTextObj :para
model/SIF_rank/Embedding.py:59
↓ 2 callersFunctionget_vector
(data, stop_words='english', max_df=0.5, min_df=10)
lda.py:74
↓ 2 callersFunctionget_word_weight
Get the weight of words by word_fre/sum_fre_words :param weightfile :param weightpara :return: word2weight[word]=weight : a dict of w
model/SIF_rank/Embedding.py:277
↓ 2 callersFunctionlda
(tf, topic_num=8, max_iter=100, doc_topic_prior=0.1, topic_word_prior=0.01, random_state=0)
lda.py:84
↓ 2 callersFunctionsave_pickle
(obj, path)
utils.py:91
↓ 2 callersFunctionset_graph_edge
(graph, tokens, word_a, word_b)
utils.py:24
↓ 2 callersFunctionsplice_embeddings
(elmo_embeddings, tokens_segmented)
model/SIF_rank/Embedding.py:162
↓ 1 callersFunctionSG_rank
()
ranks.py:46
↓ 1 callersFunctionSIFRank
@param text: @param SIF: @param cn_model: @param sent_emb_method: @param elmo_layers_weight: @param if_DS: @param if_EA:
model/SIF_rank/sif_rank.py:222
↓ 1 callersFunctionSIFRank_plus
@param text: @param SIF: @param cn_model: @param sent_emb_method: @param elmo_layers_weight: @param if_DS: @param if_EA:
model/SIF_rank/sif_rank.py:247
↓ 1 callersFunctionSIF_rank
(text, plus=True, lambda_=0.8, elmo_layers_weight=[0.0, 1.0, 0.0])
ranks.py:353
↓ 1 callersMethod_create_phrases_with_embeddings
(self, document)
model/embed_rank.py:90
↓ 1 callersMethod_mmr
(self, document, _lambda=0.5)
model/embed_rank.py:54
↓ 1 callersMethod_tokenize
(self, document)
model/embed_rank.py:100
↓ 1 callersFunctionalgorithm_switch
(arg, topic_x_word_matrix, docx_x_topic_matrix, tf_feature_names, txt, article_id, alpha, lambda_,
main.py:45
↓ 1 callersFunctionargs
()
main.py:24
↓ 1 callersFunctioncalc_distance
Calculate distance between 2 topics :param topic_a: list if phrases in a topic A :param topic_b: list if phrases in a topic B
ranks.py:134
↓ 1 callersFunctioncontext_embeddings_alignment
Embeddings Alignment :param elmo_embeddings: The embeddings from elmo :param tokens_segmented: The list of tokens list <class 'list'
model/SIF_rank/Embedding.py:99
↓ 1 callersFunctioncos_sim_transformer
计算两个向量之间的余弦相似度 :param vector_a: 向量 a :param vector_b: 向量 b :return: sim
model/SIF_rank/sif_rank.py:116
↓ 1 callersFunctionembed_rank
@param text: list,分词后的文章输入 @return: 该文档的关键词汇得分排序列表
ranks.py:339
↓ 1 callersFunctionexpand_rank
()
ranks.py:127
↓ 1 callersFunctionextract_candidates
Based on part of speech return a list of candidate phrases :param text_obj: Input text Representation see @InputTextObj :param no_subset:
model/SIF_rank/sif_rank.py:33
↓ 1 callersMethodextract_keyword
(self, document, _lambda=0.5)
model/embed_rank.py:43
↓ 1 callersFunctionget_best_topic_num
根据困惑度获取最优的主题数目 @param tf: @param n_max_topics:
lda.py:49
↓ 1 callersFunctionget_candidate_weighted_average
(tokenized_sents, weight_list, embeddings_list, start, end, embeddings_type="elmo")
model/SIF_rank/Embedding.py:206
↓ 1 callersFunctionget_matrix
获取文档主题分布矩阵和主题词语分布矩阵 @param topic_num: @param data_path: @return:
lda.py:99
↓ 1 callersFunctionget_position_score
(keyphrase_candidate_list, position_bias)
model/SIF_rank/sif_rank.py:201
↓ 1 callersFunctionget_topic_and_keyword
(path)
distribution_statistics.py:27
↓ 1 callersFunctionget_topic_distribution
(keyword_topic_count, topic_set)
distribution_statistics.py:95
↓ 1 callersFunctionget_topic_topk_words
(topic_keyword_count, k=10)
distribution_statistics.py:58
↓ 1 callersFunctionget_weight_list
(word2weight_pretrain, word2weight_finetune, tokenized_sents, lamda, database="")
model/SIF_rank/Embedding.py:250
↓ 1 callersFunctionget_weighted_average
(tokenized_sents, sents_tokened_tagged, weight_list, embeddings_list, embeddings_type="elmo")
model/SIF_rank/Embedding.py:178
↓ 1 callersFunctioninit_queue
(split_text)
utils.py:37
↓ 1 callersFunctionkeyword_statistic
(topic_keyword_list)
distribution_statistics.py:75
↓ 1 callersFunctionmain
(arg, data_path, topic_num, top_k, alpha, lambda_, window_size, max_d, plus)
main.py:71
↓ 1 callersFunctionposition_rank
@param text: list,分词后的文章输入 @param window_size: int,共现窗口大小 @param lambda_: float,PageRank参数,0-1之间 @return: 该文档的关键词汇得分排序列表
ranks.py:50
↓ 1 callersFunctionpred_docx_topic
根据每个文档预测其主题并写入xlsx表格中 @param data: @param docx_x_topic_matrix:
lda.py:36
↓ 1 callersFunctionprint_top_words
打印每个主题下的关键的词语 @param topic_x_word_matrix: @param feature_names: @param n_top_words: @return:
lda.py:19
↓ 1 callersFunctionprocess_first_window
(graph, tokens, split_text)
utils.py:31
↓ 1 callersFunctionprocess_text
(graph, tokens, split_text)
utils.py:64
↓ 1 callersFunctionprocess_word
(graph, tokens, queue, word)
utils.py:53
↓ 1 callersFunctionqueue_iterator
(queue)
utils.py:45
↓ 1 callersFunctionread_corpus
(fname)
model/embed_rank_train.py:38
↓ 1 callersFunctionsalience_rank
@param topic_x_word_matrix: matrix,主题词汇分布矩阵(归一化后的) @param docx_x_topic_matrix: matrix,文档主题分布矩阵(未归一化的) @param tf_feature_names: list,词汇字典
ranks.py:281
↓ 1 callersFunctionshow_topic_topk_words
(top_k_dict)
distribution_statistics.py:68
↓ 1 callersFunctionsingle_tpr
@param topic_x_word_matrix: matrix,主题词汇分布矩阵(归一化后的) @param docx_x_topic_matrix: matrix,文档主题分布矩阵(未归一化的) @param tf_feature_names: list,词汇字典
ranks.py:232
↓ 1 callersFunctionsoftmax
(x)
model/SIF_rank/sif_rank.py:195
↓ 1 callersFunctiontext_rank
@param text: list,分词后的文章输入 @param lambda_: float,PageRank参数,0-1之间 @return: 该文档的关键词汇得分排序列表
ranks.py:26
↓ 1 callersFunctiontokenize
(text)
model/embed_rank_train.py:25
↓ 1 callersFunctiontopic_statistic
(topic_keyword_list)
distribution_statistics.py:38
↓ 1 callersFunctiontpr
@param topic_x_word_matrix: matrix,主题词汇分布矩阵(归一化后的) @param docx_x_topic_matrix: matrix,文档主题分布矩阵(未归一化的) @param tf_feature_names: list,词汇字典
ranks.py:182
↓ 1 callersFunctiontr
(text, max_d, lambda_)
ranks.py:131
↓ 1 callersFunctiontrain
(input_file, model_path, vocab_path, vector_size=100, min_count=5, works=8, epochs=10)
model/embed_rank_train.py:46
↓ 1 callersFunctionupdate_queue
(queue, word)
utils.py:58
↓ 1 callersFunctionweight_total
Sum weights of adjacent nodes. Choose 'j'th nodes which is adjacent to 'i'th node. Sum weight in 'j'th column, then devide wij(weight
ranks.py:60
↓ 1 callersFunctionwrite_to_excel
(obj, save_path)
utils.py:77
Method__init__
(self, model_path)
model/embed_rank.py:40
Method__init__
:param is_sectioned: If we want to section the text. :param zh_model: the pipeline of Chinese tokenization and POS-tagger :pa
model/SIF_rank/sif_rank.py:61
Method__init__
(self, model_path=r'./zhs.model/', cuda_device=0)
model/SIF_rank/Embedding.py:31
Method__init__
(self, word_embeddor, weightfile_pretrain='./model/SIF_rank/dict.txt', weightfile_finetune='./dict.txt',
model/SIF_rank/Embedding.py:49
Functionchinese_word_cut
(article)
process.py:34
Functioncos_sim_gpu
(x, y)
model/SIF_rank/sif_rank.py:83
Functiondraw_topic_distribution
画指定word的主题分布图 @param word: 想要观察主题分布的词语 @param topic_distribution: 主题分布矩阵 @param keyword_dict: 词汇检索表 @param topic_set: 主题检索表
distribution_statistics.py:113
Functionget_effective_words_num
(tokened_sents)
model/SIF_rank/Embedding.py:170
Functionget_normalized_weight
(weight_list)
model/SIF_rank/Embedding.py:266
Functionget_runtime
(fn)
utils.py:81
Functionmat_division
(vector_a, vector_b)
model/SIF_rank/Embedding.py:136
Functionread_pickle
(path)
utils.py:96
Functionwrapper
(*args, **kwargs)
utils.py:82