MCPcopy Create free account

hub / github.com/WenSongWang/BERT-BILSTM-CRF-NER-Pytorch-Chinese / functions

Functions43 in github.com/WenSongWang/BERT-BILSTM-CRF-NER-Pytorch-Chinese

↓ 3 callersFunctionevaluate
(iterable, options=None)
torch_ner/source/conlleval.py:65
↓ 3 callersMethodget_dataset
对指定数据集进行预处理,进一步封装数据,包括: examples:[InputExample(guid=index, text=text, label=label)] features:[InputFeatures( input_ids=input_
torch_ner/source/processor.py:49
↓ 3 callersMethodpredict
模型预测 :param input_ids: :param token_type_ids: :param attention_mask: :return:
torch_ner/source/models.py:123
↓ 2 callersFunctioncalculate_metrics
(correct, guessed, total)
torch_ner/source/conlleval.py:152
↓ 2 callersFunctionend_of_chunk
(prev_tag, tag, prev_type, type_)
torch_ner/source/conlleval.py:233
↓ 2 callersFunctionload_pkl
加载pkl文件 :param fp: :return:
torch_ner/source/utils.py:48
↓ 2 callersFunctionmetrics
(counts)
torch_ner/source/conlleval.py:160
↓ 2 callersFunctionparse_args
(argv)
torch_ner/source/conlleval.py:43
↓ 2 callersFunctionparse_tag
(t)
torch_ner/source/conlleval.py:60
↓ 2 callersFunctionsave_pkl
保存pkl文件,数据序列化 :param data: :param fp: :return:
torch_ner/source/utils.py:59
↓ 2 callersFunctionstart_of_chunk
(prev_tag, tag, prev_type, type_)
torch_ner/source/conlleval.py:258
↓ 1 callersFunction_bio_data_handler
处理BIO开头的标签信息 输入:sentence=['张', '三', '的', '老', '婆', '是', '谁', '?'], predict_label=['B-FNAME', 'B-LNAME', 'O', 'O', 'O', 'O', 'O', 'O'] 输出:
torch_ner/source/predict.py:102
↓ 1 callersFunction_check_entity_continuous
判断两个实体是否连续 :param e1: :param e2: :return:
torch_ner/source/predict.py:184
↓ 1 callersMethod_init_train_config
(self)
torch_ner/source/config.py:38
↓ 1 callersFunction_merge_entity
合并一组实体
torch_ner/source/predict.py:194
↓ 1 callersMethodclean_output
清理output_xxx目录,若output_xxx目录存在,将会被删除, 然后初始化输出目录 :param config: :return:
torch_ner/source/processor.py:261
↓ 1 callersMethodconvert_examples_to_features
对输入数据进行特征转换 例如: ****** Example ****** guid: 0 tokens: [CLS] 王 辉 生 前 驾 驶 机 械 洒 药 消 毒 9 0 后 王 辉 ,
torch_ner/source/processor.py:90
↓ 1 callersFunctionevaluate
(config: Config, data, model, id2label, all_ori_tokens)
torch_ner/source/train.py:241
↓ 1 callersFunctionget_entities_result
进一步封装识别结果,最终结果格式如下: [ {'type': 'address_value', 'value': '江苏南京', 'begin': 3, 'end': 7}, {'type': 'first_name', 'value': '张', 'beg
torch_ner/source/predict.py:11
↓ 1 callersMethodget_input_examples
通过读取输入数据,封装输入样本 :param separator: :param input_file: :return:
torch_ner/source/processor.py:174
↓ 1 callersMethodget_label2id_id2label
获取label2id、id2label的映射 :param output_path: :param label_list: :return:
torch_ner/source/processor.py:214
↓ 1 callersMethodget_labels
读取训练数据获取标签 :param config: :return:
torch_ner/source/processor.py:191
↓ 1 callersFunctionload_file
读取文件; 若sep为None,按行读取,返回文件内容列表,格式为:[xxx,xxx,xxx,...] 若不为None,按行读取分隔,返回文件内容列表,格式为: [[xxx,xxx],[xxx,xxx],...] :param fp: :param sep:
torch_ner/source/utils.py:27
↓ 1 callersFunctionmain
(argv)
torch_ner/source/conlleval.py:289
↓ 1 callersFunctionpredict
模型预测 :param sentence: :param model_path: :return:
torch_ner/source/predict.py:47
↓ 1 callersMethodread_data
读取输入数据 :param input_file: :param separator: :return:
torch_ner/source/processor.py:232
↓ 1 callersFunctionreport
(counts, out=None)
torch_ner/source/conlleval.py:173
↓ 1 callersFunctionreport_notprint
(counts, out=None)
torch_ner/source/conlleval.py:199
↓ 1 callersFunctiontrain
模型训练 :return:
torch_ner/source/train.py:33
↓ 1 callersFunctionuniq
(iterable)
torch_ner/source/conlleval.py:147
Method__init__
(self)
torch_ner/source/config.py:14
Method__init__
(self, filename=None, stream=None)
torch_ner/source/utils.py:7
Method__init__
(self, config, need_birnn=False, rnn_dim=128)
torch_ner/source/models.py:12
Method__init__
(self, guid, text, label=None)
torch_ner/source/processor.py:24
Method__init__
:param input_ids: 单词在词典中的编码 :param attention_mask: 指定 对哪些词 进行self-Attention操作 :param token_type_ids: 区分两个句子的编码(上句全为0,
torch_ner/source/processor.py:33
Method__init__
(self)
torch_ner/source/conlleval.py:30
Method__new__
单例类 :param args:cls表示当前的class :param kwargs: :return:
torch_ner/source/config.py:24
Function_combine_person_entity
组合人物实体 :param entity: :return:
torch_ner/source/predict.py:149
Methoddel_file
(path)
torch_ner/source/processor.py:270
Functiondump_json
(obj, fp, encoding='utf-8', indent=4, ensure_ascii=False, json_lines=False)
torch_ner/source/utils.py:71
Methodforward
BERT_BiLSTM_CRF模型的正向传播函数 :param input_ids: torch.Size([batch_size,seq_len]), 代表输入实例的tensor张量 :param token_type_ids: tor
torch_ner/source/models.py:104
Functionload_json
(fp, encoding='utf-8', json_lines=False)
torch_ner/source/utils.py:82
Functionreturn_report
(input_file)
torch_ner/source/conlleval.py:283