MCPcopy
hub / github.com/codemayq/chinese-chatbot-corpus / format_refine

Function format_refine

util.py:47–59  ·  view source on GitHub ↗
(file_name)

Source from the content-addressed store, hash-verified

45
46
47def format_refine(file_name):
48 file = codecs.open(file_name, encoding=Config.encoding)
49 valid_lines = []
50 for index, line in enumerate(file):
51 pair = line.split("\t")
52 if len(pair) == 2:
53 valid_lines.append(line)
54 file.close()
55
56 file = codecs.open(file_name, "w", encoding=Config.encoding)
57 for line in valid_lines:
58 file.write(line)
59 file.close()

Callers 8

qingyun_process_pipelineFunction · 0.85
douban_process_pipelineFunction · 0.85
ptt_process_pipelineFunction · 0.85
tieba_process_pipelineFunction · 0.85
weibo_process_pipelineFunction · 0.85

Calls

no outgoing calls

Tested by

no test coverage detected