MCPcopy Create free account
hub / github.com/pytorch/tutorials / prepareData

Function prepareData

intermediate_source/seq2seq_translation_tutorial.py:265–277  ·  view source on GitHub ↗
(lang1, lang2, reverse=False)

Source from the content-addressed store, hash-verified

263#
264
265def prepareData(lang1, lang2, reverse=False):
266 input_lang, output_lang, pairs = readLangs(lang1, lang2, reverse)
267 print("Read %s sentence pairs" % len(pairs))
268 pairs = filterPairs(pairs)
269 print("Trimmed to %s sentence pairs" % len(pairs))
270 print("Counting words...")
271 for pair in pairs:
272 input_lang.addSentence(pair[0])
273 output_lang.addSentence(pair[1])
274 print("Counted words:")
275 print(input_lang.name, input_lang.n_words)
276 print(output_lang.name, output_lang.n_words)
277 return input_lang, output_lang, pairs
278
279input_lang, output_lang, pairs = prepareData('eng', 'fra', True)
280print(random.choice(pairs))

Callers 2

get_dataloaderFunction · 0.85

Calls 3

readLangsFunction · 0.85
filterPairsFunction · 0.70
addSentenceMethod · 0.45

Tested by

no test coverage detected