MCPcopy Create free account
hub / github.com/HA-Transformer/MAT / encode_line

Method encode_line

fairseq/data/dictionary.py:244–262  ·  view source on GitHub ↗
(self, line, line_tokenizer=tokenize_line, add_if_not_exist=True,
                    consumer=None, append_eos=True, reverse_order=False)

Source from the content-addressed store, hash-verified

source not stored for this graph (policy: none)

Callers 15

test_finalizeMethod · 0.95
make_batchesFunction · 0.80
scoreFunction · 0.80
mainFunction · 0.80
mainFunction · 0.80
make_batchesFunction · 0.80
scoreFunction · 0.80
mainFunction · 0.80
mainFunction · 0.80
binarizeMethod · 0.80
post_process_predictionFunction · 0.80
binarizeMethod · 0.80

Calls 3

add_symbolMethod · 0.95
indexMethod · 0.95
consumerFunction · 0.50

Tested by 2

test_finalizeMethod · 0.76
get_idsMethod · 0.64