Function _compare

tasks/seq2seq/dataset.py:246–247

(s1, s2)

        return unidecode.unidecode(s.lower())

    def _compare(s1, s2):
        return _normalize(s1) == _normalize(s2)

    tokens = [tokenizer.IdToToken(token) for token in tokens]
    text = raw_text