Function get_tokens

src/licensedcode/match_unknown.py:158–159

(_toks)

        tokens_by_tid = idx.tokens_by_tid

        def get_tokens(_toks):
            return (' '.join(tokens_by_tid[t] for t in _toks))

        print('match_unknowns: matched_ngrams')