Function get_tokens_and_segments

nlp/bert.py:9–16 · view source on GitHub ↗

Get tokens of the BERT input sqeuence and their segment IDs

(tokens_a, tokens_b=None)

Source from the content-addressed store, hash-verified

source not stored for this graph (policy: none)

nothing calls this directly

no outgoing calls

no test coverage detected