MCPcopy Create free account
hub / github.com/ShannonAI/CorefQA / FullTokenizer

Class FullTokenizer

bert/tokenization.py:161–182  ·  view source on GitHub ↗

Runs end-to-end tokenziation.

Source from the content-addressed store, hash-verified

source not stored for this graph (policy: none)

Callers 1

prepare_train_datasetFunction · 0.90

Calls

no outgoing calls

Tested by

no test coverage detected