MCPcopy Create free account
hub / github.com/ACL2020SpellGCN/SpellGCN / whitespace_tokenize

Function whitespace_tokenize

tokenization.py:153–159  ·  view source on GitHub ↗

Runs basic whitespace cleaning and splitting on a piece of text.

(text)

Source from the content-addressed store, hash-verified

source not stored for this graph (policy: none)

Callers 2

tokenizeMethod · 0.85
tokenizeMethod · 0.85

Calls

no outgoing calls

Tested by

no test coverage detected