MCPcopy Create free account
hub / github.com/apache/singa / _is_control

Function _is_control

examples/onnx/bert/tokenization.py:370–379  ·  view source on GitHub ↗

Checks whether `chars` is a control character.

(char)

Source from the content-addressed store, hash-verified

368
369
370def _is_control(char):
371 """Checks whether `chars` is a control character."""
372 # These are technically control characters but we count them as whitespace
373 # characters.
374 if char == "\t" or char == "\n" or char == "\r":
375 return False
376 cat = unicodedata.category(char)
377 if cat in ("Cc", "Cf"):
378 return True
379 return False
380
381
382def _is_punctuation(char):

Callers 1

_clean_textMethod · 0.85

Calls

no outgoing calls

Tested by

no test coverage detected