MCPcopy Create free account
hub / github.com/apache/singa / _is_whitespace

Function _is_whitespace

examples/onnx/bert/tokenization.py:358–367  ·  view source on GitHub ↗

Checks whether `chars` is a whitespace character.

(char)

Source from the content-addressed store, hash-verified

356
357
358def _is_whitespace(char):
359 """Checks whether `chars` is a whitespace character."""
360 # \t, \n, and \r are technically contorl characters but we treat them
361 # as whitespace since they are generally considered as such.
362 if char == " " or char == "\t" or char == "\n" or char == "\r":
363 return True
364 cat = unicodedata.category(char)
365 if cat == "Zs":
366 return True
367 return False
368
369
370def _is_control(char):

Callers 1

_clean_textMethod · 0.85

Calls

no outgoing calls

Tested by

no test coverage detected