MCPcopy Index your code
hub / github.com/YunwenTechnology/QueryGeneration / convert_to_unicode

Function convert_to_unicode

tokenization/tokenization.py:65–82  ·  view source on GitHub ↗

Converts `text` to Unicode (if it's not already), assuming utf-8 input.

(text)

Source from the content-addressed store, hash-verified

source not stored for this graph (policy: none)

Callers 3

load_vocabFunction · 0.85
tokenizeMethod · 0.85
tokenizeMethod · 0.85

Calls

no outgoing calls

Tested by

no test coverage detected