MCPcopy Create free account
hub / github.com/binbinjiang/CVT-SLR / split_utf8_str

Function split_utf8_str

ctcdecode/ctcdecode/src/decoder_utils.cpp:83–100  ·  view source on GitHub ↗

Source from the content-addressed store, hash-verified

81}
82
83std::vector<std::string> split_utf8_str(const std::string &str) {
84 std::vector<std::string> result;
85 std::string out_str;
86
87 for (char c : str) {
88 if ((c & 0xc0) != 0x80) // new UTF-8 character
89 {
90 if (!out_str.empty()) {
91 result.push_back(out_str);
92 out_str.clear();
93 }
94 }
95
96 out_str.append(1, c);
97 }
98 result.push_back(out_str);
99 return result;
100}
101
102std::vector<std::string> split_str(const std::string &s,
103 const std::string &delim) {

Callers 2

split_labelsMethod · 0.85
add_word_to_dictionaryFunction · 0.85

Calls

no outgoing calls

Tested by

no test coverage detected