MCPcopy Create free account
hub / github.com/DeepRec-AI/DeepRec / SplitOnCharSet

Function SplitOnCharSet

tensorflow/core/kernels/string_split_op.cc:87–103  ·  view source on GitHub ↗

Source from the content-addressed store, hash-verified

85// Based on str_util::Split.
86template <typename Predicate>
87std::vector<StringPiece> SplitOnCharSet(const string& str,
88 const string& delim_set, Predicate p) {
89 std::vector<StringPiece> result;
90 StringPiece text(str);
91 StringPiece delims(delim_set);
92 size_t token_start = 0;
93 for (size_t i = 0; i < text.size() + 1; i++) {
94 if ((i == text.size()) || (delims.find(text[i]) != StringPiece::npos)) {
95 StringPiece token(text.data() + token_start, i - token_start);
96 if (p(token)) {
97 result.emplace_back(token);
98 }
99 token_start = i + 1;
100 }
101 }
102 return result;
103}
104
105// Split input string `str` based on given delimiter.
106// Returns a vector of StringPieces which are valid as long as input `str`

Callers 1

SplitFunction · 0.85

Calls 4

sizeMethod · 0.45
findMethod · 0.45
dataMethod · 0.45
emplace_backMethod · 0.45

Tested by

no test coverage detected