MCPcopy Create free account
hub / github.com/catboost/catboost / GetSubtokenCount

Method GetSubtokenCount

library/cpp/tokenizer/nlpparserbase.h:41–43  ·  view source on GitHub ↗

Source from the content-addressed store, hash-verified

39 return NlpType;
40 }
41 size_t GetSubtokenCount() const {
42 return Subtokens.size();
43 }
44 size_t GetStart() const {
45 Y_ASSERT(!Subtokens.empty());
46 return Subtokens[0].Pos - Subtokens[0].PrefixLen;

Callers 2

AddTokenMethod · 0.80
AddLastTokenMethod · 0.80

Calls 1

sizeMethod · 0.45

Tested by

no test coverage detected