MCPcopy Create free account
hub / github.com/davisking/dlib / get_token

Method get_token

dlib/cpp_tokenizer/cpp_tokenizer_kernel_c.h:114–129  ·  view source on GitHub ↗

Source from the content-addressed store, hash-verified

112 typename tokenizer
113 >
114 void cpp_tokenizer_kernel_c<tokenizer>::
115 get_token (
116 int& type,
117 std::string& token
118 )
119 {
120 // make sure requires clause is not broken
121 DLIB_CASSERT( this->stream_is_set() == true,
122 "\tvoid cpp_tokenizer::get_token()"
123 << "\n\tyou must set a stream for this object before you can get tokens from it."
124 << "\n\tthis: " << this
125 );
126
127 // call the real function
128 tokenizer::get_token(type,token);
129 }
130
131// ----------------------------------------------------------------------------------------
132

Callers

nothing calls this directly

Calls

no outgoing calls

Tested by

no test coverage detected