MCPcopy Create free account

hub / github.com/OpenNMT/Tokenizer / types & classes

Types & classes32 in github.com/OpenNMT/Tokenizer

⨍ Functions 261◇ Types & classes 32
↓ 1 callersClassSubwordEncoder
include/onmt/Tokenizer.h:16
↓ 1 callersClassTokenizerWrapper
bindings/python/pyonmttok/Python.cc:20
ClassBPELearnerWrapper
bindings/python/pyonmttok/Python.cc:367
EnumCaseMarkupType
src/Casing.h:22
EnumCaseType
include/onmt/unicode/Unicode.h:36
EnumCasing
include/onmt/Token.h:11
ClassChange
src/BPELearner.cc:67
EnumCharType
include/onmt/unicode/Unicode.h:21
EnumFlags
include/onmt/Tokenizer.h:158
EnumMode
include/onmt/Tokenizer.h:21
ClassOPENNMTTOKENIZER_EXPORT
include/onmt/SentencePieceLearner.h:13
ClassOPENNMTTOKENIZER_EXPORT
include/onmt/Vocab.h:16
ClassOPENNMTTOKENIZER_EXPORT
include/onmt/Token.h:27
ClassOPENNMTTOKENIZER_EXPORT
include/onmt/ITokenizer.h:17
ClassOPENNMTTOKENIZER_EXPORT
include/onmt/BPE.h:15
ClassOPENNMTTOKENIZER_EXPORT
include/onmt/SentencePiece.h:17
ClassOPENNMTTOKENIZER_EXPORT
include/onmt/SubwordLearner.h:13
ClassOPENNMTTOKENIZER_EXPORT
include/onmt/BPELearner.h:10
ClassOPENNMTTOKENIZER_EXPORT
include/onmt/SubwordEncoder.h:12
ClassOPENNMTTOKENIZER_EXPORT
include/onmt/Tokenizer.h:18
ClassOPENNMTTOKENIZER_EXPORT
include/onmt/unicode/Unicode.h:53
ClassOptions
See https://github.com/OpenNMT/Tokenizer/blob/master/docs/options.md for more details.
include/onmt/Tokenizer.h:34
ClassSentencePieceLearnerWrapper
bindings/python/pyonmttok/Python.cc:411
ClassSentencePieceProcessor
include/onmt/SentencePiece.h:11
ClassSentencePieceTokenizerWrapper
bindings/python/pyonmttok/Python.cc:296
EnumState
src/Tokenizer.cc:70
ClassSubwordLearnerWrapper
bindings/python/pyonmttok/Python.cc:313
ClassTokenCaseMarkup
src/Casing.h:34
EnumTokenType
include/onmt/Token.h:20
ClassTokensBuilder
src/Tokenizer.cc:536
Enum_type_letter
include/onmt/unicode/Unicode.h:124
Classpair_hash
src/BPELearner.cc:84