github.com/buckhx/gobert @main

Tokenizer is an interface for chunking a string into it's tokens as per the BERT implematation [3 implementers]

Predictor creates tensors for prediction [1 implementers]

Provider is an interface for exposing a vocab [1 implementers]

TensorInputFunc maps tensors to an estimator.InputFunc in the Predict pipeline

BertOption configures a BERT model

VocabTokenizer comprises of a Tokenizer and VocabProvider

Tokenizer is an interface for chunking a string into it's tokens as per the BERT implematation

ValueProvider is a simple interface for tensors responses without the baggage

Bert is a model that translates features to values from an exported model. It processes as follows: Pipeline: text -> …

Dict is a container for tokens NOTE: python uses an OrderedDict, unsure of implications

Provider is an interface for exposing a vocab

FeatureFactory will create features with the supplied tokenizer and sequence length