Converts an index (integer) in a token (str) using the vocab.

Converts an index (integer) in a token (str) using the vocab.

Converts an index (integer) in a token (str) using the vocab.

Converts an index (integer) in a token (str) using the vocab.

Converts a token (str) in an id using the vocab.

Converts a token (str) in an id using the vocab.

Converts a token (str) in an id using the vocab.

Converts a token (str) in an id using the vocab.

Converts a token (str) in an id using the vocab.

Initialize the weights.

Initialize the weights.

Initialize the weights.

Initialize the weights.

Initialize the weights.

Initialize the weights.

Initialize the weights.

Initialize the weights.

Returns a tokenized string.

Returns a tokenized string.

Returns a tokenized string.

Returns a tokenized string.

Returns a tokenized string.

Args: tokenizer: the tokenizer of telechat question: question which the model reply in this turn hi

Args: tokenizer: the tokenizer of telechat question: question which the model reply in this turn histor

Args: tokenizer: the tokenizer of telechat question: question which the model reply in this turn hi

Args: tokenizer: the tokenizer of telechat question: question which the model reply in this turn hi

Args: tokenizer: the tokenizer of telechat question: question which the model reply in this turn histor

Args: tokenizer: the tokenizer of telechat question: question which the model reply in this turn histor

Args: tokenizer: the tokenizer of telechat question: question which the model reply in this turn histor

Converts a sequence of tokens (string) in a single string.

Converts a sequence of tokens (string) in a single string.

Converts a sequence of tokens (string) in a single string.

Converts a sequence of tokens (string) in a single string.

Converts a sequence of tokens (string) in a single string.

Creates a mask from the two sequences passed to be used in a sequence-pair classification task. An ALBERT sequence pair mask has the

Creates a mask from the two sequences passed to be used in a sequence-pair classification task. An ALBERT sequence pair mask has the

Creates a mask from the two sequences passed to be used in a sequence-pair classification task. An ALBERT sequence pair mask has the

Creates a mask from the two sequences passed to be used in a sequence-pair classification task. An ALBERT sequence pair mask has the

Creates a mask from the two sequences passed to be used in a sequence-pair classification task. An ALBERT sequence pair mask has the

Flushes any remaining cache and prints a newline to stdout.

Flushes any remaining cache and prints a newline to stdout.

Flushes any remaining cache and prints a newline to stdout.

Flushes any remaining cache and prints a newline to stdout.

Flushes any remaining cache and prints a newline to stdout.