Returns list of utf-8 byte and a corresponding list of unicode strings. The reversible bpe codes work on unicode strings. This means you

Return set of symbol pairs in a word. Word is represented as tuple of symbols (symbols being variable-length strings).

Merge results into one file. Arguments: - prefix (:obj:`str`): dir/results.rank

From the pytorch discussion Forum: https://discuss.pytorch.org/t/soft-cross-entropy-loss-tf-has-it-does-pytorch-have-it/69501

Update the dict 'dest' recursively. Elements in src could be a callable function with signature f(key, curr_dest_val)

Dequeue the oldest batch and add the latest one Args: batch: The latest batch of keys to add to the memory bank.

Dequeue the oldest batch and add the latest one Args: batch: The latest batch of keys to add to the memory bank.

Initialize the memory bank if it's empty Args: dim: The dimension of the which are stored in the bank.

Initialize the memory bank if it's empty Args: dim: The dimension of the which are stored in the bank.

broadcast model parameters

checkpoint forward

checkpoint forward