Transform a human-readable string into a sequence of int ids. The ids should be in the range [num_reserved_ids, vocab_size). Ids [0, num_reserved_ids) are reserved. EOS is not appended. Args: s: human-readable string to be converted. Returns:
(self, s)
| 60 | return self._num_reserved_ids |
| 61 | |
| 62 | def encode(self, s): |
| 63 | """Transform a human-readable string into a sequence of int ids. |
| 64 | |
| 65 | The ids should be in the range [num_reserved_ids, vocab_size). Ids [0, |
| 66 | num_reserved_ids) are reserved. |
| 67 | |
| 68 | EOS is not appended. |
| 69 | |
| 70 | Args: |
| 71 | s: human-readable string to be converted. |
| 72 | |
| 73 | Returns: |
| 74 | ids: list of integers |
| 75 | """ |
| 76 | return [int(w) + self._num_reserved_ids for w in s.split()] |
| 77 | |
| 78 | def decode(self, ids, strip_extraneous=False): |
| 79 | """Transform a sequence of int ids into a human-readable string. |
no outgoing calls
no test coverage detected