Scores an iterable of tokenized examples. This method is built on top of :meth:`ctranslate2.Generator.score_batch` to efficiently score an arbitrarily large stream of data. It enables the following optimizations: * stream processing (the iterable is not fully materialized in memory
(
generator: Generator,
tokens: Iterable[List[str]],
max_batch_size: int = 64,
batch_type: str = "examples",
**kwargs,
)
| 165 | |
| 166 | |
| 167 | def generator_score_iterable( |
| 168 | generator: Generator, |
| 169 | tokens: Iterable[List[str]], |
| 170 | max_batch_size: int = 64, |
| 171 | batch_type: str = "examples", |
| 172 | **kwargs, |
| 173 | ) -> Iterable[ScoringResult]: |
| 174 | """Scores an iterable of tokenized examples. |
| 175 | |
| 176 | This method is built on top of :meth:`ctranslate2.Generator.score_batch` |
| 177 | to efficiently score an arbitrarily large stream of data. It enables |
| 178 | the following optimizations: |
| 179 | |
| 180 | * stream processing (the iterable is not fully materialized in memory) |
| 181 | * parallel scoring (if the generator has multiple workers) |
| 182 | * asynchronous batch prefetching |
| 183 | * local sorting by length |
| 184 | |
| 185 | Arguments: |
| 186 | tokens: An iterable of tokenized examples. |
| 187 | max_batch_size: The maximum batch size. |
| 188 | batch_type: Whether :obj:`max_batch_size` is the number of "examples" or "tokens". |
| 189 | **kwargs: Any score options accepted by |
| 190 | :meth:`ctranslate2.Generator.score_batch`. |
| 191 | |
| 192 | Returns: |
| 193 | A generator iterator over :class:`ctranslate2.ScoringResult` instances. |
| 194 | """ |
| 195 | yield from _process_iterable( |
| 196 | generator.score_batch, |
| 197 | [tokens], |
| 198 | max_batch_size, |
| 199 | batch_type, |
| 200 | **kwargs, |
| 201 | ) |
| 202 | |
| 203 | |
| 204 | def translator_generate_tokens( |
nothing calls this directly
no test coverage detected