Controls the behavior of output pooling in pooling models.
| 1286 | |
| 1287 | |
| 1288 | class PoolerConfig: |
| 1289 | """Controls the behavior of output pooling in pooling models.""" |
| 1290 | |
| 1291 | pooling_type: Optional[str] = None |
| 1292 | """ |
| 1293 | The pooling method of the pooling model. |
| 1294 | """ |
| 1295 | # for embeddings models |
| 1296 | normalize: Optional[bool] = None |
| 1297 | """ |
| 1298 | Whether to normalize the embeddings outputs. Defaults to True. |
| 1299 | """ |
| 1300 | dimensions: Optional[int] = None |
| 1301 | """ |
| 1302 | Reduce the dimensions of embeddings if model |
| 1303 | support matryoshka representation. Defaults to None. |
| 1304 | """ |
| 1305 | enable_chunked_processing: Optional[bool] = None |
| 1306 | """ |
| 1307 | Whether to enable chunked processing for long inputs that exceed the model's |
| 1308 | maximum position embeddings. When enabled, long inputs will be split into |
| 1309 | chunks, processed separately, and then aggregated using weighted averaging. |
| 1310 | This allows embedding models to handle arbitrarily long text without CUDA |
| 1311 | errors. Defaults to False. |
| 1312 | """ |
| 1313 | max_embed_len: Optional[int] = None |
| 1314 | """ |
| 1315 | Maximum input length allowed for embedding generation. When set, allows |
| 1316 | inputs longer than max_embed_len to be accepted for embedding models. |
| 1317 | When an input exceeds max_embed_len, it will be handled according to |
| 1318 | the original max_model_len validation logic. |
| 1319 | Defaults to None (i.e. set to max_model_len). |
| 1320 | """ |
| 1321 | |
| 1322 | |
| 1323 | class EPLBConfig: |
no outgoing calls