Called twice in inference generation loop to get the input_token_count and output_token_count. This step can be skipped by setting the OVConfig as follows: `from llmware.configs import OVConfig OVConfig().set_config("get_token_counts", False)` In our testin
(self, text)
| 4525 | return img_path |
| 4526 | |
| 4527 | def ov_token_counter(self, text): |
| 4528 | |
| 4529 | """ Called twice in inference generation loop to get the input_token_count and |
| 4530 | output_token_count. This step can be skipped by setting the OVConfig as follows: |
| 4531 | |
| 4532 | `from llmware.configs import OVConfig |
| 4533 | OVConfig().set_config("get_token_counts", False)` |
| 4534 | |
| 4535 | In our testing, the performance impact is negligible, but may be different in your |
| 4536 | environment and use case. |
| 4537 | |
| 4538 | If this is set to False, then no token counts will be provided in the usage totals. |
| 4539 | """ |
| 4540 | |
| 4541 | if self.tokenizer: |
| 4542 | toks = len(self.tokenizer.encode(text)) |
| 4543 | else: |
| 4544 | toks = 0 |
| 4545 | |
| 4546 | return toks |
| 4547 | |
| 4548 | def prompt_engineer(self, query, context, inference_dict): |
| 4549 |
no test coverage detected