File performance_benchmark.py

tests/performance_benchmark.py:None–None · view source on GitHub ↗

Source from the content-addressed store, hash-verified

1	#!/usr/bin/env python3
2	"""
3	Rigorous performance comparison between TokenDagger and TikToken.
4	Tests various edge cases, input lengths, and tokenization scenarios.

nothing calls this directly

mainFunction · 0.70

no test coverage detected