Function get_preprocessor

tests/data/test_preprocessor.py:11–13 · view source on GitHub ↗

()

Source from the content-addressed store, hash-verified

9
10
11	def get_preprocessor():
12	tokenizer = build_tokenizer("Qwen/Qwen2-7B")
13	return MultiModalPreprocessor(tokenizer=tokenizer)
14
15
16	def _test_tokenization(messages, n_at_start=None, preprocessor=None):

_test_tokenizationFunction · 0.85

test_multi_messageFunction · 0.85

build_tokenizerFunction · 0.90

MultiModalPreprocessorClass · 0.90

no test coverage detected