MCPcopy Create free account
hub / github.com/AMAP-ML/EMF / insert_separator

Function insert_separator

blip3o/mm_utils.py:14–15  ·  view source on GitHub ↗
(X, sep)

Source from the content-addressed store, hash-verified

12 prompt_chunks = [tokenizer(chunk).input_ids for chunk in prompt.split("<image>")]
13
14 def insert_separator(X, sep):
15 return [ele for sublist in zip(X, [sep] * len(X)) for ele in sublist][:-1]
16
17 input_ids = []
18 offset = 0

Callers 1

tokenizer_image_tokenFunction · 0.85

Calls

no outgoing calls

Tested by

no test coverage detected