MCPcopy
hub / github.com/Audio-AGI/AudioSep / tokenizer

Function tokenizer

models/CLAP/training/infer_demo.py:18–26  ·  view source on GitHub ↗
(text)

Source from the content-addressed store, hash-verified

16
17
18def tokenizer(text):
19 result = tokenize(
20 text,
21 padding="max_length",
22 truncation=True,
23 max_length=77,
24 return_tensors="pt",
25 )
26 return {k: v.squeeze(0) for k, v in result.items()}
27
28
29PRETRAINED_PATH = "/mnt/fast/nobackup/users/hl01486/projects/contrastive_pretraining/CLAP/assets/checkpoints/epoch_top_0_audioset_no_fusion.pt"

Callers 1

infer_textFunction · 0.70

Calls 1

tokenizeFunction · 0.85

Tested by

no test coverage detected