MCPcopy Create free account
hub / github.com/togethercomputer/OpenChatKit / build_deberta_tokenizer

Function build_deberta_tokenizer

training/modules/tokenizer.py:15–17  ·  view source on GitHub ↗
(args)

Source from the content-addressed store, hash-verified

13 return tokenizer
14
15def build_deberta_tokenizer(args):
16 tokenizer = DebertaV2Tokenizer.from_pretrained(args.tokenizer_name)
17 return tokenizer
18

Callers

nothing calls this directly

Calls 1

from_pretrainedMethod · 0.45

Tested by

no test coverage detected