MCPcopy Index your code
hub / github.com/microsoft/BitNet / load_vocab

Method load_vocab

utils/convert.py:1462–1473  ·  view source on GitHub ↗
(self, vocab_types: list[str] | None, model_parent_path: Path)

Source from the content-addressed store, hash-verified

1460 return vocab
1461
1462 def load_vocab(self, vocab_types: list[str] | None, model_parent_path: Path) -> tuple[BaseVocab, gguf.SpecialVocab]:
1463 vocab: BaseVocab
1464 if vocab_types is None:
1465 vocab = NoVocab()
1466 else:
1467 vocab = self._create_vocab_by_path(vocab_types)
1468 # FIXME: Respect --vocab-dir?
1469 special_vocab = self._create_special_vocab(
1470 vocab,
1471 model_parent_path,
1472 )
1473 return vocab, special_vocab
1474
1475
1476def default_outfile(model_paths: list[Path], file_type: GGMLFileType) -> Path:

Callers 1

mainFunction · 0.95

Calls 3

_create_vocab_by_pathMethod · 0.95
_create_special_vocabMethod · 0.95
NoVocabClass · 0.70

Tested by

no test coverage detected