MCPcopy Create free account
hub / github.com/cosdata/cosdata / hash_token

Function hash_token

tests/test-tf-idf-bm25.py:119–120  ·  view source on GitHub ↗
(token: str)

Source from the content-addressed store, hash-verified

117
118
119def hash_token(token: str) -> int:
120 return xxhash.xxh32(token.encode("utf-8")).intdigest() & 0xFFFFFFFF
121
122
123def construct_sparse_vector(tokens: List[str]) -> Tuple[List[Tuple[int, int]], int]:

Callers 1

construct_sparse_vectorFunction · 0.85

Calls

no outgoing calls

Tested by

no test coverage detected