MCPcopy Index your code
hub / github.com/ceifa/tiktoken-node / encode_batch

Method encode_batch

src/lib.rs:33–44  ·  view source on GitHub ↗
(&self, texts: Vec<String>)

Source from the content-addressed store, hash-verified

31
32 #[napi]
33 pub fn encode_batch(&self, texts: Vec<String>) -> Vec<Vec<u32>> {
34 texts
35 .par_iter()
36 .map(|text| {
37 self.encoding
38 .encode_with_special_tokens(&text)
39 .iter()
40 .map(|x| *x as u32)
41 .collect()
42 })
43 .collect()
44 }
45}
46
47#[napi]

Callers

nothing calls this directly

Calls

no outgoing calls

Tested by

no test coverage detected