MCPcopy Create free account
hub / github.com/AI45Lab/Code / estimate_tokens

Function estimate_tokens

core/src/compaction.rs:227–229  ·  view source on GitHub ↗

Rough token estimation (~4 chars per token for English/code)

(text: &str)

Source from the content-addressed store, hash-verified

225
226/// Rough token estimation (~4 chars per token for English/code)
227fn estimate_tokens(text: &str) -> usize {
228 text.len() / 4
229}
230
231// ============================================================================
232// Tests

Callers 2

prune_tool_outputsFunction · 0.70

Calls 1

lenMethod · 0.45

Tested by 1