Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/atomicdotdev/atomic
/ size_hint
Method
size_hint
atomic-core/src/diff/token/tokenizer.rs:540–544 ·
view source on GitHub ↗
(&self)
Source
from the content-addressed store, hash-verified
538
}
539
540
fn size_hint(&self) -> (usize, Option<usize>) {
541
let remaining = self.content.len() - self.position;
542
// At minimum 0 tokens, at maximum one token per byte
543
(0, Some(remaining))
544
}
545
}
Callers
1
test_tokenizer_size_hint
Function · 0.45
Calls
1
len
Method · 0.45
Tested by
1
test_tokenizer_size_hint
Function · 0.36