MCPcopy Create free account
hub / github.com/QodeXcli/QodeX / estimateTokens

Function estimateTokens

src/utils/compaction.ts:68–89  ·  view source on GitHub ↗
(messages: Message[])

Source from the content-addressed store, hash-verified

66Drop:
67- Greetings, pleasantries, acknowledgments
68- The agent's intermediate narrations ("let me look at..." / "now I'll...")
69- Long tool outputs (replace with "[examined X]" if relevant)
70- Code blocks longer than 3 lines (replace with a 1-line description)
71
72Output format: plain markdown, 200-500 tokens TOTAL. Start with "${SUMMARY_PREFIX}".
73DO NOT explain what you're doing — just produce the summary.`;
74
75export function estimateTokens(messages: Message[]): number {
76 let total = 0;
77 for (const m of messages) {
78 const content: any = m.content;
79 if (typeof content === 'string') total += countTokens(content);
80 else if (Array.isArray(content)) {
81 for (const part of content) {
82 if (typeof part === 'object' && part && 'text' in part && typeof part.text === 'string') {
83 total += countTokens(part.text);
84 }
85 }
86 }
87 if ('tool_calls' in m && Array.isArray((m as any).tool_calls)) {
88 for (const tc of (m as any).tool_calls) {
89 total += countTokensJson(tc);
90 }
91 }
92 // Per-message overhead (role tags, delimiters) — ~4 tokens each in chat formats.

Callers 5

compactMessagesFunction · 0.70
shouldCompactFunction · 0.70
index.tsFile · 0.50

Calls 2

countTokensFunction · 0.85
countTokensJsonFunction · 0.85

Tested by

no test coverage detected