Pino proxy

A tiny local proxy to inspect, cache, and reshape Anthropic API requests.

See exactly what your Claude client ships — then cache and trim what it leaves on the table. Zero runtime dependencies, no build step, no SaaS. A ~500-line local reverse proxy in front of api.anthropic.com.

A tiny local HTTP reverse proxy in front of api.anthropic.com. It forwards everything to upstream untouched except /v1/messages requests, where it optionally:

• Logs raw request/response bodies so you can see precisely what Claude Code (or any client) sends — tool schemas, system blocks, cache breakpoints, TTLs. This observability is how every optimization below was found, and it's the proxy's most durable use.
• Auto-injects prompt-cache breakpoints on the chunks the client leaves uncached — the tools catalog (still shipped with zero breakpoints) and the static reminders block in messages[0].
• Normalizes cache TTL to 1h on stable content and reclaims breakpoint slots the client wastes on tiny blocks.
• Reshapes the request body via a user-supplied transform(body) hook — drop unused tools, strip ANSI escapes, override the model, rewrite system text.

Origin story

Pino started with exactly one job. Claude Code was shipping a large tool catalog (~25–29 schemas, ~15k tokens at its peak) uncached on every turn, so Pino injected a single prompt-cache breakpoint and turned that re-shipped block into a 0.1× cache read instead of a 1× re-bill — up to ~90% off on that chunk. That was the whole pitch: a 500-line proxy that cached what the client forgot to.

Then Anthropic shipped progressive tool disclosure in v2.1.172 (mid-June 2026). The inline catalog shrank (via ToolSearch lazy-loading) to ~5k tokens and system blocks gained a 1h TTL. The per-turn numbers changed; the mechanism didn't — that uncached catalog and the messages[0] anchor still ship at full price every turn.

The caching mission is intact — Anthropic shrank the prefix but still ships it uncached, so Pino still flips it to a 0.1× read every turn. And the proxy was always more than that one number: it's a local seam where you can see and rewrite anything in a /v1/messages request.

• Want to override the model? One env var (MODEL_OVERRIDE) — it even rewrites the model's name in the system prompt so its self-description stays consistent.
• Want to rewrite prompts or drop tools? A transform(body) hook gets the parsed body before it leaves your machine.
• Want to investigate what your client actually sends? LOG_BODIES=1 dumps every request and response to disk — which is how this entire writeup was produced.
• Still want the caching? It's still there: the tool catalog + messages[0] anchor are shipped uncached on every turn, and Pino caches them.

The 90% headline was the spark; the engine is a general-purpose inspect-and-modify proxy for Anthropic API traffic — caching included.

Status: per-turn overhead changed across Claude Code v2.1.158–183 — see how the overhead evolved

Over spring–summer 2026 Anthropic trimmed Claude Code's per-request overhead in two waves (measured from captured logs): the system prompt was cut ~3× around v2.1.158, then progressive tool disclosure (ToolSearch, v2.1.172) shrank the inline tool catalog from a ~15–16k-token peak to ~5k. System blocks now also ship at ttl: "1h". The tool catalog and the messages[0] anchor are still shipped uncached, and the logging/transform features are client-version-independent. The numbers below are measured from the logs and labeled by era.

How Claude Code's request overhead evolved

This project started because Claude Code re-shipped a large, uncached tool catalog on every request. Capturing raw request bodies across versions (Apr–Jun 2026) shows Anthropic trimmed the per-turn overhead in two separate waves, not one — and confirms the single gap that survived all of them. Token counts are calibrated against the real usage numbers in the paired responses (~0.23 tok/char for this payload); the catalog scales with loaded MCP servers, so treat these as a standard config.

Version (date)	Inline tools	system	tools breakpoints	What changed
v2.1.119 (Apr 27)	28 schemas, ~15.2k tok	~6.5k tok	0	baseline
v2.1.146	29 schemas, ~16.6k tok (peak)	~6.7k tok	0	catalog peak
v2.1.158 (late May)	28 schemas, ~12.9k tok	~2.3k tok	0	⬅ system prompt cut ~3×
v2.1.169 (Jun 9)	25 schemas, ~11.6k tok	~1.7k tok	0	last pre-ToolSearch
v2.1.172 (Jun 12)	lazy-loaded via ToolSearch	~1.9k tok	0	⬅ progressive tool disclosure¹
v2.1.177	~15 inline, ~7.4k tok	~1.9k tok	0	catalog still mid-shrink
v2.1.183	~8 inline, ~5.3k tok	~2.7k tok (1h)	0	settled

¹ gated behind the advanced-tool-use-2025-11-20 beta flag.

Three things stand out:

1. The catalog peaked at ~16.6k tokens (v2.1.146) — not the "~24k" earlier back-of-envelope estimates claimed. Peak static prefix (tools + system + msg[0]) was ~26.6k tokens at v2.1.119, down to ~10k by v2.1.183.
2. Two waves, weeks apart. The system prompt was cut ~3× around v2.1.158 (~6.7k → ~2.3k tok), well before any tool change. Then ToolSearch (v2.1.172) attacked the tool catalog — and even that shrank in stages (v2.1.177 still inlined ~15 tools).
3. tools carried zero cache breakpoints in every single version, Apr → Jun. System gained a 1h TTL; the tool catalog never gained a breakpoint. That is the durable gap Pino fills — alongside the still-uncached messages[0] reminders and a breakpoint slot the client still wastes on a ~57-char system block.

Bottom line: On current Claude Code (v2.1.183), Pino reclaims roughly ~7.5k uncached tokens/turn (tool catalog ~5.3k + msg[0] anchor ~2.4k) plus the wasted tiny-system slot — the pre-ToolSearch peak was ~20k. Confirm it for your version with the 60-second check — catalog size and TTL behavior vary by version and loaded MCP servers.

Quickstart

1. Clone and install

git clone https://github.com/alxsuv/pino
cd pino

No npm install needed — zero runtime dependencies. Requires Node >= 20.

2. Start the proxy

Linux / macOS (bash/zsh):

# pure pass-through on :8787 (great for just inspecting traffic)
LOG_BODIES=1 npm start

# caching + transforms + logs
AUTO_CACHE=1 \
TRANSFORM_FILE=./src/transforms/default.js \
LOG_BODIES=1 \
npm start

# or invoke the bin directly
node bin/pino-proxy.js

Windows (PowerShell):

$env:AUTO_CACHE=1
$env:TRANSFORM_FILE="./src/transforms/default.js"
$env:LOG_BODIES=1
npm start

Windows (cmd.exe):

set AUTO_CACHE=1
set TRANSFORM_FILE=./src/transforms/default.js
set LOG_BODIES=1
npm start

3. Point your client at it

Linux / macOS:

export ANTHROPIC_BASE_URL=http://127.0.0.1:8787

Windows (PowerShell):

$env:ANTHROPIC_BASE_URL="http://127.0.0.1:8787"

Windows (cmd.exe):

set ANTHROPIC_BASE_URL=http://127.0.0.1:8787

Verify it yourself (60 seconds)

You don't have to take any number here on faith — the proxy's whole point is that you can see what your client actually ships in 60 seconds, on your own version. (Pre-ToolSearch this check surfaced a ~15k-token tool catalog re-shipped uncached every turn; the current catalog is ~5k. The only way to know what your client and version do is to look.)

1. Start the proxy in pass-through mode (captures logs, mutates nothing): bash LOG_BODIES=1 npm start

2. Run any command in Claude Code (e.g., claude "hi").

3. Inspect the captured request: ```bash # How big is the inline tool catalog, and is it cached? jq '{tools: (.body.tools|length), uncached: ([.body.tools[]|select(.cache_control)]|length)}' logs/*.req.json

# System blocks: are TTLs set? Any slot wasted on a tiny block? jq '.body.system[]? | {len: (.text|length), cache_control}' logs/*.req.json

# Is the messages[0] anchor (CLAUDE.md / reminders) cached? jq '[.body.messages[0].content[]?|objects|select(.cache_control)]|length' logs/*.req.json ```

On v2.1.172 you'll typically see: ~8 tools, 0 cached; system blocks at ttl: "1h" (with one breakpoint wasted on a ~57-char block); and messages[0] with 0 breakpoints. The tool catalog and the msg[0] anchor are the two chunks still left on the table.

Heads up: the inline catalog isn't always small. On the occasional turn where ToolSearch actually loads a deferred tool, its full schema is shipped inline for that request — in one captured corpus, ~1 request in 40 ballooned from ~14 KB back to ~48 KB (21 tools). So don't be alarmed by an outlier; check several logs/*.req.json, not just one.

1. Enable caching and re-run the queries: bash AUTO_CACHE=1 LOG_BODIES=1 npm start You'll now see a breakpoint added to tools, every ephemeral block normalized to ttl: "1h" (except the rolling tail), the wasted tiny-block slot reclaimed, and anthropic-beta carrying extended-cache-ttl-2025-04-11.

2. Run another command and watch the hits in the response usage field — cache_read_input_tokens should now cover the tool catalog and msg[0] anchor instead of re-billing them.

How the caching works

The Anthropic API allows up to 4 cache breakpoints per request. Each tells the API "cache everything up to and including this block," so subsequent requests with the same prefix hit the cache at 0.1× base input price instead of 1×.

This proxy places them as follows (within the 4-slot ceiling):

1. Last tools entry → 1h TTL. The client still ships zero breakpoints on tools (true in every version Apr–Jun). Pre-ToolSearch this was a ~15k-token catalog (~16.6k at the v2.1.146 peak) and the single biggest win; on v2.1.183 it's ~5.3k tokens — uncached, and the biggest single chunk Pino still caches.
2. Last system block → 1h TTL, and strip wasteful breakpoints on system blocks <500 chars. On current Claude Code the system prompt already arrives at ttl: "1h", so the TTL rewrite is mostly a no-op there — but the client still burns a full slot on a ~57-char block (caching ~14 tokens), which this pass reclaims for the msg[0] reminders instead.
3. Last cacheable block of messages[0] → 1h TTL. The client stuffs static reminders (CLAUDE.md, skills catalog, deferred-tools list — ~1.7k+ tokens) into the first user message and leaves them uncached. Cached once per session. Still a real win on v2.1.172.
4. Rolling tail → 5m TTL by default (TAIL_TTL=1h to change). The last text/tool_result/image block across all messages. Moves each turn, so every new turn reads the prior turn's prefix from cache and only pays base price for the delta. 5m is the right default because the tail rarely survives an hour of reuse; paying the 2.0× write multiplier on a breakpoint that moves constantly is wasteful. (Recent Claude Code versions also place a 1h breakpoint near the tail themselves.)

Decide if caching pays off for your workload

The per-turn payoff depends on your client version and tool set. To check yours, Pino ships two zero-dependency analysis scripts that turn the LOG_BODIES dumps into hard numbers.

• scripts/usage-tally.mjs — tallies the real billed usage (input / cache_create / cache_read / output, with the per-TTL 5m-vs-1h split) from a directory of *.resp.log. Run a proxy-on session and a direct (pass-through) session over the same task, point it at both, and it diffs the actual tokens: bash node scripts/usage-tally.mjs logs/run-proxy-on logs/run-direct
• scripts/ab-cache-harness.mjs — replays a captured corpus through five breakpoint/TTL strategies against a simulated prompt cache, so you can compare designs without spending a token: bash node scripts/ab-cache-harness.mjs logs/<your-corpus>

What the numbers say on a modern (v2.1.183) client:

• The shipped caching design holds up. Across every alternative tried — selective 1h, dropping the rolling tail, breakpoints-only — the current strategy (1h on the stable tools/system/messages[0] prefix + a 5m rolling tail) was the best of the intervention options. The rolling tail does most of the work; removing it reprocesses all history at full price every turn.
• Dense, back-to-back traffic is the one profile to check first. When turns arrive faster than the client's 5m cache expires, its own breakpoints already cache well, so Pino's reshaping