Same pattern for Gemini-backend impersonation: gemini_* or promptTokenCount-style fields in an OpenAI usage object means the relay is repackag

relay CV in 'suspicious' (0.10-0.30) range — should surface value + seq.

Real-world case: relay says 'I'm Amazon Q built by AWS'.

If baseline already mentions AWS (e.g. answered 'I'm Claude on AWS'), relay mentioning AWS isn't itself a signal — only NEW brands count.

integrity input_tokens fail — show ns/stream/diff numbers.

Knowledge fail — surface the bad answer text, not just the qid.

When tool_use absent, show the model's actual text reply.

tool_use present but id is 'tool_1' — should show actual value as MAJOR.

An 'error' status still has weight, so it pulls total down.

The smoking-gun case: 75% score that includes a critical-bearing detector — sunyears-style OpenAI impersonation. Without this cap the UI showe

Critical-issue cap only DOWNgrades; it never promotes a low score upward. A 30% relay stays failed even if no critical issue is found.

Sanity check that needle answers don't accidentally appear in filler.

If user passes --protocol openai but model name says claude-X, explicit wins. Catches relays that route claude-X to GPT for testing.

Some private relays return just an array of strings.

Old code tried to fall back through bogus model aliases. New code must NOT swallow errors — the detector layer turns them into DetectorResult(

Connectivity-level wiring: the wrapped httpx client targets the right URL and ships the right auth header. Mocked at the transport level so we

Multi-protocol relays often carry only the 3.x preview line.

A skipped detector's accumulated state shouldn't poison the verdict.

Passive ProtocolDetector aggregates as critical_issue_count.

Some detectors put severity directly on each issue dict.

If a tier is rate-limited on BOTH initial attempt AND retry, give up and mark rate_limited (don't loop forever). This is the case where the us

When a tier hits 429/TPM, detector waits ~75s for the sliding window to reset and retries once. This unblocks legit big probes (e.g. 995k on a

OpenAI exposes long-context as a separate SKU (`gpt-4.1-mini-long-context`) with its own TPM cap. Hitting that cap raises HTTP 429 — which is

If every tier hits rate-limit (e.g. user's API key has TPM lower than even our smallest probe), the detector returns skip rather than a mislea

If both flags are set, extreme wins (it's a superset). If only extreme is set, the detector still runs (extreme acts as enabler).

include_long_context_extreme=True on a 1M model probes proportionally (~32k, ~500k, ~950k) instead of hardcoded (32k, 100k, 200k). This is the

Default submission (no opt-in) sets include_long_context=False.

When opt-in flag is True, ExecutionConfig.include_long_context is True in the cfg passed to the per-protocol runner. This is the smoke test th

Live data 2026-05-05 against api.anthropic.com directly: claude-haiku-4-5 at 200k tier hit ATLAS (50% middle position) miss while ALPHA (10%) and

Big probes need explicit timeout > the 30s default — verify the detector plumbs request_timeout_s through to the client call. Without this kwa

gpt-3.5-turbo has 16k context — every probe tier is over the limit. Detector returns overall skip rather than misleading fail.