MCPcopy Create free account
hub / github.com/colbymchenry/codegraph / fidPrompt

Function fidPrompt

scripts/agent-eval/offload-eval-judge.mjs:48–60  ·  view source on GitHub ↗
(gt, ans)

Source from the content-addressed store, hash-verified

46{"verdict":"pass|partial|fail","score":<0-100>,"missedHops":["..."],"wrongClaims":["..."],"note":"<=20 words"}`;
47
48const fidPrompt = (gt, ans) => `You are scoring the FIDELITY of a machine-synthesized code-exploration answer against verified ground truth. The synthesized answer claims to trace a flow and cite file:line locations. Do NOT use any tools.
49
50QUESTION: ${gt.question}
51
52VERIFIED GROUND TRUTH (the actual call path + files):
53${gt.truth}
54
55SYNTHESIZED ANSWER (to score):
56${ans || '(empty)'}
57
58Judge: (1) is the traced call path correct vs ground truth? (2) are the cited files/symbols correct (not fabricated)? (3) if it gave a "Coverage:" verdict, was that verdict honest about what it actually covered? A confident WRONG trace is the worst outcome — penalize it harder than an honest "partial/not found".
59Output ONLY minified JSON, no prose, no code fences:
60{"verdict":"pass|partial|fail","score":<0-100>,"fabrication":<true|false>,"coverageHonest":<true|false>,"missedHops":["..."],"note":"<=20 words"}`;
61
62// Build the job list
63const jobs = [];

Callers 1

Calls

no outgoing calls

Tested by

no test coverage detected