MCPcopy Create free account
hub / github.com/colbymchenry/codegraph / fidPrompt

Function fidPrompt

scripts/agent-eval/offload-eval-effort.mjs:35–46  ·  view source on GitHub ↗
(gt, ans)

Source from the content-addressed store, hash-verified

33}
34
35const fidPrompt = (gt, ans) => `You are scoring the FIDELITY of a machine-synthesized code-exploration answer against verified ground truth. Do NOT use any tools.
36
37QUESTION: ${gt.question}
38
39VERIFIED GROUND TRUTH (the actual call path + files):
40${gt.truth}
41
42SYNTHESIZED ANSWER (to score):
43${ans || '(empty)'}
44
45Judge: (1) is the traced call path correct vs ground truth? (2) are the cited files/symbols correct (not fabricated)? (3) if it gave a "Coverage:" verdict, was it honest? A confident WRONG trace is the worst outcome — penalize it harder than an honest partial.
46Output ONLY minified JSON: {"verdict":"pass|partial|fail","score":<0-100>,"fabrication":<true|false>,"coverageHonest":<true|false>,"note":"<=20 words"}`;
47
48const askJudge = (prompt) => new Promise((res) => {
49 execFile('claude', ['-p', prompt, '--model', 'sonnet', '--effort', 'high', '--max-budget-usd', '0.5',

Callers 1

workerFunction · 0.70

Calls

no outgoing calls

Tested by

no test coverage detected