(gt, ans)
| 46 | {"verdict":"pass|partial|fail","score":<0-100>,"missedHops":["..."],"wrongClaims":["..."],"note":"<=20 words"}`; |
| 47 | |
| 48 | const fidPrompt = (gt, ans) => `You are scoring the FIDELITY of a machine-synthesized code-exploration answer against verified ground truth. The synthesized answer claims to trace a flow and cite file:line locations. Do NOT use any tools. |
| 49 | |
| 50 | QUESTION: ${gt.question} |
| 51 | |
| 52 | VERIFIED GROUND TRUTH (the actual call path + files): |
| 53 | ${gt.truth} |
| 54 | |
| 55 | SYNTHESIZED ANSWER (to score): |
| 56 | ${ans || '(empty)'} |
| 57 | |
| 58 | Judge: (1) is the traced call path correct vs ground truth? (2) are the cited files/symbols correct (not fabricated)? (3) if it gave a "Coverage:" verdict, was that verdict honest about what it actually covered? A confident WRONG trace is the worst outcome — penalize it harder than an honest "partial/not found". |
| 59 | Output ONLY minified JSON, no prose, no code fences: |
| 60 | {"verdict":"pass|partial|fail","score":<0-100>,"fabrication":<true|false>,"coverageHonest":<true|false>,"missedHops":["..."],"note":"<=20 words"}`; |
| 61 | |
| 62 | // Build the job list |
| 63 | const jobs = []; |
no outgoing calls
no test coverage detected