MCPcopy Create free account
hub / github.com/colbymchenry/codegraph / e2ePrompt

Function e2ePrompt

scripts/agent-eval/offload-eval-judge.mjs:34–46  ·  view source on GitHub ↗
(gt, ans)

Source from the content-addressed store, hash-verified

32}
33
34const e2ePrompt = (gt, ans) => `You are scoring whether an AI coding agent correctly answered a code-flow question about a repository. Judge ONLY against the verified ground truth. Do NOT use any tools.
35
36QUESTION: ${gt.question}
37
38VERIFIED GROUND TRUTH (the actual call path + files):
39${gt.truth}
40
41AGENT'S ANSWER:
42${ans || '(empty)'}
43
44Score how correct the agent's answer is vs the ground truth. A "pass" means it identifies the core mechanism and the major hops with the right files/symbols and makes no materially wrong claim. "partial" = right area but misses major hops or has notable errors. "fail" = wrong layer, fabricated, or misses the mechanism.
45Output ONLY minified JSON, no prose, no code fences:
46{"verdict":"pass|partial|fail","score":<0-100>,"missedHops":["..."],"wrongClaims":["..."],"note":"<=20 words"}`;
47
48const fidPrompt = (gt, ans) => `You are scoring the FIDELITY of a machine-synthesized code-exploration answer against verified ground truth. The synthesized answer claims to trace a flow and cite file:line locations. Do NOT use any tools.
49

Callers 1

Calls

no outgoing calls

Tested by

no test coverage detected