MCPcopy Create free account
hub / github.com/NVIDIA/SOL-ExecBench / test_consistency

Function test_consistency

tests/examples/test_examples.py:276–296  ·  view source on GitHub ↗

Verify extracted source files match the content in definition.json and solution JSON.

(case: Example)

Source from the content-addressed store, hash-verified

274 [pytest.param(c, id=c.test_id) for c in _EXAMPLES],
275)
276def test_consistency(case: Example):
277 """Verify extracted source files match the content in definition.json and solution JSON."""
278 example_dir = _EXAMPLES_DIR / case.language / case.problem
279
280 # Check reference.py matches definition.json "reference" field
281 definition = json.loads((example_dir / "definition.json").read_text())
282 if "reference" in definition:
283 ref_path = example_dir / "reference.py"
284 assert ref_path.exists(), f"reference.py missing for {case.test_id}"
285 assert ref_path.read_text() == definition["reference"], (
286 f"reference.py does not match definition.json reference for {case.test_id}"
287 )
288
289 # Check each solution source file matches its "content" field
290 solution = json.loads((example_dir / case.solution_file).read_text())
291 for source in solution.get("sources", []):
292 src_path = example_dir / source["path"]
293 assert src_path.exists(), f"{source['path']} missing for {case.test_id}"
294 assert src_path.read_text() == source["content"], (
295 f"{source['path']} does not match {case.solution_file} content for {case.test_id}"
296 )

Callers

nothing calls this directly

Calls

no outgoing calls

Tested by

no test coverage detected