MCPcopy Create free account
hub / github.com/NVIDIA/SOL-ExecBench / execute

Method execute

src/sol_execbench/driver/problem_packager.py:191–214  ·  view source on GitHub ↗

Stage execution files and return the command to run. Writes eval_driver.py, definition.json, workload.jsonl, solution.json to output_dir. For Python solutions, also writes source files. For C++ solutions, expects benchmark_kernel.so to already exist in output_dir (pr

(self)

Source from the content-addressed store, hash-verified

189 return cmd, artifact_path
190
191 def execute(self) -> list[str]:
192 """Stage execution files and return the command to run.
193
194 Writes eval_driver.py, definition.json, workload.jsonl, solution.json
195 to output_dir. For Python solutions, also writes source files. For C++
196 solutions, expects benchmark_kernel.so to already exist in output_dir
197 (produced by a prior compile() call).
198
199 The CLI should run the command in output_dir.
200 Trace JSON will be emitted on stdout (one JSON object per line).
201 """
202 if self._is_cpp:
203 so_path = self.output_dir / "benchmark_kernel.so"
204 if not so_path.exists():
205 raise FileNotFoundError(
206 f"benchmark_kernel.so not found at {so_path} — "
207 "run compile() first for C++/CUDA solutions"
208 )
209
210 (self.output_dir / "eval_driver.py").write_text(
211 (_TEMPLATES_DIR / "eval_driver.py").read_text()
212 )
213
214 return ["python", "eval_driver.py"]
215
216 def convert_stdout_to_traces(self, stdout: str) -> list[Trace]:
217 """Parse JSONL stdout from eval_driver.py into Trace objects.

Callers 9

cliFunction · 0.95
test_e2eFunction · 0.95
test_reward_hack_e2eFunction · 0.95
test_exampleFunction · 0.95
test_returns_commandMethod · 0.80
test_cudnn_rmsnormFunction · 0.80

Calls

no outgoing calls

Tested by 8

test_e2eFunction · 0.76
test_reward_hack_e2eFunction · 0.76
test_exampleFunction · 0.76
test_returns_commandMethod · 0.64
test_cudnn_rmsnormFunction · 0.64