Run a shell command in cwd, capturing output.

Run a shell command in cwd, capturing output.

Read ``stop_after`` SSE frames from an ``httpx`` streaming response. Ignores heartbeat comment lines (``: ping``). Returns a list of dicts with

``GET /benchmarks/{id}`` → parsed JSON dict.

``POST /ingest`` → ``IngestResponse``. Drains any pending outbox first (preserving each line's original ``client_ts``), then sends ``

Compact two-chart × two-group BenchmarkConfig for chart-aware tests.

Build a ``LiveBenchClient`` whose underlying httpx client uses ``transport``.

Return a configured logger for the given module name.

Fan out ``event`` to every subscriber for ``session_id``. Synchronous — called from request handlers running inside the asyncio loop.

Upsert the multi-chart session into the TestClient's DB.

Return every chart in this benchmark as a flat list. For the multi-chart form, returns ``self.charts`` directly (each chart already c

``GET /benchmarks`` → parsed JSON dict.

Wait until available RAM stabilizes (N consecutive readings within tolerance).

Build a compact but complete ``BenchmarkConfig`` for tests.

Current wall-clock time as Unix epoch milliseconds (UTC).

``POST /benchmarks/{id}/reset`` → parsed JSON dict.

Unload all Ollama models from VRAM.

Parse one JSONL line into a ``DataPoint``. Stamps ``client_ts = now_ts()`` immediately if the line does not carry its own. The stamping happe

Build + POST an ``IngestRequest``; return (points_sent, groups_completed). On transport error the failure is logged and counted, but the stream

Yield one ``DataPoint`` per non-empty line, stamped at read time.

POST ``body`` to ``path`` with exponential backoff on 5xx. Returns parsed JSON on 2xx. Raises ``LiveBenchError`` immediately on 4xx (

Wait for available RAM to stabilize, return average.

Wait for available RAM to stabilize, return average.

Convert a datetime (assumed UTC if naive) to Unix-ms integer.

Emit all 4 chart values + metadata, then check completion.

Emit all 4 chart values + metadata, then check completion.

Emit all 3 chart values + metadata, then check if this prompt is now complete.

Emit all 3 chart values + metadata, then check if this prompt is now complete.

Emit a memory measurement to the memory chart.

Emit a memory measurement to the memory chart.

Compute set-level F1 between predicted and ground-truth node sets.

Compute set-level F1 between predicted and ground-truth node sets.

Generate a synthetic GraphWalks-style sample with a 'parents' query. Creates a random directed graph, picks a target node that has at least 1

Generate a synthetic GraphWalks-style sample with a 'parents' query. Creates a random directed graph, picks a target node that has at least 1

Compute memory delta from baseline. Returns GB consumed.

Compute memory delta from baseline. Returns GB consumed.

Take a stable baseline reading of available RAM.

Take a stable baseline reading of available RAM.

Convert a Unix-ms integer to a timezone-aware UTC datetime.

Single Ollama /api/chat call with keep_alive=-1. Always returns the same metrics dict shape as MLXRunner.generate(), so the same `run_warm_is

Single Ollama /api/chat call with keep_alive=-1. Always returns the same metrics dict shape as MLXRunner.generate(), so the same `run_warm_is

Parse the ISO-8601 TEXT representation we store for ``client_ts``.

Extract node IDs from 'Final Answer: [node1, node2, ...]' in the response.

Extract node IDs from 'Final Answer: [node1, node2, ...]' in the response.

Build a ``DataPointStored`` from a ``datapoints`` row. ``session_id`` may be supplied explicitly when the SELECT did not include that column

Run a single (prompt, settings) measurement with warm-isolated semantics. Flow: 1. One warmup call with `warmup_prompt` — discarded. This

Run a single (prompt, settings) measurement with warm-isolated semantics. Flow: 1. One warmup call with `warmup_prompt` — discarded. This

Run a single (prompt, settings) measurement with warm-isolated semantics. Flow: 1. One warmup call with `warmup_prompt` — discarded. This

Run a single (prompt, settings) measurement with warm-isolated semantics. Flow: 1. One warmup call with `warmup_prompt` — discarded. This

Store the prompt text for a group (called once per prompt).

Store the prompt text for a group (called once per prompt).

Append the raw JSON request body as one line to the outbox. ``body`` is the already-serialised ``IngestRequest`` from the caller, so

Current system available RAM in GB.

Current system available RAM in GB.