Two-chart × two-group benchmark with DIFFERENT metric keys per chart. Chart ``recall`` owns metric ``recall``; chart ``latency`` owns metric

Render a Pydantic ``client_ts`` field back to its wire TEXT form.

Merge new into existing. For nested dicts (like 'responses'), merge recursively.

Resolve the default benchmarks directory relative to apps/backend. ``apps/backend/src/live_bench_server/main.py`` → go up 4 parents to reach

Replay any pending outbox lines through ``/ingest``. Each line is a JSON-serialised ``IngestRequest`` from a previous run. Lines that

Idempotent additive migration for the chart_id columns. Detects whether the existing dev DB pre-dates the multi-chart schema and ALTERs the t

Extract token counts, tool calls, turns, and response text from agent_end.

Extract token counts, tool calls, turns, and response text from agent_end.

Find and parse the agent_end JSON event from Pi's stdout.

Find and parse the agent_end JSON event from Pi's stdout.

Parse an ISO-8601 string into an aware UTC ``datetime``. Accepts ``Z`` and ``+00:00`` suffixes; naive inputs are assumed UTC.

Render one chart's matrix + dataset status bar. Uses the shared ``latest`` map keyed by ``(chart_id, group_key, metric_key)``.

Return the chart list, synthesising a default chart for legacy configs. Mirrors ``BenchmarkConfig.all_charts()`` from the Pydantic side: if the

Render a datetime as fixed-width ISO-8601 UTC with microsecond precision. - Naive datetimes are assumed UTC. - Aware datetimes are converted

Return every benchmark defined in this file as a flat list. Hides the single-vs-multi shape from callers — they always get a list.

Current system available RAM in GB.

Benchmark Gemma 4 MLX via long-lived MLXRunner at each context size.

Benchmark Gemma 4 MLX via long-lived MLXRunner at each context size.

Benchmark Gemma 4 MLX via long-lived MLXRunner.

Benchmark Gemma 4 MLX via long-lived MLXRunner.

Benchmark every Ollama-hosted MLX backend at each context size.

Benchmark Qwen MLX via Ollama at each context size.

Benchmark all Ollama-hosted backends, emitting per-prompt results.

Benchmark all Ollama-hosted backends, emitting per-prompt results.

Fan out ``event`` to every subscriber across ALL sessions.

Build one realistic instance of every public model.

Remove a temp dir.

Remove a temp dir.

live-bench — push benchmark results at the server from anywhere on the LAN.

Close all subscriptions. Called on app shutdown.

Pull 7 samples from the OpenAI GraphWalks dataset.

Pull 7 samples from the OpenAI GraphWalks dataset.

Send one generate request and return the metrics dict.

Send one generate request and return the metrics dict.

Send one generate request and return the metrics dict.

Send one generate request and return the metrics dict.

Load the full ``BenchmarkConfig`` for a session, or ``None`` if missing.

Load pre-extracted GraphWalks samples keyed by context size key.

Load pre-extracted GraphWalks samples keyed by context size key.

Generate a 10-char hex node ID from a seed string, matching GraphWalks format.

Generate a 10-char hex node ID from a seed string, matching GraphWalks format.

Measure memory delta for MLX native backend.

Measure memory delta for an Ollama model.

Prefix a unique nonce so every measured run is a true cache miss.