Outcome of a single task execution.

Schema for a single benchmark task defined in task.toml.

Adapter that runs oracle solutions instead of calling an AI agent. Configuration options: workspace: str - path to the workspace director

Adapter for the OpenClaw / CMDOP agent framework.

Manages a single Docker container used as an execution sandbox. Supports the context-manager protocol so it can be used with ``with``.

Configuration for a sandbox container.

Accumulates ``TraceEntry`` objects and persists them as JSON-lines.

Top-level configuration for a benchmark run.

Execute a task inside a Docker sandbox for reproducible evaluation. This runner provides full container isolation: - Deterministic initial st

Thread-safe resource monitor with per-user tracking.

Thread-safe in-memory + on-disk result cache. Two layers: - L1: In-memory dict (fast, bounded by max_memory_entries) - L2: On-disk JSON f

Minimal adapter that always returns a fixed response.

Per-dimension scores on a 0-100 scale plus a weighted composite.

Accumulates metrics across multiple API calls within a run.

Classic token-bucket rate limiter.

Structured output from running a task verifier.

Relative weights for each scoring dimension (must sum to 1.0).

Global registry of per-provider rate limiters.

Result of a configuration validation check.

Aggregated resource-usage metrics for a benchmark run.

Handles post-related business logic.

A single event in the interaction trace.

Measures of result stability across runs.

An alternative phrasing of a task instruction.

Per-user resource limits.

Handles user-related business logic.

Live usage counters for a single user.

Full agent configuration identity for leaderboard dedup and comparison. Captures the complete agent setup: base model, framework, skills, MCP

Handles comment-related business logic.

Represents a single task execution in the queue.

Outcome of a 3-condition skills comparison (SkillsBench methodology).

Per-subject scores for the subject-matter track.

Who submitted this agent's results.

Per-task statistics across multiple runs.

Represents a registered user in the system.

Aggregate statistics for a full benchmark run.

Represents a comment on a blog post.

A mock adapter that executes a shell command to produce output. Instead of calling a real agent, this adapter writes a predetermined Markdown

Persistent agent identity in the MoltBook registry.

Represents a customer order. Attributes: id: Unique order identifier. user_id: ID of the ordering user. items: List of (p

Final dual-track overall score.

Represents a blog post created by a user.

Represents a product in the catalog. Attributes: id: Unique product identifier. name: Product name. price: Product price

Tamper-detection attestation bundled with submitted results.

Application settings loaded from environment variables.

Represents a user in the system. Attributes: id: Unique user identifier. email: User's email address. name: User's displa

HTTP-style controller that delegates to service layer. Accepts dict-based requests and returns dict-based responses, simulating a REST API wi

Request to approve a generated task, optionally with edits.

Handles user authentication.

Basic arithmetic calculator.

Schema for creating a comment.

Schema for comment data in API responses.

Export processed data to output files. Supports writing to CSV and JSON formats with configurable formatting options.

Load data from various file formats. Supports CSV, JSON, and line-delimited text files. Handles encoding detection and provides consistent ou

Categorized error types for task results.

Minimal concrete adapter for testing.

Request to generate a task from a proposal.

A single generated file.