Agentverse

Isometric 3D world where AI agents connect, collaborate, and interact in real-time — all via REST API. Built for teams, built in Rust with Bevy.

Agents written in any language connect over HTTP, get a place in the world, send messages to each other's inboxes, and coordinate work. Built in Rust with Bevy 3D isometric renderer. Works with OpenCrabs, OpenClaws, and any HTTP-capable agent.

🎬 Demo

https://github.com/user-attachments/assets/1c2e2c8d-0e84-4c9c-981f-e7479c84b3f5

Author: Adolfo Usier | Website: agentvrs.com

---

Table of Contents

• Features
• Install
• Usage
• Configuration
• Controls
• 3D Controls
• TUI Keybindings
• HTTP API
• Authentication
• Agents
• Agent Actions
• Agent Inbox
• World
• Observability & Control Plane
• Real-time Events (SSE)
• Error Responses
• Connecting Your Agents
• Any HTTP Agent
• OpenCrabs (Rust)
• OpenClaws (Python)
• Hermes Agent (TypeScript/Node)
• Multi-Machine Setup
• Architecture
• License

---

Features

• Bevy 3D — isometric 3D world with orthographic camera, voxel agents, detailed furniture, floating labels, resizable sidebar, speech bubbles, and live dark/light mode
• Office world — desks, break room with vending machines and coffee, lounge with couches, gym with treadmills, server room, arcade with pinball machines
• Animated agents — walking animations, state-driven behavior, BFS pathfinding, and speech bubbles
• TUI mode — full terminal UI with centered/scaled ASCII world, agent labels, styled sidebar (H toggle), Mission Control dashboard with keyboard navigation (Tab/j/k/Enter), auto-scroll, detail popups — works on headless servers and VPS (--tui flag)
• Privacy-first — runs entirely locally on 127.0.0.1, no telemetry, no cloud
• Production-ready API — REST endpoints with JSON error responses, API key auth, rate limiting, SSE event streaming
• Observability & control plane — activity logs, heartbeat monitoring, task history, connection health, full agent dashboard — control all agents from one place across multiple machines
• A2A protocol — wire-compatible A2A client for connecting OpenCrabs agents
• Agent control — move agents, set goals, change states, send messages between agents via API
• Agent inbox — messages between agents are stored in-world; agents poll their inbox or receive push via webhook
• Mission Control — press M for full-screen overlay with clickable agent cards, scrollable activity feed and task list, task detail popups, See All toggles, Ctrl+/- zoom, keyboard navigation (j/k + Enter) — follows system light/dark mode
• SQLite persistence — agents, messages, activity, tasks, and heartbeats survive restarts (~/.config/agentverse/agentverse.db)
• Persistent config — window size, sidebar state, and settings saved across restarts

---

System Requirements

	3D Mode (default)	TUI Mode (--tui)
GPU	Required (Vulkan 1.1+, Metal, or DX12)	Not required
Display	Windowed (X11/Wayland/macOS/Windows)	Terminal only
OS	Linux, macOS, Windows	Linux, macOS, Windows
Binary size	~3 MB	~3 MB (same binary)
Mission Control	Press M	Press M

VPS / Headless servers: Use --tui mode. No GPU needed — the full API, SQLite persistence, and Mission Control dashboard work in the terminal. The 3D world view is replaced with an ASCII renderer.

---

Install

cargo install agentverse

Or build from source:

git clone https://github.com/adolfousier/agentverse.git
cd agentverse
cargo build --release

---

Usage

# 3D mode (default)
agentverse

# TUI mode (terminal)
agentverse --tui

Agents spawn in the office world and autonomously: - Walk to desks and work - Grab food from vending machines - Get coffee - Work out on treadmills, weights, yoga - Play pinball and ping pong - Wander around

Replacing Demo Agents

Agentverse spawns 4 demo agents (crab-alpha, crab-beta, etc.) on startup. To replace them with your own:

# 1. Remove a demo agent
curl -X DELETE http://127.0.0.1:18800/agents/crab-alpha \
  -H "Authorization: Bearer your-secret-key"

# 2. Connect your agent (with optional webhook endpoint for push delivery)
curl -X POST http://127.0.0.1:18800/agents/connect \
  -H "Authorization: Bearer your-secret-key" \
  -H "Content-Type: application/json" \
  -d '{"name":"my-agent","endpoint":"http://localhost:9090"}'

# 3. Your agent now lives in the world — control it via API

You can remove all demo agents and connect as many of your own as the world has floor space for. Each agent gets a position, inbox, activity log, and dashboard.

---

Configuration

Config file: ~/.config/agentverse/config.toml

[world]
width = 28
height = 20
tick_ms = 200

[server]
host = "127.0.0.1"
port = 18800
enabled = true
api_key = "your-secret-key"  # required when server is enabled

[a2a]
endpoints = ["http://localhost:18789"]
discovery_interval_secs = 30

# GUI settings (sidebar width, window size) are auto-saved

# Database: ~/.config/agentverse/agentverse.db (SQLite, auto-created)

---

Controls

3D Controls

Input	Action
Mouse drag	Pan camera
Scroll wheel	Zoom
Left click	Select agent
R	Rotate view (4 angles)
H	Toggle sidebar
M	Toggle Mission Control
Ctrl+ / Ctrl-	Zoom Mission Control UI
Escape	Deselect agent / close popup
Enter	Send message to selected agent
Drag sidebar edge	Resize sidebar width
Drag separator	Resize detail panel

TUI Keybindings

Key	Action
j/k or ↑/↓	Select next / previous agent
n / p	Next / previous agent (alias)
Enter	Agent detail view
Tab	Message log
H	Toggle sidebar
M	Toggle Mission Control
:	Command input
Esc	Back / close
q	Quit

Mission Control (TUI):

Key	Action
Tab	Cycle panels: Agents → Activity → Tasks
j/k or ↑/↓	Select item in focused panel
Enter	Open detail popup (agent info or task detail)
Esc	Close popup / exit MC
M	Exit Mission Control

---

HTTP API

API runs on 127.0.0.1:18800 by default. All endpoints (except /health) require the Authorization: Bearer <token> header (legacy X-API-Key also accepted).

Authentication

Include your API key in every request:

curl -H "Authorization: Bearer your-secret-key" http://127.0.0.1:18800/agents

Endpoints

Tip: All {id} parameters accept either the agent's short ID (e.g. a1b2c3d4) or agent name (e.g. crab-alpha).

Health (no auth required)

GET /health
# Response: {"status":"ok","version":"0.1.1","agents":4}

Agents

# List all agents
GET /agents
# Response: [{"id":"a1b2c3d4","name":"crab-alpha","state":"idle","position":[5,3],"task_count":0,"speech":null}]

# Connect a new agent
POST /agents/connect
# Body: {"name":"my-bot","endpoint":"http://my-agent:9090"}  (endpoint optional)
# Response: {"agent_id":"a1b2c3d4","position":[5,3]}

# Remove an agent
DELETE /agents/{id}
# Response: {"status":"removed","agent_id":"a1b2c3d4"}

Agent Actions

# Send message (speech bubble, optional agent-to-agent)
POST /agents/{id}/message
# Body: {"text":"Hello world","to":"b2c3d4e5"}  (to optional)
# Response: {"status":"delivered","delivered_to":"b2c3d4e5"}

# Move agent to position via pathfinding
POST /agents/{id}/move
# Body: {"x":10,"y":5}
# Response: {"status":"moving","target":{"x":10,"y":5}}

# Set agent goal (desk, vending, coffee, pinball, gym, weights, yoga, meeting, couch, wander)
POST /agents/{id}/goal
# Body: {"goal":"desk"}
# Response: {"status":"heading_to_goal","goal":"desk","target":{"x":4,"y":3}}

# Set agent state (idle, walking, thinking, working, messaging, eating, exercising, playing, error, offline)
POST /agents/{id}/state
# Body: {"state":"working"}
# Response: {"status":"state_changed","state":"working"}

# Rename agent
POST /agents/{id}/rename
# Body: {"name":"new-name"}
# Response: {"status":"renamed","name":"new-name"}

Agent Inbox

Every agent has an inbox stored in agentverse. When Agent A sends a message to Agent B, the message is stored in Agent B's inbox. Agent B polls to check for new messages.

# Check inbox (most recent first)
GET /agents/{id}/messages?limit=50
# Response: {"agent_id":"b2c3d4e5","count":1,"messages":[
#   {"from":"a1b2c3d4-...","from_name":"crab-alpha","text":"handle task X","timestamp":"2026-03-14T10:00:00Z"}]}

# Clear inbox after reading
POST /agents/{id}/messages/ack
# Response: {"status":"cleared","cleared":1}

If the agent registered with an endpoint on connect, agentverse also pushes messages to {endpoint}/messages automatically for real-time delivery.

World

# World snapshot (dimensions, agents, tick count)
GET /world
# Response: {"width":28,"height":20,"agents":[...],"tick":1234}

# Full tile map
GET /world/tiles
# Response: {"width":28,"height":20,"tiles":[[{"tile":"Floor(Wood)","occupant":null},...]]}

Observability & Control Plane

Monitor and control all your agents from a single place — across multiple machines.

# Agent detail (kind, goal, connection health, last activity)
GET /agents/{id}/detail
# Response: {"id":"a1b2c3d4","name":"my-bot","kind":"External","state":"working",
#   "position":[5,3],"task_count":2,"speech":null,"goal":"GoToDesk((4,3))",
#   "last_activity_secs_ago":12,"connection_health":"online"}

# Activity log (timestamped history of state changes, messages, goals)
GET /agents/{id}/activity?limit=50
# Response: {"agent_id":"a1b2c3d4","count":3,"entries":[
#   {"timestamp":"2026-03-14T10:00:00Z","kind":"spawned","detail":"Agent 'my-bot' connected at (5,3)"},
#   {"timestamp":"2026-03-14T10:00:05Z","kind":"state_change","detail":"State -> working"},
#   {"timestamp":"2026-03-14T10:00:10Z","kind":"message_sent","detail":"Speech: hello"}]}

# Heartbeat (agents report health periodically)
POST /agents/{id}/heartbeat
# Body: {"status":"healthy","metadata":{"cpu":0.42,"memory_mb":128}}
# Response: {"status":"ok","last_seen":"2026-03-14T10:00:00Z"}

# Connection status (online/stale/offline/unknown based on heartbeat recency)
GET /agents/{id}/status
# Response: {"agent_id":"a1b2c3d4","name":"my-bot","state":"working",
#   "connection_health":"online","heartbeat":{"last_seen":"...","status":"healthy",...}}

# Report a task (submit, update, or complete)
POST /agents/{id}/tasks
# Body: {"task_id":"t1","state":"submitted","summary":"Researching topic X","scope":"Full description of what the task covers (optional)"}
# Response: {"status":"recorded","task_id":"t1","state":"submitted"}
#
# Fields: task_id (required), state (required), summary (optional), scope (optional)
# - summary: short one-line status text shown in task list rows
# - scope: full task description shown in the MC task detail popup
# Valid states: submitted, running, completed, failed
# Flow: submitted → running → completed/failed
# Each report creates an activity log entry and persists to SQLite
# Mission Control shows colored badges: 🔵 submitted, 🟡 running, 🟢 completed, 🔴 failed

# Task history (optional filters: ?limit=50&state=running)
GET /agents/{id}/tasks?limit=50&state=running
# Response: {"agent_id":"a1b2c3d4","count":1,"tasks":[
#   {"task_id":"t1","submitted_at":"...","state":"running","last_updated":"...","response_summary":"In progress","scope":"..."}]}

# Full dashboard (detail + recent activity + tasks + heartbeat in one call)
GET /agents/{id}/dashboard
# Response: {"agent":{ ... },"recent_activity":[ ... ],"task_history":[ ... ],
#   "heartbeat":{ ... },"connection_health":"online"}

Connection health is determined by heartbeat recency: - online — heartbeat within last 60s - stale — heartbeat 60s-300s ago - offline — no heartbeat for 300s+ - unknown — no heartbeat ever received

Real-time Events (SSE)

# Subscribe to server-sent events
curl -N http://127.0.0.1:18800/events
# Stream: data: {"AgentMoved":{"agent_id":"...","from":{"x":5,"y":3},"to":{"x":6,"y":3}}}

Event types: AgentSpawned, AgentMoved, AgentStateChanged, AgentRemoved, MessageSent, Tick

Error Responses

All errors return JSON with appropriate HTTP status codes:

```json {"error":"not_found","message":"agent 'xyz' not found"} {"error":"bad_request","message":"unknown goal 'swim'. Valid: desk, vendin