
One Command. A Complete Team.
Claw AI Lab is a lab-native multi-agent research platform for interactive and scalable AI-driven science. It enables users to create a full AI research lab from a single prompt, with customizable roles, research directions, and collaborative workflows, rather than relying on a single-agent or fixed serial pipeline. Claw orchestrates multiple agents and projects in parallel through a FIFO-based scheduling framework, maximizing compute utilization while supporting cross-project knowledge sharing and mutual improvement. Crucially, the system keeps humans in the loop: users can intervene whenever needed, provide feedback under ambiguity, inject new ideas, and iteratively refine the research process through rollback and continuation. Combined with a simple UI that reduces everything to prompts and clicks, Claw transforms automated research into a more intuitive, steerable, and laboratory-like experience.
We welcome contributions from the community to make this project better together!
Each project autonomously produces a full research deliverable: Paper · Code · Figures · Experiment Logs
![]() |
![]() |
![]() |
| FlowEdit on Wan2.1-T2V Reproduce · Video Editing | Multi-Concept Customization Lab Explore · Discussion · Image Gen | Cross-Domain Dropout Analysis Lab Explore · Discussion · CV / NLP / Speech |
Multi-agent discussion on: "What is the most deployable direction for Video Action Models in Embodied AI?"
Agent A — World Model + MPC (Model Predictive Control) is the most industrially stable path.
Agent B — "Train with video, infer with action" is the most deployable policy paradigm.
Agent C — Execution monitoring & SOP (Standard Operating Procedure) automation lands fastest as a product.
Consensus: The most deployable form is not a single end-to-end model, but a layered, modular system — use video supervision during training to learn rich dynamics, output actions directly at inference for low latency, and layer planning/MPC/safety modules on top for closed-loop robustness and recovery.
Top 3 Research Directions (ranked by deployability)
| # | Direction | Deployability |
|---|---|---|
| 1 | Layered Video-Action Stack — video-action joint training + direct action inference + MPC safety | Highest — best balance of latency, interpretability & safety |
| 2 | Video-to-Plan / SOP — demo videos → step sequences & skill graphs for existing robots | High — smallest embodiment gap, clearest commercial path |
| 3 | Execution Monitor — real-time step tracking, anomaly detection, re-planning triggers | High — fastest to production; critical for industrial reliability |
Key Contradictions Resolved
| Debate | Resolution |
|---|---|
| World Model + MPC vs. Direct Action? | Combine both — world model for representation, direct action for control, MPC for safety |
| Human video: valuable or too much gap? | Pre-training yes; direct low-level transfer not yet reliable |
| Is monitoring a "real" action model? | Not the backbone, but fastest to reach production value |
→ Full Transcript · → Consensus Synthesis
| End-to-End Hierarchical Pipeline | Topic in → paper + code + figures out, 22 stages fully autonomous |
| Three Research Modes | Lab Explore · Lab Discussion (multi-agent debate) · Reproduce |
| Real-Time Dashboard | Live web UI with event logs, data shelf, and project monitoring |
| GPU & NPU Support | Dynamic scheduling on your own hardware (CUDA / Ascend NPU) |
git clone https://github.com/Claw-AI-Lab/Claw-AI-Lab.git
cd Claw-AI-Lab
# Create python environment
conda create -n clawailab python=3.11
conda activate clawailab
# Backend
cd backend/agent
pip install -e ".[all]"
pip install websockets
# Frontend
cd ../../frontend
npm install
# ML dependencies
# You can add more packages based on your research project
pip install torch torchvision diffusers transformers accelerate safetensors datasets \
huggingface_hub opencv-python pandas matplotlib scikit-image scipy einops tqdm
# OpenHands Beast Mode (optional, recommended)
pip install openhands
Fill in following configurations in examples/config_template.yaml:
llm:
provider: "openai-compatible"
api_key: "your-api-key"
primary_model: "gpt-5.4"
coding_model: "claude-opus-4-6"
image_model: "gemini-3-pro-image-preview"
fallback_models:
- "gpt-4o"
sandbox:
python_path: "/path/to/your/python3"
Thanks a lot for KOKONI's support for this project, and api_key can be obtained here.
./start.sh # Start all services
./start.sh stop # Stop
./start.sh restart # Restart
./start.sh status # Status check
./start.sh fresh # Clean restart (reset all data)
Open http://localhost:5903/ → Submit your research topic and let the agents work.
Tip: For better results, prepare local copies of codebases, datasets, and checkpoints, then input the local path when submitting a new project. Removing unused files in codebases significantly speeds up agent analysis.
Description of each configuration in examples/config_template.yaml.
Click to expand
# === Project ===
project:
name: "my-project" # Project identifier, used for directory naming and UI display
mode: "full-auto" # Pipeline mode: "full-auto" runs all stages without human gates
# === Research ===
research:
topic: "Your research topic" # The research topic or paper to reproduce (required)
domains: # Research domains for literature search scope
- "deep-learning"
daily_paper_count: 5 # Number of papers to retrieve per search query
quality_threshold: 3.0 # Minimum relevance score (1-5) for literature screening
# === Runtime ===
runtime:
timezone: "Asia/Shanghai" # Timezone for timestamps in logs and reports
max_parallel_tasks: 1 # Max concurrent tasks per agent (keep 1 for stability)
approval_timeout_hours: 1 # Timeout for human approval at gate stages
retry_limit: 2 # Number of retries on stage failure before giving up
# === Notifications ===
notifications:
channel: "console" # Notification channel: "console" | "discord" | "slack"
target: "" # Channel target (e.g. Discord webhook URL, leave empty for console)
on_stage_start: true # Notify when a stage begins
on_stage_fail: true # Notify when a stage fails
on_gate_required: true # Notify when human approval is needed
# === Knowledge Base ===
knowledge_base:
backend: "markdown" # Storage format: "markdown" | "obsidian"
root: "docs/kb" # Root directory for knowledge base files
# === OpenClaw Bridge ===
openclaw_bridge:
use_cron: false # Enable scheduled research runs
use_message: false # Enable progress notifications via messaging platforms
use_memory: false # Enable cross-session knowledge persistence
use_sessions_spawn: false # Enable spawning parallel sub-sessions
use_web_fetch: false # Enable live web search during literature review
use_browser: false # Enable browser-based paper collection
# === LLM ===
llm:
provider: "openai-compatible" # LLM provider: "openai-compatible" | "openai" | "deepseek" | "acp"
base_url: "https://api.example.com/v1" # API endpoint (OpenAI-compatible format)
api_key: "sk-your-key" # API key (can also use api_key_env to read from environment)
api_key_env: "RESEARCHCLAW_API_KEY" # Environment variable name for API key (fallback if api_key is empty)
primary_model: "gpt-5.4" # Main model for research, analysis, and writing stages
coding_model: "gpt-5.4" # Model for code generation (S11). Leave empty to use primary_model
image_model: "gemini-3-pro-image-preview" # Model for figure generation in paper writing (L5)
fallback_models: # Fallback model chain — used when primary model fails
- "gpt-4o"
- "gpt-4.1"
timeout_sec: 600 # LLM API request timeout in seconds
# === Security ===
security:
hitl_required_stages: [] # Stage numbers requiring human approval (e.g. [5, 9, 20])
allow_publish_without_approval: true # Allow paper export without human review
redact_sensitive_logs: false # Redact API keys and sensitive data in logs
# === Experiment ===
experiment:
mode: "sandbox" # Execution mode: "sandbox" (local Python) | "docker" | "simulated"
time_budget_sec: 2400 # Max time budget per experiment run in seconds
max_iterations: 3 # Number of iterative refinement cycles in S15 (Edit-Run-Eval loop)
metric_key: "primary_metric" # Name of the primary evaluation metric
metric_direction: "minimize" # Optimization direction: "minimize" | "maximize"
datasets_dir: "/path/to/datasets" # Absolute path to datasets directory
checkpoints_dir: "/path/to/checkpoints" # Absolute path to model weights directory
codebases_dir: "/path/to/codebases" # Absolute path to reference codebases directory
shared_results_dir: "/path/to/shared_results" # Directory for cross-project shared results
paper_length: "short" # Paper length: "short" (~4 pages) | "long" (~8 pages)
# Sandbox execution environment
sandbox:
python_path: "/path/to/python3" # Python interpreter path for running experiments
gpu_required: true # Whether experiments require GPU
gpus_per_project: 1 # Number of GPUs allocated per project
max_memory_mb: 16384 # Max memory limit for experiment processes (MB)
allowed_imports: # Whitelist of allowed Python packages in sandbox
- "numpy"
- "torch"
- "transformers"
- "diffusers"
# ... add packages as needed
sanity_check_max_iterations: 6 # Max fix attempts in S12 code testing. 0 = skip fixes, trigger intervention immediately
# Legacy code agent (disabled by default, use opencode instead)
code_agent:
enabled: false
# OpenHands Beast Mode — delegates complex code generation to OpenHands agent
opencode:
enabled: true # Master switch for Beast Mode
auto: true # Auto-trigger based on complexity score (vs. manual)
complexity_threshold: 0.2 # Complexity score threshold (0.0-1.0). Lower = more likely to use Beast Mode
model: "claude-opus-4-6" # LLM model used by OpenHands agent
timeout_sec: 2400 # Max time for Beast Mode code generation (seconds)
max_retries: 1 # Number of retries if Beast Mode fails to produce main.py
workspace_cleanup: false # Whether to delete temporary workspace after completion
# === Prompts ===
prompts:
custom_file: "" # Path to custom prompts YAML file (empty = use defaults)
<!-- ---
$ claude mcp add Claw-AI-Lab \
-- python -m otcore.mcp_server <graph>