Plug in all your MCP servers, APIs, and LLM providers. Route everything through a unified endpoint.

Aggregate, govern, and control your AI stack.

Features

• MCP Server Aggregation: Connect multiple MCP servers (STDIO, SSE, HTTP) through a single endpoint
• LLM Provider Routing: Unified interface for OpenAI, Anthropic, Google, and AWS Bedrock LLM providers with full tool calling support
• Context-Aware Tool Search: Intelligent fuzzy search across all connected tools using natural language queries
• Protocol Support: Supports STDIO (subprocess), SSE (Server-Sent Events), and streamable HTTP MCP servers
• Flexible Configuration: TOML-based configuration with environment variable substitution
• Security: Built-in CORS, CSRF protection, OAuth2, and TLS support
• Rate Limiting: Multi-level rate limiting with in-memory or Redis backends
• Docker Ready: Available as a container image with minimal configuration needed

Installation

Quick Install (Linux/Windows (WSL)/macOS)

curl -fsSL https://nexusrouter.com/install | bash

Docker

Pull the latest image:

docker pull ghcr.io/grafbase/nexus:latest

Or use the stable version:

docker pull ghcr.io/grafbase/nexus:stable

Or use a specific version:

docker pull ghcr.io/grafbase/nexus:X.Y.Z

Build from Source

git clone https://github.com/grafbase/nexus
cd nexus
cargo build --release

Running Nexus

Using the Binary

nexus

Using Docker

docker run -p 8000:8000 -v /path/to/config:/etc/nexus.toml ghcr.io/grafbase/nexus:latest

Docker Compose Example

services:
  nexus:
    image: ghcr.io/grafbase/nexus:latest
    ports:
      - "8000:8000"
    volumes:
      - ./nexus.toml:/etc/nexus.toml
    environment:
      - GITHUB_TOKEN=${GITHUB_TOKEN}
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
      interval: 30s
      timeout: 10s
      retries: 3

Configuration

Create a nexus.toml file to configure Nexus:

# LLM Provider configuration
[llm.providers.openai]
type = "openai"
api_key = "{{ env.OPENAI_API_KEY }}"
forward_token = true

# Model configuration (at least one model required per provider)
[llm.providers.openai.models.gpt-4]
[llm.providers.openai.models.gpt-3-5-turbo]

[llm.providers.anthropic]
type = "anthropic"
api_key = "{{ env.ANTHROPIC_API_KEY }}"

[llm.providers.anthropic.models.claude-3-5-sonnet-20241022]

# MCP Server configuration
[mcp.servers.github]
url = "https://api.githubcopilot.com/mcp/"
auth.token = "{{ env.GITHUB_TOKEN }}"

[mcp.servers.filesystem]
cmd = ["npx", "-y", "@modelcontextprotocol/server-filesystem", "/home/YOUR_USERNAME/Desktop"]

[mcp.servers.python_server]
cmd = ["python", "-m", "mcp_server"]
env = { PYTHONPATH = "/opt/mcp" }
cwd = "/workspace"

Configuration Options

Server Configuration

• server.listen_address: The address and port Nexus will listen on (default: 127.0.0.1:8000)
• server.health.enabled: Enable health endpoint (default: true)
• server.health.path: Health check endpoint path (default: /health)

LLM Configuration

• llm.enabled: Enable LLM functionality (default: true)
• llm.protocols.openai.enabled: Enable OpenAI protocol endpoint (default: true)
• llm.protocols.openai.path: OpenAI endpoint path (default: /llm/openai)
• llm.protocols.anthropic.enabled: Enable Anthropic protocol endpoint (default: false)
• llm.protocols.anthropic.path: Anthropic endpoint path (default: /llm/anthropic)

For detailed LLM provider configuration, see the LLM Provider Configuration section below.

MCP Configuration

• mcp.enabled: Enable MCP functionality (default: true)
• mcp.path: MCP endpoint path (default: /mcp)
• mcp.enable_structured_content: Control MCP search tool response format (default: true)
• When true: Uses modern structuredContent field for better performance and type safety
• When false: Uses legacy content field with Content::json objects for compatibility with older MCP clients

MCP Server Types

1. STDIO Servers: Launch local processes that communicate via standard input/output ```toml [mcp.servers.my_tool] cmd = ["path/to/executable", "--arg1", "--arg2"]

# Optional: Set environment variables env = { DEBUG = "1", API_KEY = "{{ env.MY_API_KEY }}" }

# Optional: Set working directory cwd = "/path/to/working/directory"

# Optional: Configure stderr handling (default: "null") stderr = "inherit" # Show in console # or stderr = { file = "/var/log/mcp/server.log" } # Log to file ```

Note: STDIO servers must output valid JSON-RPC messages on stdout. The cmd array must have at least one element (the executable).

1. SSE Servers: Connect to Server-Sent Events endpoints toml [mcp.servers.my_sse_server] protocol = "sse" url = "http://example.com/sse" message_url = "http://example.com/messages" # Optional

2. HTTP Servers: Connect to streamable HTTP endpoints toml [mcp.servers.my_http_server] protocol = "streamable-http" url = "https://api.example.com/mcp"

For remote MCP servers, if you omit the protocol Nexus will first try streamable HTTP and then SSE.

Authentication

Add service token authentication to any server:

[mcp.servers.my_server.auth]
token = "your-token-here"
# Or use environment variables
token = "{{ env.MY_API_TOKEN }}"

If you enable OAuth2 authentication to your server, and your downstream servers all use the same authentication server, you can configure Nexus to forward the request access token to the downstream server.

[mcp.servers.my_server.auth]
type = "forward"

Header Insertion for MCP Servers

Nexus supports inserting static headers when making requests to MCP servers. Headers can be configured globally (for all MCP servers) or per-server.

Note: MCP currently only supports header insertion with static values. Headers from incoming requests are not forwarded.

Global MCP Headers

Configure headers that apply to all MCP servers:

# Global headers for all MCP servers
[[mcp.headers]]
rule = "insert"
name = "X-Application"
value = "nexus-router"

[[mcp.headers]]
rule = "insert"
name = "X-API-Version"
value = "v1"

Server-Specific Headers

Configure headers for individual HTTP-based MCP servers:

[mcp.servers.my_server]
url = "https://api.example.com/mcp"

# Insert headers for this specific server
[[mcp.servers.my_server.headers]]
rule = "insert"
name = "X-API-Key"
value = "{{ env.MY_API_KEY }}"  # Environment variable substitution

[[mcp.servers.my_server.headers]]
rule = "insert"
name = "X-Service-Name"
value = "my-service"

MCP Header Features

• Static Values Only: Headers are set at client initialization time with static values
• Environment Variables: Use {{ env.VAR_NAME }} syntax for environment variable substitution
• HTTP Servers Only: Headers only apply to HTTP-based MCP servers (not STDIO servers)
• Insert Rule: Currently only the insert rule is supported for MCP

MCP Access Control

Restrict access to MCP servers and tools based on user groups:

# Server-level access control
[mcp.servers.premium_tools]
cmd = ["premium-server"]
allow = ["premium", "enterprise"]  # Only these groups can access
deny = ["suspended"]                # Block specific groups

# Tool-level override (more specific than server-level)
[mcp.servers.premium_tools.tools.expensive_feature]
allow = ["enterprise"]  # Only enterprise can use this tool

[mcp.servers.premium_tools.tools.deprecated_tool]
allow = []  # Empty allow list blocks all access (no client ID needed)

Access control rules: - If allow is set, only listed groups can access (requires client identification) - If deny is set, listed groups are blocked (requires client identification) - Empty allow = [] blocks all access without requiring client identification - Tool-level rules override server-level rules - Deny takes priority over allow

OAuth2 Authentication

Configure OAuth2 authentication to protect your Nexus endpoints:

[server.oauth]
url = "https://your-oauth-provider.com/.well-known/jwks.json"
poll_interval = "5m"
expected_issuer = "https://your-oauth-provider.com"
expected_audience = "your-service-audience"

[server.oauth.protected_resource]
resource = "https://your-nexus-instance.com"
authorization_servers = ["https://your-oauth-provider.com"]

OAuth2 configuration options: - url: JWKs endpoint URL for token validation - poll_interval: How often to refresh JWKs (optional, default: no polling) - expected_issuer: Expected iss claim in JWT tokens (optional) - expected_audience: Expected aud claim in JWT tokens (optional) - protected_resource.resource: URL of this protected resource - protected_resource.authorization_servers: List of authorization server URLs

When OAuth2 is enabled, all endpoints except /health and /.well-known/oauth-protected-resource require valid JWT tokens in the Authorization: Bearer <token> header.

Rate Limiting

Nexus supports rate limiting to prevent abuse and ensure fair resource usage:

# Global rate limiting configuration
[server.rate_limits]
enabled = true

# Storage backend configuration
[server.rate_limits.storage]
type = "memory"  # or "redis" for distributed rate limiting
# For Redis backend:
# url = "redis://localhost:6379"
# key_prefix = "nexus:rate_limit:"

# Global rate limit (applies to all requests)
[server.rate_limits.global]
limit = 1000
interval = "60s"

# Per-IP rate limit
[server.rate_limits.per_ip]
limit = 100
interval = "60s"

# Per-MCP server rate limits
[mcp.servers.my_server.rate_limits]
limit = 50
interval = "60s"

# Tool-specific rate limits (override server defaults)
[mcp.servers.my_server.rate_limits.tools]
expensive_tool = { limit = 10, interval = "60s" }
cheap_tool = { limit = 100, interval = "60s" }

Rate Limiting Features: - Multiple levels: Global, per-IP, per-server, and per-tool limits - Storage backends: In-memory (single instance) or Redis (distributed) - Flexible intervals: Configure time windows for each limit - Tool-specific overrides: Set different limits for expensive operations

Redis Backend Configuration:

[server.rate_limits.storage]
type = "redis"
url = "redis://localhost:6379"
key_prefix = "nexus:rate_limit:"
response_timeout = "1s"
connection_timeout = "5s"

# Connection pool settings
[server.rate_limits.storage.pool]
max_size = 16
min_idle = 0
timeout_create = "5s"
timeout_wait = "5s"
timeout_recycle = "300s"

# TLS configuration for Redis (optional)
[server.rate_limits.storage.tls]
enabled = true
ca_cert_path = "/path/to/ca.crt"
client_cert_path = "/path/to/client.crt"  # For mutual TLS
client_key_path = "/path/to/client.key"
# insecure = true  # WARNING: Only for development/testing, skips certificate validation

Note: When configuring tool-specific rate limits, Nexus will warn if you reference tools that don't exist.

LLM Token Rate Limiting

Nexus provides token-based rate limiting for LLM providers to help control costs and prevent abuse. Unlike request-based rate limits, token rate limits count an estimate of actual tokens consumed.

Prerequisites

IMPORTANT: LLM rate limiting requires client identification to be enabled:

[server.client_identification]
enabled = true

# Choose identification methods (at least one required)
client_id.jwt_claim = "sub"                    # Extract ID from JWT 'sub' claim
# or
client_id.http_header = "X-Client-ID"          # Extract ID from HTTP header

# Optional: Limit groups per user (at most one allowed)
group_id.jwt_claim = "groups"                  # JWT claim containing user's group
# or
group_id.http_header = "X-Group-ID"            # Extract ID from HTTP header

# You must provide a list of allowed groups
[server.client_identification.validation]
group_values = ["free", "pro", "max"]

Without client identification, rate limits cannot be enforced and requests will fail with a configuration error.

Configuration Hierarchy

Token rate limits can be configured at four levels, from most to least specific:

1. Model per user + group: Specific model for specific each user in a group
2. Model per user: Specific model for each user
3. Provider per user + group: All models from provider for each user in a group
4. Provider per user: All models from provider for each user

The most specific applicable limit is always used.

Basic Configuration

# Provider-level default rate limit (applies to all models)
[llm.providers.openai.rate_limits.per_user]
input_token_limit = 100000        # 100K input tokens
interval = "1m"                   # Per minute

# Model-specific rate limit (overrides provider default)
[llm.providers.openai.models.gpt-4.rate_limits.per_user]
input_token_limit = 50000         # More restrictive for expensive model
interval = "30s"

Group-Based Rate Limits

Configure different limits for user groups (requires group_id and group_values in client identification):

```toml

Provider-level group limits

[llm.providers.openai.rate_limits.per_user.groups] free = { input_token_limit = 10000, interval = "60s" } pro = { input_token_limit = 100000, interval = "60s" } enterprise = {