MCPcopy Create free account
hub / github.com/KnockOutEZ/wigolo

github.com/KnockOutEZ/wigolo

Chat with this repo
repository ↗ · DeepWiki ↗ · release v0.2.1 ↗ · + Follow · compare 2 versions
4,712 symbols 16,749 edges 1,227 files ⚖ AGPL-3.0 406 documented · 9% updated 5d agov0.2.1 · 2026-07-19★ 3,48125 open issues

Browse by type

Functions 3,724 Types & classes 988
What it actually does AI analysis from the code graph
loading…
README

wigolo — the go-to web for your agent

Local-first web intelligence for AI agents — no keys, no cloud, no metered bill.

works with  Claude Code · Cursor · Codex · Gemini CLI · VS Code · Windsurf · Zed · Antigravity

and beyond  LangChain · CrewAI · LlamaIndex · Vercel AI SDK · n8n & self-hosted agents · any MCP client · plain REST

npm npm downloads GitHub stars CI node MCP license status

wigolo on Trendshift

Quickstart · Tools · Why wigolo · Benchmark · Docs · Examples · Feedback · FAQ


wigolo gives an AI agent one durable surface for everything web-related — search, fetch, crawl, extract, cache, find-similar, research, and autonomous gather loops. It runs wherever your agent runs: as an MCP server next to your coding agent, as a REST/MCP endpoint on the box where your self-hosted agents live, or embedded through an SDK inside your own app. The core tools need no API keys, nothing it touches leaves ~/.wigolo/, and there's no bill that grows with how much your agent thinks.

wigolo demo — Claude Code answering a live web question through wigolo, no API keys

Quickstart

Requires Node ≥ 20 and ~1.5 GB of free disk. macOS, Linux, and Windows.

One command wires the local engine into your agent. init is unattended by default — no prompts, safe in scripts and CI — and does the complete setup: it downloads the browser engine and on-device models, runs a health check, and prints a per-component summary, so any setup problem surfaces right here, not silently on your agent's first call:

npx wigolo init --agents=<your-agent>
  • <your-agent> — one or more of claude-code · cursor · codex · gemini-cli · vscode · windsurf · zed · antigravity (comma-separated). wigolo writes the MCP config and instructions for you.
  • Any other MCP client? Omit --agents and register npx -y wigolo yourself — the installation guide has the exact config block for every client, plus Docker, Homebrew, and single-file-binary channels.
  • Prefer prompts? --interactive is a plain-text flow; --wizard is the full terminal TUI.
  • Skip the downloads? --no-warmup defers everything to first use. A failed component download never fails setup — init reports what's not ready with the exact fix and still wires your agent.

That's the whole setup — search, fetch, crawl, extract, cache, and find-similar work with no API key. Check it's healthy anytime:

npx wigolo doctor

Not for you? npx wigolo config --uninstall --yes removes everything, cleanly. You can also paste the installation guide at any AI assistant and let it do the setup — it's written to be self-contained.

Recommended — a free key makes research & agent shine

Search, fetch, crawl, extract, cache, and find-similar are fully keyless. But research, agent, and search format=answer use an LLM to write the synthesized, cited answer — without one they hand back a raw brief and evidence for your agent to assemble, which is a much thinner experience. A free Gemini key is all it takes, and it's the single biggest quality upgrade you can make:

export WIGOLO_LLM_PROVIDER=gemini
export GEMINI_API_KEY=<free-key>      # grab one at aistudio.google.com/apikey — the free tier is plenty

Any provider works (anthropic · openai · groq), or stay fully local and keyless with WIGOLO_LLM_PROVIDER=ollama (or any OpenAI-compatible URL). Set it in your shell or your agent's MCP env block. Providers, models, and the keyless local-model ladder: configuration guide.

What your agent gets back

Not snippets — evidence. Every search result carries a verbatim excerpt pinned to its exact position in the source, a citation ID the agent can quote, and a score it can inspect (abridged real shape):

{
  "results": [{
    "title": "Logical replication - PostgreSQL docs",
    "url": "https://www.postgresql.org/docs/current/logical-replication.html",
    "excerpt": "Logical replication is a method of replicating data objects…",
    "citation_id": "src-1",
    "source_span": { "start": 1042, "end": 1305 },          // byte-exact provenance
    "evidence_score": { "final": 0.86, "semantic": 0.91, "lexical": 0.78, "engine_consensus": 3 }
  }],
  "citations": [{ "id": "src-1", "url": "…" }],
  "freshness_signal": { "published": "2026-05-12", "confidence": "high" }
}

Weak results get flagged as junk by wigolo's own scorer, failed engines are reported, stale cache is labeled — the agent always knows what it's standing on. Full response contracts per tool: tools reference.

Tools

Tool What it does
🔎 search Multi-engine web search (18 direct adapters) with rank fusion, ML reranking, and an explainable per-result score. Pass a query array for parallel breadth.
📄 fetch Load one URL through a tiered router that auto-escalates from plain HTTP to a headless browser engine on anti-bot challenges or SPA shells. Clean markdown + metadata + links.
🕸️ crawl Multi-page crawl — BFS, DFS, sitemap, or map-only. Per-domain rate limits, robots.txt respect, boilerplate dedup.
🧩 extract Structured data from a page: tables, metadata, JSON-LD, brand identity, named schemas (Article / Recipe / Product / …), or any custom JSON Schema.
💾 cache Query everything already seen — keyword or hybrid semantic. Plus stats, clear, and change detection.
🧲 find_similar Pages similar to a URL or a concept, via 3-way fusion of keyword + semantic + live web.
🧠 research Decompose a question → fan out sub-queries → fetch sources → synthesize a cited report (or a structured brief the host LLM writes from).
🤖 agent Autonomous gather loop: plan → search → fetch → extract → synthesize, with a step log, time budget, and optional output schema.
🔁 diff + ⏱️ watch See exactly what changed on a page since last visit; re-check on demand and deliver changes to a webhook.

Every tool also runs from the terminal (wigolo search "…" --json), from an interactive shell with NDJSON piping (wigolo shell), over REST, and through the SDKs — CLI reference.

What that actually lets you do

Each tool goes well past its one-liner. A sampler — every line links to the guide and, where there's one, a runnable example:

  • Search that fans out — pass a query array for parallel breadth, scope to include_domains, bound by time_range/recency, exact-phrase match, choose a depth tier, even image results. → guide · example
  • Fetch almost anything — JS-rendered SPAs, PDFs, a single heading section, authenticated pages (via a browser profile or remote browser), or drive the page with actions (click / type / scroll / screenshot). → guide
  • Crawl a whole site — sitemap, BFS, DFS, or map-only; robots.txt-respecting, per-domain rate-limited, boilerplate-deduped. → guide
  • Extract structure — tables, JSON-LD, metadata, brand assets, named schemas (Article / Recipe / Product / …), or your own JSON Schema. → guide
  • A memory that compounds — every page is cached; re-query by keyword or meaning, instantly and offline; detect what changed since last visit. → guide · example
  • Research & autonomous gather — decompose a question into a cited brief, or turn agent loose to plan → fetch → extract → synthesize against a JSON Schema and a time budget. → guide · example
  • Watch & diff — monitor a URL, get a change report, deliver it to a webhook. → guide · example
  • Drive it your way — one-shot CLI, an NDJSON shell for pipelines, REST, SDKs, or as skills your agent installs. → CLI & shell · example
  • Extend it — add a search engine or a site extractor as a plugin in ~100 lines. → plugins · example
  • Tune & inspectwigolo tune shows what it learned per domain (which fetch tier, challenge clearances, backoff); doctor / verify health-check every component. → CLI · troubleshooting

Why it's different

wigolo isn't the free stand-in you settle for until the budget clears — it's built to hold the same line as the paid services in this lane, and it brings receipts. What actually separates it:

  • Built for agents, not humans. One MCP call fans out many queries across many engines in parallel — something a serial host tool-loop can't replicate — with transparent per-result scoring and budget-aware output.
  • Honest output. Stale cache, failed fetches, degraded backends, and truncation are surfaced in the result, never disguised as empty-but-successful data. When a bot-protected page can't be read, you get a labeled blocked_by_challenge failure — never a challenge shell dressed up as content.
  • $0 per query, free to re-query. Default search talks to public engines through direct adapters; the reranker and embeddings run on-device. Every response is cached, so asking again is instant and costs nothing.
  • Private by default. Cache, embeddings, models, and config live under ~/.wigolo/. Nothing reaches a third party unless you explicitly opt into an LLM for synthesis.

wigolo is a focused web layer for your agents — not a hosted SaaS, a vector database other apps query, or a scale-scraping platform. Within that lane it goes toe-to-toe with the paid services on result quality — and the meter, the key, and the data-egress simply aren't there.

Here's what one real result looks like, dissected — including the failed engine and the weak result, because those are part of the answer too:

Anatomy of a wigolo result: explainable score decomposition, live engine telemetry, surfaced degradation, self-flagged junk — one real query, captured live

Benchmark

All four tools converged on the same core answer — and only one of them handed back verbatim, byte-pinned evidence while doing it.

One cold query, run live inside a single Claude Fable 5 session and fanned out to four web tools on equal footing — built-in WebSearch, wigolo, Tavily, and Exa — then reported by the agent itself under one rule: judge on the evidence alone, no favoritism. All four converged on the same answer and the same top source — parity demonstrated, not asserted. wigolo alone returned verbatim excerpts pinned to byte-offset source spans, an explainable score decomposition, and live per-engine telemetry — and when two of its results were weak, its own scorer flagged them as junk on-screen. The cloud tools earn their line too: Exa rendered the official docs' comparison matrix in full. One honest query, not a leaderboard — run your own and you'll see the same shape.

wigolo vs built-in WebSearch, Tavily, and Exa on one real query, driven by Claude Fable 5

Same fight, different physics

wigolo Firecrawl Exa Tavily
Multi-engine web search
Fetch & structured extraction
Whole-site crawl & map
Verbatim excerpts pinned to byte-offset source spans
Explainable per-result score decomposition
Persistent local memory — re-query instantly, offline
Query data stays on your machine
API key / account none required required required
Cost per query $0 metered metered metered

Feature standing as of July 2026 — check each vendor's docs for current state.

That last row is the one that compounds — agents don't ask once, they ask in bursts:

<img alt="The meter: a metered cloud API's cost climbs with every query while wigolo stays flat at zero

Extension points exported contracts — how you extend this code

browse all types & interfaces →

Core symbols most depended-on inside this repo

browse all functions →

Shape

Function 3,098
Interface 707
Method 626
Class 281

Languages

TypeScript91%
Python9%
Ruby1%

Modules by API surface

src/types.ts81 symbols
src/fetch/router.ts55 symbols
src/cache/store.ts49 symbols
src/cli/tui/state/propagation.ts48 symbols
src/extraction/brand.ts43 symbols
src/cli/doctor.ts41 symbols
src/fetch/tls-tier.ts38 symbols
src/fetch/browser-pool.ts34 symbols
src/daemon/rest/firecrawl-compat.ts31 symbols
src/search/core/engine-base.ts29 symbols
src/cli/skills.ts29 symbols
sdks/typescript/src/local.ts29 symbols

Dependencies from manifests, versioned

@anthropic-ai/sdk0.91.1 · 1×
@google/genai1.51.0 · 1×
@iarna/toml2.2.5 · 1×
@inkjs/ui2.0.0 · 1×
@inquirer/prompts7.10.1 · 1×
@modelcontextprotocol/sdk1.29.0 · 1×
@mozilla/readability0.6.0 · 1×
@seriousme/openapi-schema-validator2.9.0 · 1×
@types/cli-progress3.11.6 · 1×
@types/node25.6.0 · 1×

For agents

$ claude mcp add wigolo \
  -- python -m otcore.mcp_server <graph>

⬇ download graph artifact

Ask about this repo answers extend the page