MCPcopy Index your code
hub / github.com/durable-streams/durable-streams

github.com/durable-streams/durable-streams @caddy-v0.3.0

Chat with this repo
repository ↗ · DeepWiki ↗ · release caddy-v0.3.0 ↗ · + Follow
3,589 symbols 11,274 edges 389 files 1,653 documented · 46% 1 cross-repo links updated 4d ago@durable-streams/client-conformance-tests@0.2.11 · 2026-06-03★ 1,61350 open issues
What it actually does AI analysis from the code graph — generated when you open this
loading…
README

License - MIT Status - Beta Chat - Discord

Memento polaroid icon

Durable Streams

Durable Streams are the data primitive for the agent loop.

Persistent, addressable, real‑time streams for building resilient agent sessions and collaborative multi-user, multi-agent systems.

The data primitive for the agent loop

Durable Streams are a flexible data primitive for resilient, collaborative AI apps and agentic systems.

The Durable Streams protocol provides a simple, production-proven protocol for creating and consuming ordered, replayable data streams over HTTP, with support for catch-up reads and live tailing.

Use it to stream data reliably to web browsers, mobile apps, and native clients with low-latency, high-scalability, offset-based resumability and exactly-once message delivery over public Internet.

[!TIP] Read the Announcing Durable Streams and Durable Sessions - the key pattern for collaborative AI posts on the Electric blog.

Why Durable Streams?

Modern applications frequently need ordered, durable sequences of data that can be replayed from arbitrary points and tailed in real time. Common patterns include:

  • AI conversation streaming - Stream LLM token responses with resume capability across reconnections
  • Agentic apps - Stream tool outputs and progress events with replay and clean reconnect semantics
  • Database synchronization - Stream database changes to web, mobile, and native clients
  • Collaborative editing - Sync CRDTs and operational transforms across devices
  • Real-time updates - Push application state to clients with guaranteed delivery
  • Event sourcing - Build event-sourced architectures with client-side replay
  • Workflow execution - Stream workflow state changes with full history

While durable streams exist throughout backend infrastructure (database WALs, Kafka topics, event stores), they aren't available as a first-class primitive for client applications. There's no simple, HTTP-based durable stream that sits alongside databases and object storage as a standard cloud primitive.

WebSocket and SSE connections are easy to start, but they're fragile in practice: tabs get suspended, networks flap, devices switch, pages refresh. When that happens, you either lose in-flight data or build a bespoke backend storage and client resume protocol on top.

AI products make this painfully visible. Token streaming is the UI for chat and copilots, and agentic apps stream progress events, tool outputs, and partial results over long-running sessions. When the stream fails, the product fails—even if the model did the right thing.

Durable Streams addresses this gap. It's a minimal HTTP-based protocol for durable, offset-based streaming designed for client applications across all platforms: web browsers, mobile apps, native clients, IoT devices, and edge workers. Based on 1.5 years of production use at Electric for real-time Postgres sync, reliably delivering millions of state changes every day.

What you get:

  • Refresh-safe - Users refresh the page, switch tabs, or background the app—they pick up exactly where they left off
  • Share links - A stream is a URL. Multiple viewers can watch the same stream together in real-time
  • Never re-run - Don't repeat expensive work because a client disconnected mid-stream
  • Multi-device - Start on your phone, continue on your laptop, watch from a shared link—all in sync
  • Multi-tab - Works seamlessly across browser tabs without duplicating connections or missing data
  • Massive fan-out - CDN-friendly design means one origin can serve millions of concurrent viewers

The protocol is:

  • 🌐 Universal - Works anywhere HTTP works: web browsers, mobile apps, native clients, IoT devices, edge workers
  • 📦 Simple - Built on standard HTTP with no custom protocols
  • 🔄 Resumable - Offset-based reads let you resume from any point
  • Real-time - Long-poll and SSE modes for live tailing with catch-up from any offset
  • 💰 Economical - HTTP-native design leverages CDN infrastructure for efficient scaling
  • 🎯 Flexible - Content-type agnostic byte streams
  • 🔌 Composable - Build higher-level abstractions on top (like Electric's real-time Postgres sync engine)

Quickstart

  • Quickstart -- start the server and create your first stream

Usage

  • CLI -- create, read, append-to and tail streams
  • Clients -- TypeScript, Python and other languages
  • JSON mode -- stream structured data using JSON messages
  • Durable Proxy -- durable proxy for AI token streams
  • Durable State -- sync structured state over durable streams
  • StreamDB -- type-safe, reactive database in a stream
  • StreamFS -- filesystem with real-time sync in a stream

Integrations

Reference

  • Servers -- official server implementations
  • Protocol -- full protocol specification

Packages

Client Libraries

Package Language Description
@durable-streams/client TypeScript Reference client with full read/write support
client-py Python Python client library
client-go Go Go client library
client-elixir Elixir Elixir client library
client-dotnet C#/.NET .NET client library
client-swift Swift Swift client library
client-php PHP PHP client library
client-java Java Java client library
client-rust Rust Rust client library
client-rb Ruby Ruby client library

Servers & Tools

Package Description
@durable-streams/server Node.js reference server (development/testing)
caddy-plugin Production Caddy server plugin
@durable-streams/cli Command-line tool
@durable-streams/state State Protocol (insert/update/delete over streams)

Testing & Benchmarks

Package Description
@durable-streams/server-conformance-tests Server protocol compliance tests
@durable-streams/client-conformance-tests Client protocol compliance tests
@durable-streams/benchmarks Performance benchmarking suite

Community Implementations

Contributing

We welcome contributions! This project follows the Contributor Covenant code of conduct.

# Clone and install
git clone https://github.com/durable-streams/durable-streams.git
cd durable-streams
pnpm install

# Build all packages
pnpm build

# Run all conformance tests
pnpm test:run

# Lint and format
pnpm lint:fix
pnpm format

We use changesets for version management. Run pnpm changeset to add a changeset before submitting a PR.

License

MIT -- see LICENSE

Extension points exported contracts — how you extend this code

Store (Interface)
Store is the interface for durable stream storage [2 implementers]
packages/caddy-plugin/store/store.go
Event (Interface)
Event represents a parsed SSE event. [2 implementers]
packages/client-go/internal/sse/parser.go
PendingEntry (Interface)
* Internal type for pending batch entries.
packages/client/src/idempotent-producer.ts
CollectionSyncHandler (Interface)
* Handler for collection sync events
packages/state/src/stream-db.ts
AllowlistPattern (Interface)
* Parsed allowlist pattern with regex matchers for each URL component.
packages/proxy/src/server/allowlist.ts
BaseExpectation (Interface)
* Base expectation fields.
packages/client-conformance-tests/src/test-cases.ts
ConnectionContext (Interface)
* Connection context bundles all state for a single connection attempt. * Each connect() creates a new context with a u
packages/y-durable-streams/src/yjs-provider.ts
HttpClientInterface (Interface)
* Interface for HTTP clients used by Durable Streams. * * This allows using either the built-in cURL client or a PSR-1
packages/client-php/src/Internal/HttpClientInterface.php

Core symbols most depended-on inside this repo

getBaseUrl
called by 952
packages/server-conformance-tests/src/index.ts
get
called by 432
packages/client-php/src/Internal/HttpClient.php
get
called by 328
packages/server/src/store.ts
stringify
called by 318
packages/client-java/conformance-adapter/src/main/java/com/durablestreams/Json.java
toString
called by 204
packages/client-java/src/main/java/com/durablestreams/model/Chunk.java
text
called by 136
packages/client/src/response.ts
put
called by 131
packages/client-php/src/Internal/HttpClient.php
append
called by 126
packages/server/src/store.ts

Shape

Method 1,834
Function 1,010
Class 362
Interface 273
Struct 85
Enum 12
FuncType 8
TypeAlias 5

Languages

TypeScript37%
Python19%
Go15%
Java9%
Ruby7%
PHP4%

Modules by API surface

packages/client-py/src/durable_streams/_response.py92 symbols
packages/client-py/tests/test_stream_api.py59 symbols
packages/client-conformance-tests/src/protocol.ts58 symbols
packages/server/src/file-store.ts56 symbols
packages/client-py/tests/test_durable_stream.py55 symbols
packages/client/src/response.ts53 symbols
packages/client-java/src/main/java/com/durablestreams/DurableStream.java51 symbols
packages/client-conformance-tests/src/test-cases.ts50 symbols
packages/server/src/subscription-manager.ts49 symbols
packages/server/src/store.ts45 symbols
packages/client-go/cmd/conformance-adapter/main.go45 symbols
packages/client-py/tests/test_async_stream.py43 symbols

Used by 1 indexed graphs manifest dependencies, hub-wide

For agents

$ claude mcp add durable-streams \
  -- python -m otcore.mcp_server <graph>

⬇ download graph artifact

Ask about this repo answers extend the page