Resile: Ergonomic Execution Resilience for Go

Resile is a resilience and retry library for Go, inspired by Python's stamina. It provides a type-safe, ergonomic, and highly observable way to handle transient failures in distributed systems.

See Resile in Action

Subscribe to our YouTube channel for deep dives and tutorials.

The "Aha!" Snippet

Resile allows you to compose complex resilience strategies with a single, type-safe call. No interface{} casting, no reflection—just clean Go.

user, err := resile.Do(ctx, func(ctx context.Context) (*User, error) {
    return client.GetUser(ctx, id)
},
    resile.WithBulkhead(10),            // Max 10 concurrent requests
    resile.WithCircuitBreaker(cb),      // Stop if failure rate > 50%
    resile.WithRetry(3),                // 3 attempts with AWS Full Jitter
    resile.WithTimeout(1*time.Second),   // Each attempt max 1s
    resile.WithFallback(func(ctx context.Context, err error) (*User, error) {
        return cache.Get(id), nil       // Return stale data on failure
    }),
)

Table of Contents

• See Resile in Action
• Installation
• Why Resile?
• Articles & Tutorials
• Examples
• API Reference
• Resilience Cookbook
• Simple Retries
• Value-Yielding Retries (Generics)
• Request Hedging (Speculative Retries)
• Stateful Retries & Endpoint Rotation
• Handling Rate Limits (Retry-After)
• Aborting Retries (Pushback Signal)
• Fallback Strategies
• Bulkhead Pattern
• Priority-Aware Bulkhead
• Rate Limiting Pattern
• Layered Defense with Circuit Breaker
• Macro-Level Protection (Adaptive Retries)
• Adaptive Concurrency (TCP-Vegas)
• Structured Logging & Telemetry
• Panic Recovery ("Let It Crash")
• Fast Unit Testing
• Reusable Clients & Dependency Injection
• Marking Errors as Fatal
• Custom Error Filtering
• Policy Composition & Chaining
• Native Multi-Error Aggregation
• Native Chaos Engineering (Fault & Latency Injection)
• Distributed Deadline Propagation
• Reliable File Downloads (HTTP Resumption)
• SQL Resilience
• Redis Resilience
• Built on Hyperscaler Research
• Configuration Reference
• Sentinel Errors
• Architecture & Design
• Community
• Acknowledgements
• License

Installation

go get github.com/cinar/resile

Requires Go 1.24+ (uses generics and errors.Join).

Why Resile?

In distributed systems, transient failures are a mathematical certainty. Resile simplifies the "Correct Way" to retry: - AWS Full Jitter: Uses the industry-standard algorithm to prevent "thundering herd" synchronization. - Adaptive Retries: Built-in token bucket rate limiting to prevent "retry storms" across a cluster. - Generic-First: No interface{} or reflection. Full compile-time type safety. - Context-Aware: Strictly respects context.Context cancellation and deadlines. - Zero-Dependency Core: The core resile, circuit, and chaos packages depend only on the Go standard library. Telemetry adapters (telemetry/) are opt-in. - Opinionated Defaults: Sensible defaults (5 attempts, exponential backoff). - Chaos-Ready: Built-in support for fault and latency injection to test your resilience policies.

Articles & Tutorials

Want to learn more about the philosophy behind Resile and advanced resilience patterns in Go? Check out these deep dives:

• Stop Writing Manual Retry Loops in Go: Why Your Current Logic is Probably Dangerous
• Python's Stamina for Go: Bringing Ergonomic Resilience to Gophers
• Beating Tail Latency: A Guide to Request Hedging in Go Microservices
• Preventing Microservice Meltdowns: Adaptive Retries and Circuit Breakers in Go
• Self-Healing State Machines: Resilient State Transitions in Go
• Resilience Beyond Counters: Sliding Window Circuit Breakers in Go
• Stop the Domino Effect: Bulkhead Isolation in Go
• Reliable Redis: Combining Retries and Bulkheads for Rock-Solid Caching
• Prioritize Your Traffic: Priority-Aware Bulkheads in Go
• Respecting Boundaries: Precise Rate Limiting in Go
• Beyond Static Limits: Adaptive Concurrency with TCP-Vegas in Go
• Debugging the Timeline: Native Multi-Error Aggregation in Go
• Native Chaos Engineering: Testing Resilience with Fault & Latency Injection
• Stopping the Zombie Requests: Distributed Deadline Propagation in Go
• Reliable File Downloads with HTTP Range Resumption
• Building Bulletproof Database Clients in Go: SQL Resilience with Resile

Also, check out our Dev.to space for more articles and discussions.

Examples

The examples/ directory contains standalone programs showing how to use Resile in various scenarios:

• Basic Retry: Simple Do and DoErr calls.
• Request Hedging: Reducing tail latency with speculative retries.
• HTTP with Rate Limits: Respecting Retry-After headers and using slog.
• Fallback Strategies: Returning stale data when all attempts fail.
• Stateful Rotation: Rotating API endpoints using RetryState.
• Circuit Breaker: Layering defensive strategies.
• Backpressure: Full backpressure demo combining circuit breaker and adaptive retries.
• Adaptive Retries: Preventing retry storms with a token bucket.
• Adaptive Concurrency: Dynamic concurrency limits based on latency (TCP-Vegas).
• Bulkhead: Standalone bulkhead isolation pattern.
• Priority Bulkhead: Load shedding based on traffic priority.
• Rate Limiter: Token bucket rate limiting.
• Pushback Signal: Aborting retries immediately using CancelAllRetries.
• Panic Recovery: Implementing Erlang's "Let It Crash" philosophy.
• State Machine: Building resilient state machines inspired by Erlang's gen_statem.
• Chaos Injection: Simulating faults and latency to test your policies.
• HTTP Resumption: Resuming large file downloads using HTTP Range.
• SQL Resilience: Using Resile with standard database/sql.
• Redis Resilience: Adding resilience to Redis operations with shared bulkheads.
• Deadline Propagation: Distributed deadline propagation with HTTP headers.

API Reference

Resilience Cookbook

1. Simple Retries

The Problem: A database connection or network request might fail intermittently due to transient blips.

The Recipe: Retry a simple operation that only returns an error. If all retries fail, Resile returns an aggregated error containing the failures from every attempt.

err := resile.DoErr(ctx, func(ctx context.Context) error {
    return db.PingContext(ctx)
})

2. Value-Yielding Retries (Generics)

The Problem: Fetching data from a microservice needs to be resilient and type-safe without boilerplate casting.

The Recipe: Fetch data with full type safety. The return type is inferred from your closure.

// val is automatically inferred as *User
user, err := resile.Do(ctx, func(ctx context.Context) (*User, error) {
    return apiClient.GetUser(ctx, userID)
}, resile.WithMaxAttempts(3))

3. Request Hedging (Speculative Retries)

The Problem: Long-tail latency (the 99th percentile) slows down your entire system even when most requests are fast.

The Recipe: Speculative retries reduce tail latency by starting a second request if the first one doesn't finish within a configured HedgingDelay. The first successful result is used, and the other is cancelled.

// For value-yielding operations
data, err := resile.DoHedged(ctx, action, 
    resile.WithMaxAttempts(3),
    resile.WithHedgingDelay(100 * time.Millisecond),
)

// For error-only operations
err := resile.DoErrHedged(ctx, action,
    resile.WithMaxAttempts(2),
    resile.WithHedgingDelay(50 * time.Millisecond),
)

Stateful hedging variants are also available for endpoint rotation with speculative retries:

data, err := resile.DoStateHedged(ctx, func(ctx context.Context, state resile.RetryState) (string, error) {
    url := endpoints[state.Attempt % uint(len(endpoints))]
    return client.Get(ctx, url)
}, resile.WithHedgingDelay(100 * time.Millisecond))

Read more: Beating Tail Latency: A Guide to Request Hedging in Go Microservices

4. Stateful Retries & Endpoint Rotation

The Problem: Retrying against the same failing endpoint is futile; you need to cycle through a list of healthy hosts.

The Recipe: Use DoState (or DoErrState) to access the RetryState, allowing you to rotate endpoints or fallback logic based on the failure history.

endpoints := []string{"api-v1.example.com", "api-v2.example.com"}

data, err := resile.DoState(ctx, func(ctx context.Context, state resile.RetryState) (string, error) {
    // Rotate endpoint based on attempt number
    url := endpoints[state.Attempt % uint(len(endpoints))]
    return client.Get(ctx, url)
})

Read more: Self-Healing State Machines: Resilient State Transitions in Go

5. Handling Rate Limits (Retry-After)

The Problem: Downstream services may