HubProxy

Overview

HubProxy is a proxy for GitHub webhooks, built for people building with GitHub at scale. It fixes a lot of stuff, and makes life easier.

It has a lot of features (most optional) and it's extremely configurable.

Key Features

• Webhook Verification: Cryptographically verifies GitHub webhook signatures to ensure authenticity
• Event Persistence: Stores webhook events in a database (SQLite/PostgreSQL/MySQL) for audit and replay
• Event Replay:
• Replay individual events by ID for testing or recovery
• Replay events within a specific time range with filtering options
• After a replay, you'll have two rows in the events table:
  1. The original event (unchanged)
  2. A new event with:
  3. Same payload as original
  4. Forwarding timestamp will be null
  5. ID format: original-id-replay-uuid
  6. Original event ID stored in replayed_from
• Range replay has a default limit of 100 events (override with limit)
• Filter range replays by type, repository, and sender
• Response includes replayed_count and list of replayed events
• Filter events by forwarding status using GET /api/events?forwarded=[true|false]
• Event Filtering:
• Filter events by type, repository, sender, and time range
• Query historical events through a RESTful API
• Get event statistics and delivery metrics
• REST API:
• List and search webhook events with pagination
• View event type statistics over time
• Replay single events or event ranges
• Filter and query capabilities for all operations
• GraphQL API:
• Alternative to REST API with the same functionality
• Flexible querying with precise field selection
• Interactive GraphiQL and Playground interfaces for testing
• Support for complex queries and mutations in a single request
• Monitoring:
• Provides metrics and logging for webhook delivery status and performance
• Track event patterns and volume through API statistics

Why HubProxy?

1. Reliability:
2. Never miss a webhook due to temporary service outages or bad deploys of your application
3. Replay events after recovering from downtime

4. Queue and retry failed deliveries automatically

5. Security:

6. Verify webhook authenticity using GitHub's HMAC signatures
7. Centralized secret management
8. Single point of security auditing

9. Automatically verify GitHub IP origins (often missed in webhooks implementations)

10. Observability:

11. Track webhook delivery status and latency
12. Debug integration issues with detailed logging

13. Monitor webhook patterns and volume

14. Development:

15. Test new integrations against real historical events
16. Debug webhook handlers without reconfiguring GitHub
17. Simulate webhook delivery for development

Architecture

HubProxy consists of three main components:

1. Webhook Handler: Receives, validates, and forwards GitHub webhooks
2. Storage Layer: Persists webhook events and delivery status
3. API Server: Provides REST endpoints for querying and replaying events

The system is designed to be horizontally scalable and can handle high webhook volumes while maintaining strict delivery guarantees.

Development and Testing

Prerequisites

• Go 1.24 or later (current codebase uses Go 1.24.2)
• SQLite (default), PostgreSQL 14+, or MySQL 8+ for event storage

Database

SQLite is used by default for development, but PostgreSQL or MySQL are recommended for production:

# SQLite (default for development)
hubproxy --db sqlite:.cache/hubproxy.db

# PostgreSQL
hubproxy --db "postgres://user:password@localhost:5432/hubproxy?sslmode=disable"

# MySQL
hubproxy --db "mysql://user:password@tcp(localhost:3306)/hubproxy"

Schema

Here's a simplified version (actual types may vary by database):

CREATE TABLE events (
    id          VARCHAR(255) PRIMARY KEY,    -- GitHub delivery ID
    type        VARCHAR(50) NOT NULL,       -- GitHub event type
    payload     TEXT NOT NULL,              -- Event payload as JSON
    headers     TEXT,                       -- HTTP headers as JSON
    created_at  TIMESTAMP NOT NULL,         -- When the event was received
    forwarded_at TIMESTAMP,                 -- When the event was forwarded
    error       TEXT,                       -- Error message if delivery failed
    repository  VARCHAR(255),               -- Repository full name
    sender      VARCHAR(255),               -- GitHub username
    replayed_from VARCHAR(255),             -- Original event ID if this is a replay
    original_time TIMESTAMP                 -- Original event time if this is a replay
);

-- Indexes for efficient querying
CREATE INDEX idx_created_at ON events (created_at);
CREATE INDEX idx_forwarded_at ON events (forwarded_at);
CREATE INDEX idx_type ON events (type);
CREATE INDEX idx_repository ON events (repository);
CREATE INDEX idx_sender ON events (sender);
CREATE INDEX idx_replayed_from ON events (replayed_from);

Query Options

The storage interface supports filtering events by: - Event type(s) - Repository name - Time range (since/until) - Forwarding status - Sender

Example queries:

// List all events
events, err := storage.ListEvents(QueryOptions{
    Types:      []string{"push", "pull_request"},
    Repository: "owner/repo",
    Since:      time.Now().Add(-24 * time.Hour),
    Forwarded:  true,
})

// List only replayed events
events, err := storage.ListEvents(QueryOptions{
    Forwarded:  false,
})

// List original events that have been replayed
events, err := storage.ListEvents(QueryOptions{
    HasReplayedEvents: true,
})

Querying Replay Events

You can query replayed events using the forwarded filter. For example, to list all replayed events:

events, err := storage.ListEvents(QueryOptions{
    Forwarded:  true,
})

You can also query original events that have been replayed using the HasReplayedEvents filter:

events, err := storage.ListEvents(QueryOptions{
    HasReplayedEvents: true,
})

Event Replay

HubProxy allows you to replay webhook events for testing, recovery, or debugging purposes.

Replay ID Format

Each replayed event has an ID in the format: original-id-replay-uuid

For example: - Original event ID: d2a1f85a-delivery-id-123 - Replayed event ID: d2a1f85a-delivery-id-123-replay-abc123

This format ensures: 1. Easy tracing back to original event 2. Unique IDs for multiple replays of same event 3. Clear identification of replayed events

Replay Single Event

// Replay a single event by its ID
event, err := storage.ReplayEvent("d2a1f85a-delivery-id-123")

Development Tools

1. Development Environment (tools/dev.sh) Sets up a complete development environment with SQLite database and test server. ```bash # Start the development environment (required before using other tools) ./tools/dev.sh

# Customize webhook secret HUBPROXY_WEBHOOK_SECRET=my-secret ./tools/dev.sh

# Customize test server port ./tools/dev.sh --target-port 8083 `` This will: - Create a SQLite database in.cache/hubproxy.db` - Start a test server to receive forwarded webhooks - Start the webhook proxy with GitHub IP validation disabled

Default settings: - Webhook secret: dev-secret (via HUBPROXY_WEBHOOK_SECRET env var) - Test server port: 8082 - SQLite database: .cache/hubproxy.db

1. Webhook Simulator (internal/cmd/dev/simulate/main.go) Simulates GitHub webhook events to test the proxy's handling and forwarding. ```bash # Send test webhooks with the default secret go run internal/cmd/dev/simulate/main.go --secret dev-secret

# Send specific event types go run internal/cmd/dev/simulate/main.go --secret dev-secret --events push,pull_request

# Add delay between events go run internal/cmd/dev/simulate/main.go --secret dev-secret --delay 2s ```

1. Query Tool (internal/cmd/dev/query/main.go) Inspects and analyzes webhook events stored in the database. ```bash # Show recent events go run internal/cmd/dev/query/main.go

# Show event statistics go run internal/cmd/dev/query/main.go --stats

# Filter by event type go run internal/cmd/dev/query/main.go --type push

# Filter by repository go run internal/cmd/dev/query/main.go --repo "owner/repo" ```

1. Test Server (internal/cmd/dev/testserver/main.go) Simple HTTP server that logs received webhooks for verification. Note: You don't need to run this directly as dev.sh starts it for you.

```bash # Start on default port 8082 go run internal/cmd/dev/testserver/main.go

# Start on custom port go run internal/cmd/dev/testserver/main.go --port 8083 ```

To verify events are flowing: bash # Watch events in real-time tail -f .cache/testserver.log

Running Tests

# Run all tests
make test

# Run specific package tests
go test ./internal/storage/...
go test ./internal/webhook/...

# Run with race detection
go test -race ./...

Testing Database Connections

# Test PostgreSQL connection
psql "postgres://user:pass@localhost:5432/hubproxy"

# Test MySQL connection
mysql -h localhost -P 3306 -u user -p hubproxy

# Test SQLite database
sqlite3 .cache/hubproxy.db

API Reference

HubProxy provides both REST and GraphQL APIs for querying and replaying webhook events.

API Security

The API server runs on a separate port (default: 8081) from the webhook handler (default: 8080). This separation allows for different security policies:

• Webhook Handler (port 8080): Should be publicly accessible to receive GitHub webhooks
• API Server (port 8081): Contains sensitive data and control functions, and should be secured

Security Recommendations:

1. Network Isolation: Keep the API port (8081) behind a firewall or internal network
2. Reverse Proxy: If exposing the API externally, use a reverse proxy with authentication
3. Access Control: Consider implementing one of these authentication methods:
4. HTTP Basic Authentication via a reverse proxy
5. API tokens with a tool like Caddy or Nginx
6. VPN or Tailscale for secure network-level access
7. TLS Encryption: Always use HTTPS for API communications
8. IP Restrictions: Limit API access to specific IP ranges

Example Nginx Configuration with Basic Auth:

server {
    listen 443 ssl;
    server_name api.hubproxy.example.com;

    ssl_certificate /path/to/cert.pem;
    ssl_certificate_key /path/to/key.pem;

    location / {
        auth_basic "HubProxy API";
        auth_basic_user_file /etc/nginx/.htpasswd;
        proxy_pass http://localhost:8081;
    }
}

REST API

All REST API endpoints return JSON responses.

List Events

GET /api/events

Lists webhook events with filtering and pagination.

Query Parameters: - type (optional): Filter by event type (e.g., "push", "pull_request") - repository (optional): Filter by repository full name (e.g., "owner/repo") - sender (optional): Filter by GitHub username - since (optional): Start time in RFC3339 format (e.g., "2024-02-01T00:00:00Z") - until (optional): End time in RFC3339 format - forwarded (optional): Filter by forwarding status (true/false) - limit (optional): Maximum number of events to return (default: 50) - offset (optional): Number of events to skip for pagination

Response:

{
  "events": [
    {
      "id": "d2a1f85a-delivery-id-123",
      "type": "push",
      "headers": {
        "X-GitHub-Event": ["push"],
        "X-GitHub-Delivery": ["d2a1f85a-delivery-id-123"],
        "X-Hub-Signature-256": ["sha256=..."]
      },
      "payload": {
        "ref": "refs/heads/main",
        "before": "6113728f27ae82c7b1a177c8d03f9e96e0adf246",
        "after": "76ae82c7b1a177c8d03f9e96e0adf2466113728f",
        "repository": {
          "full_name": "owner/repo",
          "private": false
        },
        "pusher": {
          "name": "username",
          "email": "user@example.com"
        }
      },
      "created_at": "2024-02-06T00:00:00Z",
      "forwarded_at": "2024-02-06T00:00:01Z",
      "repository": "owner/repo",
      "sender": "username"
    }
  ],
  "total": 100
}

Get Event Statistics

GET /api/stats

Returns event type statistics for a given time period.

Query Parameters: - since (optional): Start time in RFC3339 format (default: 24 hours ago)

Response:

{
  "push": 50,
  "pull_request": 25,
  "issues": 10
}

Replay Single Event

POST /api/events/{id}/replay

Replays a specific webhook event by its ID. The ID should be GitHub's original delivery ID.

Response Fields: - id: Unique event ID in format original-id-replay-uuid - type: GitHub event type (e.g., "push", "pull_request") - payload: Original webhook payload from GitHub - created_at: When the event was replayed - forwarded_at: When the event was forwarded (null if not yet forwarded) - repository: Repository full name - sender: GitHub username that triggered the event - replayed_from: ID of the original event that was replayed

Response Example:

{
  "id": "d2a1f85a-delivery-id-123-replay-abc123",
  "type": "push",
  "payload": {
    "ref": "refs/heads/main",
    "before": "6113728f27ae82c7b1a177c8d03f9e96e0adf246",
    "after": "76ae82c7b1a177c8d03f9e96e0adf2466113728f",
    "repository": {
      "full_name": "owner/repo",
      "private": false
    },
    "pusher": {
      "name": "username",
      "email": "user@example.com"
    }
  },
  "created_at": "2024-02-06T00:00:00Z",
  "forwarded_at": null,
  "repository": "owner/repo",
  "sender": "username",
  "replayed_from": "d2a1f85a-delivery-id-123"
}

Replay Events by Time Range

```http POST /