GitHub Contributor Analytics

A modern analytics pipeline for tracking and analyzing GitHub contributions across repositories. The system processes contributor data, generates AI-powered summaries, and maintains a leaderboard of developer activity.

Prerequisites

• Bun (recommended) or Node.js 18+
• GitHub Personal Access Token with repo scope
• OpenRouter API Key (optional, for AI summaries)
• uv (optional, for syncing from production DB)

Features

• Tracks pull requests, issues, reviews, and comments
• Calculates contributor scores based on activity and impact
• Generates AI-powered summaries of contributions
• Exports daily summaries to JSON files
• Maintains contributor expertise levels and focus areas
• Interactive contributor profile pages
• Activity visualizations and metrics
• Daily, weekly, and monthly reports
• Smart contributor scoring system

Wallet Linking (Optional Feature)

Contributors can optionally link their Ethereum and Solana wallet addresses to their GitHub profiles. When configured, users can authenticate via GitHub OAuth and store wallet addresses in their profile README.

Setup: See auth-worker/README.md for Cloudflare Worker deployment and OAuth configuration.

Required secrets (if enabling):

• NEXT_PUBLIC_GITHUB_CLIENT_ID - GitHub OAuth App Client ID
• NEXT_PUBLIC_AUTH_WORKER_URL - Deployed Cloudflare Worker URL

Note: The leaderboard works perfectly fine without this feature. It's purely additive.

Setup

1. Install dependencies:

bun install

1. Set up environment variables in .env using .env.example for reference:

# Required for Github Ingest
GITHUB_TOKEN=your_github_personal_access_token_here
# Required for AI summaries
OPENROUTER_API_KEY=your_api_key_here
# configure local environment to use cheaper models
LARGE_MODEL=openai/gpt-4o-mini

# Optional site info (auto-detected in CI if not set)
SITE_URL=https://your-deployment-url.com
SITE_NAME="Contributor Leaderboard"

# Optional: For wallet linking feature
NEXT_PUBLIC_GITHUB_CLIENT_ID=
NEXT_PUBLIC_AUTH_WORKER_URL=

Then load the environment variables:

source .envrc
# Or if using direnv: direnv allow

1. Configure repositories to track in config/pipeline.config.ts. See the config file for the full schema and options.

2. Initialize Database

You can either initialize an empty database or sync the latest data from production:

Option A - Initialize Empty Database:

# Apply migrations
bun run db:migrate

Option B - Sync Production Data:

If you want to download all historical data from the production data branch instead of having to reingest / generate it on your own, you can use the data:sync command, which depends on uv.

# Install uv first if you don't have it (required for database restoration)

pipx install uv  # Recommended method
# OR
brew install uv  # macOS with Homebrew

# More installation options: https://docs.astral.sh/uv/getting-started/installation/

# Download the latest data from production
bun run data:sync
# Or, if you are on a fork:
bun run data:sync --remote upstream
# This will:
# - Fetch the latest data from the _data branch
# - Copy all data files (stats, summaries, etc.)
# - Restore the SQLite database from the diffable dump

# If you made local changes to the schema that don't exist in prod DB:
bun run db:generate
bun run db:migrate

The data sync utility supports several options:

# View all options
bun run data:sync --help

# Skip confirmation prompts (useful in scripts)
bun run data:sync -y

# Sync from a different remote (if you've added one)
bun run data:sync --remote upstream

# Skip database restoration (only sync generated JSON/MD files)
bun run data:sync --skip-db

# Delete all local data and force sync
bun run data:sync --force

After syncing or initializing the database, you can explore it using Drizzle Studio:

# Launch the database explorer
bun run db:studio

If you encounter any issues with Drizzle Studio due to Node.js version mismatches, you can use a different SQLite browser tool like SQLite Browser.

Quick Start

# Ingest recent data
bun run pipeline ingest --days 90

# Process and calculate scores
bun run pipeline process

# Build the site
bun run build

Commands and Capabilities

You can see the main pipelines and their usages with these commands below:

bun run pipeline ingest -h
bun run pipeline process -h
bun run pipeline export -h
bun run pipeline summarize -h

Data Ingestion

# Ingest latest Github data (default since last fetched, or 7 days)
bun run pipeline ingest

# Ingest from beginning
bun run pipeline ingest --after 2024-10-15

# Ingest with specific date range
bun run pipeline ingest --after 2025-01-01 --before 2025-02-20

# Ingest data for a specific number of days
bun run pipeline ingest --days 30 --before 2024-03-31

# Ingest with verbose logging
bun run pipeline ingest -v

# Ingest with custom config file
bun run pipeline ingest --config custom-config.ts

Data Processing and Analysis

# Process and analyze all repositories
bun run pipeline process

# Force recalculation of scores even if they already exist
bun run pipeline process --force

# Process specific repository
bun run pipeline process --repository owner/repo

# Process with verbose logging
bun run pipeline process -v

# Process with custom config
bun run pipeline process --config custom-config.ts

Generating Stats and Exports

# Export repository stats (defaults to 30 days)
bun run pipeline export

# Export with specific date range
bun run pipeline export --after 2025-01-01 --before 2025-02-20

# Export for a specific number of days
bun run pipeline export --days 60

# Export all data since contributionStartDate
bun run pipeline export --all

# Export for specific repository
bun run pipeline export -r owner/repo

# Export to custom directory
bun run pipeline export --output-dir ./custom-dir/

# Export with verbose logging
bun run pipeline export -v

# Regenerate and overwrite existing files
bun run pipeline export --force

AI Summary Generation

Generated project summaries are stored in data/<owner_repo>/<interval>/summaries/summary_<date>.json.

# Generate repository-level summaries
bun run pipeline summarize -t repository

# Generate overall summaries (after repository summaries are generated)
bun run pipeline summarize -t overall

# Generate contributor summaries
bun run pipeline summarize -t contributors

# Generate summaries with specific date range
bun run pipeline summarize -t repository --after 2025-01-01 --before 2025-02-20

# Force overwrite existing summaries
bun run pipeline summarize -t repository --force

# Generate and overwrite summaries for a specific number of days (default 7 days)
bun run pipeline summarize -t repository --days 90 --force

# Generate repository summaries for all data since contributionStartDate
bun run pipeline summarize -t repository --all

# Generate summaries for a specific repository
bun run pipeline summarize -t repository --repository owner/repo

# Generate only daily and weekly contributor summaries
bun run pipeline summarize -t contributors --daily --weekly

# Generate summaries with verbose logging
bun run pipeline summarize -t repository -v

# Generate summaries for a specific contributor only
bun run pipeline summarize -t contributors -u username

By default, the summarize command wont regenerate summaries that already exist for a given day. To regenerate summaries, you can pass in the -f/--force flag.

Lifetime Summaries

Generate all-time contributor briefings with strategic insights (manual generation only, not automated):

# Single user (recommended for testing)
bun run pipeline summarize -t contributors --lifetime -u username

# All users (expensive - generates AI summaries for every contributor)
bun run pipeline summarize -t contributors --lifetime --force

Note: Lifetime summaries are memory-intensive and make many AI calls. Always use the -u/--username filter when testing prompt changes or debugging.

Static JSON API

The pipeline generates static JSON API endpoints that can be consumed by external tools, dashboards, or AI agents. These files are generated during pipeline execution and served as static files.

Leaderboard API

# Generate leaderboard API endpoints
bun run pipeline export-leaderboard

# With custom limit (default 100, 0 = unlimited)
bun run pipeline export-leaderboard --limit 50

# Output to custom directory
bun run pipeline export-leaderboard --output-dir ./custom-dir/

Endpoints:

Endpoint	Description
/api/leaderboard-monthly.json	Current month's leaderboard
/api/leaderboard-weekly.json	Current week's leaderboard
/api/leaderboard-lifetime.json	All-time leaderboard
/api/contributors/{username}/profile.json	Complete character sheet for user
/api/index.json	API discovery endpoint

API Base URL: https://{your-domain}/api/

For GitHub Pages deployments, your base URL follows this pattern:

• Org/user site (username.github.io repo): https://{username}.github.io/api/
• Project site (any other repo): https://{username}.github.io/{repo-name}/api/

Response structure:

{
  "version": "1.0",
  "period": "monthly",
  "startDate": "2025-01-01",
  "endDate": "2025-01-31",
  "generatedAt": "2025-01-15T12:00:00Z",
  "totalUsers": 150,
  "leaderboard": [
    {
      "rank": 1,
      "username": "contributor1",
      "avatarUrl": "https://...",
      "characterClass": "Maintainer",
      "tier": "elite",
      "score": 1250,
      "prScore": 800,
      "issueScore": 200,
      "reviewScore": 150,
      "commentScore": 100,
      "wallets": { "solana": "...", "ethereum": "..." },
      "focusAreas": [
        {
          "tag": "core",
          "score": 565.5,
          "percentage": 45.2,
          "rank": 3,
          "totalInArea": 45
        }
      ],
      "scoreBreakdown": {
        "total": 1250,
        "tier": "elite",
        "percentile": 95.3,
        "characterClass": "Maintainer",
        "distribution": {
          "prs": { "score": 800, "percentage": 64.0, "label": "Builder" },
          "issues": { "score": 200, "percentage": 16.0, "label": "Hunter" },
          "reviews": { "score": 150, "percentage": 12.0, "label": "Reviews" },
          "comments": { "score": 100, "percentage": 8.0, "label": "Engagement" }
        }
      },
      "achievements": [
        { "type": "level", "tier": "elite", "earnedAt": "2024-11-15T10:00:00Z" }
      ],
      "profile": {
        "contributorType": "maintainer",
        "prMergeRate": 93.3,
        "reviewActivity": "high"
      },
      "links": {
        "profile": "https://elizaos.github.io/profile/contributor1",
        "profileApi": "https://elizaos.github.io/api/contributors/contributor1/profile.json",
        "summary": "https://elizaos.github.io/api/summaries/contributors/contributor1/day/latest.json",
        "github": "https://github.com/contributor1"
      }
    }
  ]
}

Character System: Leaderboard entries include MMORPG-style progression:

• Tiers: beginner → regular → active → veteran → elite → legend (based on total score)
• Classes: Builder (PRs), Hunter (Issues), Scribe (Docs), Maintainer (Builder + Reviews), Pathfinder (Builder + Hunter)
• Focus Areas: Top 3 expertise tags with global rankings (e.g., "#3 in core out of 45 contributors")
• Percentile: Shows what % of all contributors this user outscores

See the API documentation page for complete schemas and examples.

Summary API

Summaries are generated alongside markdown files during the summarize command. JSON API artifacts include metadata for caching and validation.

Endpoints:

Endpoint Pattern	Description
/api/summaries/overall/{interval}/{date}.json	Overall summary for a specific date
/api/summaries/overall/{interval}/latest.json	Most recent overall summary
/api/summaries/overall/{interval}/index.json	Index of all overall summaries
/api/summaries/repos/{owner}_{repo}/{interval}/{date}.json	Repository summary
/api/summaries/repos/{owner}_{repo}/{interval}/latest.json	Most recent repo summary
/api/summaries/repos/{owner}_{repo}/{interval}/index.json	Index of all repo summaries
/api/summaries/contributors/{username}/{interval}/{date}.json	Contributor summary
/api/summaries/contributors/{username}/{interval}/latest.json	Most recent contributor summary
/api/summaries/contributors/{username}/{interval}/index.json	Index of contributor summaries
/api/summaries/contributors/{username}/lifetime.json	All-time contributor summary

Where {interval} is one of: day, week, month (for overall/repos/contributors), or lifetime (contributors only)

Response structure:

```json { "version": "1.0", "type": "overall", "interval": "day", "date": "2025-01-15", "generatedAt": "2025-01-15T23:00:00Z", "sourceLastUpdated": "2025-01-15T23:00:00Z", "contentFormat": "markdown", "contentHash": "sha256...", "entity": { "repoId": "owner/repo" }, "content": "# Summary