MCPcopy Index your code
hub / github.com/Kodiqa-Solutions/VaultS3

github.com/Kodiqa-Solutions/VaultS3 @v4.4.4

Chat with this repo
repository ↗ · DeepWiki ↗ · release v4.4.4 ↗ · + Follow
2,554 symbols 9,206 edges 277 files 950 documented · 37%
What it actually does AI analysis from the code graph — generated when you open this
loading…
README
<img alt="VaultS3" src="https://github.com/Kodiqa-Solutions/VaultS3/raw/v4.4.4/assets/logo-with-text.svg" width="480">

Lightweight S3-compatible object storage. Single binary, <80MB RAM, built-in dashboard.

CI Docker Pulls License Go S3 Ops

Quick Start · Features · S3 API · Dashboard · CLI · Docker


VaultS3 Dashboard Demo


Why VaultS3?

MinIO stripped the admin console from its Community Edition in mid-2025 and archived the open-source repository in February 2026, it is no longer actively maintained, and full management now requires the paid AIStor tier. SeaweedFS is capable but spreads across multiple components (master / volume / filer). It does have a web admin UI (weed admin) and a working FUSE mount. Garage still has no object versioning, WORM/object-lock, or event notifications.

VaultS3 keeps everything in one self-contained binary under 80 MB RAM:

VaultS3 MinIO SeaweedFS Garage
RAM (small deploy) <80 MB 512 MB+ 50-200 MB 50-150 MB
Single binary Yes Yes No Yes
Web dashboard Built-in Paid¹ Yes No
Open-source maintained Yes Archived¹ Yes Yes
Raft clustering Yes Yes Yes Yes
Erasure coding Yes Yes Yes No
Active-active replication Yes Yes No No
FUSE mount Built-in No Yes No
Full-text search Yes No No No
Version diff/tags Yes No No No
Lambda triggers Yes No No No
Virus scanning Yes No No No
Backup scheduler Yes No No No

¹ MinIO removed the admin console from its Community Edition in 2025 and archived the open-source repository in February 2026. Full management now requires the paid AIStor product. This comparison reflects publicly available information as of June 2026, please open an issue if a cell is out of date.

make build && ./vaults3
# Server at http://localhost:9000
# Dashboard at http://localhost:9000/dashboard/

Production Readiness

VaultS3 is honest about what's battle-tested versus still maturing. Pick the lane that matches your risk tolerance:

Path Maturity Notes
Single-node (S3 API, versioning, IAM, dashboard) ✅ Stable The default deployment. Broad test coverage. Runs in production today.
Erasure coding (single-node, multi-disk) ✅ Stable Reed-Solomon encode/reconstruct and the background healer have fault-injection tests (lose disks → reconstruct → heal).
Tiering & backup ✅ Stable Hot/cold migration, transparent promotion, and full/incremental backup are tested. Restore is a manual file copy.
Multi-node Raft clustering 🟡 Beta Metadata writes replicate via Raft consensus (writes accepted on any node via leader-forwarding), object data is placed/served by a live-membership hash ring, inter-node calls are authenticated, and on Kubernetes the cluster auto-forms (leader bootstrap + auto-join + self-heal). Validated end-to-end on a real 3-node cluster: leader election & failover, node recovery with catch-up, cross-node reads, and concurrent load. Still operationally newer, not yet stress/scale/multi-region hardened, so validate against your workload before trusting it as the only copy of critical data.
Active-active replication 🟡 Beta Vector-clock conflict resolution is unit-tested. The cross-site sync worker is less exercised in the wild.

Recommendation: run single-node (optionally with erasure coding across local disks) for production data you care about, and treat clustering/active-active as advanced opt-in features you validate first. Always keep an independent backup. See the Scaling & Operations Guide for redundancy layering and recovery runbooks, and the Benchmarks guide for a reproducible way to measure throughput and RAM on your own hardware.

Features

  • S3-compatible API: Works with any S3 client (AWS CLI, mc, boto3, minio-js)
  • Single binary: One file, no runtime dependencies, no Docker required
  • Low memory: Targets <80MB RAM (vs MinIO's 300-500MB)
  • BoltDB metadata: Embedded key-value store, no external database needed
  • S3 Signature V4: Standard AWS authentication
  • AES-256-GCM encryption at rest: SSE-S3 (static key) and SSE-KMS (HashiCorp Vault or local key provider) encryption modes
  • Per-bucket encryption keys: For bucket-per-tenant setups, each bucket can be encrypted with its own key that is not shared with other tenants (or opt out and stay plaintext). Envelope encryption (master KEK wraps a per-bucket data key). Opt in per bucket via PUT /{bucket}?encryption or the dashboard. Supports key rotation and crypto-shredding. Enable with encryption.per_bucket: true, see design doc
  • SSE-C (customer-provided keys): Operator-blind per-object encryption: the client supplies the key per request (x-amz-server-side-encryption-customer-*). The server encrypts/decrypts with it and stores only the key's MD5, never the key
  • Bucket policies: Public-read, private, custom S3-compatible JSON policies
  • Quota management: Per-bucket size and object count limits
  • Rate limiting: Token bucket rate limiter per client IP and per access key to prevent abuse
  • S3 Select: Execute SQL queries on CSV, JSON, and Parquet objects without downloading the full file
  • Multipart upload: Full lifecycle (Create, UploadPart, UploadPartCopy, Complete, Abort, ListUploads, ListParts)
  • Bucket tagging: S3-compatible tag sets with PUT/GET/DELETE
  • Bucket/Object ACL: S3-compatible ACL responses (GET/PUT)
  • Multiple access keys: Dynamic key management via BoltDB
  • Object tagging: Up to 10 tags per object
  • Range requests: Partial content downloads (206 responses)
  • Copy object: Same-bucket and cross-bucket copies
  • Batch delete: Multi-object delete with XML body
  • Virtual-hosted style URLs: bucket.domain/key in addition to path-style
  • Bucket default retention: Set default GOVERNANCE or COMPLIANCE retention on a bucket, auto-applied to new objects
  • Per-bucket Prometheus metrics: Request counts, bytes in/out, and errors with bucket labels at /metrics
  • Prometheus metrics: /metrics endpoint with storage, request, and runtime stats
  • Presigned URLs: Pre-authenticated URL generation
  • Web dashboard: Built-in React UI at /dashboard/ with home overview page, file browser (grid or list layout with file-type icons, sortable columns, pagination, file preview, metadata panel, version history panel with diff viewer/rollback/tagging, multi-select, bulk delete, bulk zip download, breadcrumb navigation), drag-and-drop file and folder upload, copy-to-clipboard buttons, access key management, activity log, storage stats with auto-refresh, read-only settings viewer, IAM management, audit trail viewer (sortable, paginated), search (sortable, paginated), notifications, replication status, lambda triggers, backup management, bucket config (versioning toggle with status indicator, lifecycle editor, CORS editor), keyboard shortcuts (/ search, ? help), toast notifications (success/error/info), dark/light theme, collapsible sidebar, remember-me sign-in, responsive layout
  • Health checks: /health (liveness) and /ready (readiness) endpoints for load balancers and Kubernetes
  • Graceful shutdown: Drains in-flight requests on SIGTERM/SIGINT with configurable timeout
  • TLS support: Optional HTTPS with configurable cert/key paths
  • Separate dashboard port: Optionally serve the Web UI + its API on a dedicated port (server.console_port, e.g. 9001) apart from the S3 API, so each can have its own firewall rules / TLS / reverse proxy (MinIO-style)
  • Object versioning: Per-bucket versioning with version IDs, delete markers, version-specific GET/DELETE/HEAD
  • Object locking (WORM): Legal hold and retention (GOVERNANCE/COMPLIANCE) to prevent deletion
  • Lifecycle rules: Per-bucket object expiration (auto-delete after N days) with background worker
  • Zstandard compression: Transparent compress-on-write, decompress-on-read with zstd (better ratio and speed than gzip). Objects written by older gzip builds are still read transparently (codec auto-detected by magic number)
  • Small-file packing (experimental): Optionally pack objects up to a size threshold into large append-only volume files (each as an independent zstd frame) with byte-offset locations in BoltDB, plus background dead-space compaction (POST /api/v1/compact), avoids the per-file overhead (inodes, syscalls, disk blocks) of millions of tiny objects. Larger objects fall through to individual files. Not yet composable with encryption or erasure coding (skipped if either is enabled)
  • Scales to millions of objects: Listing and storage stats are served from a sorted BoltDB metadata index with maintained per-bucket counters (size/count updated incrementally on every write), never a filesystem walk. So dashboard stats are O(1) and the object browser pages in milliseconds regardless of bucket size, verified at 1M+ objects (stats 13s → 0.4ms)
  • Access logging: Structured JSON lines log file of all S3 operations
  • Static website hosting: Serve index/error documents from buckets, no auth required
  • IAM users, groups & policies: Fine-grained access control with S3-compatible policy evaluation, default deny, wildcard matching
  • CORS per bucket: S3-compatible CORS configuration with OPTIONS preflight support
  • STS temporary credentials: Short-lived access keys with configurable TTL, auto-cleanup of expired keys
  • Audit trail: Persistent audit log with filtering by user, bucket, time range. Auto-pruning via lifecycle worker
  • IP allowlist/blocklist: Global and per-user CIDR-based IP restrictions with IPv4/IPv6 support
  • S3 event notifications: Per-bucket webhook notifications on object mutations with event type and key prefix/suffix filtering, plus Kafka, NATS, Redis, AMQP/RabbitMQ, PostgreSQL, and Elasticsearch backends
  • Raft clustering: Multi-node cluster with Hashicorp Raft consensus for strongly consistent distributed metadata, automatic leader election, and node join/leave via HTTP API
  • Consistent hashing: xxhash64-based hash ring with virtual nodes for automatic data placement and request routing across cluster nodes via reverse proxy
  • Erasure coding: Reed-Solomon encoding (configurable data/parity shards) for disk-failure protection with background healer that auto-reconstructs degraded objects
  • High availability: Automatic failure detection (health probes with suspect/down state machine), failover proxy routing to healthy replicas, and background rebalancer for membership changes
  • Scalable listing: Object listing is served from the sorted BoltDB metadata index (seek to the page, O(log n + page_size)), so ListObjectsV2 page latency stays flat (~0.7 ms per 1000-key page) whether a prefix holds a thousand or a hundred million objects (measured, not extrapolated), no full-bucket scan
  • 📖 See the Scaling & Operations Guide for multi-disk erasure coding, multi-node cluster setup, large-prefix listing, and lost-disk / lost-server recovery runbooks
  • Active-active replication: Bidirectional site-to-site sync with vector clocks for causal ordering, pluggable conflict resolution (last-writer-wins, largest-object, site-preference), and change log for efficient delta sync
  • Async replication: One-way async replication to peer VaultS3 instances with BoltDB-backed queue, retry with exponential backoff, and loop prevention
  • CLI tool: Standalone vaults3-cli binary for bucket, object, user, and replication management without AWS CLI
  • Presigned upload restrictions: Enforce max file size, content type whitelist, and key prefix on presigned PUT URLs
  • Full-text search: In-memory search index over object metadata, tags, content type, and key patterns with incremental updates
  • Semantic / vector search (optional): Embeds object text via any OpenAI-compatible endpoint (Ollama, llama.cpp, OpenAI…) and serves similarity search + RAG retrieval from POST /api/v1/vectors/query, all in the single binary, no external vector database. Searchable from the dashboard (Keyword / Semantic toggle)
  • Migrate from S3: Import buckets and objects from MinIO, SeaweedFS, Garage, Ceph, AWS S3, Cloudflare R2, Wasabi, Backblaze B2, or any S3-compatible source via a dashboard wizard (pick a source preset that pre-fills the endpoint and SigV4 regi

Extension points exported contracts — how you extend this code

Engine (Interface)
Engine defines the interface for object storage backends. [8 implementers]
internal/storage/engine.go
Backend (Interface)
Backend is the interface for notification delivery backends. [7 implementers]
internal/notify/notify.go
ConflictResolver (Interface)
ConflictResolver picks a winner when two concurrent writes conflict. [3 implementers]
internal/replication/conflict.go
LatencyRecorder (Interface)
LatencyRecorder is the interface for recording request latency. [2 implementers]
internal/middleware/middleware.go
Embedder (Interface)
Embedder turns text into embedding vectors. [2 implementers]
internal/vector/embedder.go
KeyStore (Interface)
KeyStore persists wrapped DEKs per bucket, keyed by version. In production this is backed by BucketEncryptionConfig in t [2 …
internal/bucketcrypto/bucketcrypto.go
RaftApplier (Interface)
RaftApplier is the interface the DistributedStore uses to submit writes. Implemented by cluster.Node. [1 implementers]
internal/metadata/distributed.go
ActivityFunc (FuncType)
ActivityFunc is a callback for recording S3 activity.
internal/s3/handler.go

Core symbols most depended-on inside this repo

writeS3Error
called by 305
internal/s3/xml.go
Get
called by 272
internal/bucketcrypto/bucketcrypto.go
Close
called by 262
internal/backup/target.go
writeError
called by 226
internal/api/helpers.go
Error
called by 207
internal/notify/notify.go
WriteHeader
called by 119
internal/s3/handler.go
Join
called by 109
internal/cluster/node.go
BucketExists
called by 99
internal/metadata/storeiface.go

Shape

Method 1,084
Function 991
Struct 385
Interface 78
FuncType 10
TypeAlias 6

Languages

Go87%
TypeScript13%

Modules by API surface

internal/metadata/store.go164 symbols
internal/metadata/storeiface.go128 symbols
internal/s3/bucket.go69 symbols
internal/metadata/distributed.go66 symbols
internal/s3/object.go40 symbols
internal/s3/integration_test.go40 symbols
internal/config/config.go40 symbols
internal/s3/handler.go36 symbols
internal/bucketcrypto/bucketcrypto.go32 symbols
internal/fuse/mount.go31 symbols
internal/storage/packed.go29 symbols
internal/s3/select.go29 symbols

Datastores touched

vaults3Database · 1 repos

For agents

$ claude mcp add VaultS3 \
  -- python -m otcore.mcp_server <graph>

⬇ download graph artifact