Deutsch | Português (BR) | Русский | Español | Українська

DBLab Engine

⚡ Blazing-fast PostgreSQL cloning and branching 🐘

🛠️ Build powerful dev/test environments.

🔃 Cover 100% of DB migrations with CI tests.

💡 Quickly verify ChatGPT ideas to get rid of hallucinations.

Available for any PostgreSQL, including self-managed and managed services^* like AWS RDS, GCP Cloud SQL, Supabase, and Timescale.

It can be installed and used anywhere: across all cloud environments and on-premises.

Features | Documentation | Blog | Community & Support | Contributing

_{^*For managed PostgreSQL cloud services like AWS RDS or Heroku, direct physical connection and PGDATA access aren't possible. In these cases, DBLab should run on a separate VM within the same region. It will routinely auto-refresh its data, effectively acting as a database-as-a-service solution. This setup then offers thin database branching ideal for development and testing.}

Why DBLab?

• Build dev/QA/staging environments using full-scale, production-like databases.
• Provide temporary full-size database clones for SQL query analysis and optimization (see also: SQL optimization chatbot Joe).
• Automatically test database changes in CI/CD pipelines, minimizing risks of production incidents.
• Rapidly validate ChatGPT or other LLM concepts, check for hallucinations, and iterate towards effective solutions.

For example, cloning a 1 TiB PostgreSQL database takes just about 10 seconds. On a single machine, you can have dozens of independent clones running simultaneously, supporting extensive development and testing activities without any added hardware costs.

Try it yourself right now: - Visit Postgres.ai Console, set up your first organization, and provision a DBLab Standard Edition (DBLab SE) to any cloud or on-premises environment. - Pricing (starting at $62/month) - Documentation: How to install DBLab SE - Demo: https://demo.dblab.dev (use the token demo-token to access) - Looking for a free version? Install the DBLab Community Edition by following this tutorial.

How it works

Thin cloning is fast because it is based on Copy-on-Write (CoW). DBLab employs two technologies for enabling thin cloning: ZFS (default) and LVM.

Using ZFS, DBLab routinely takes new snapshots of the data directory, managing a collection of them and removing old or unused ones. When requesting a fresh clone, users have the option to select their preferred snapshot.

Read more: - How it works - Database Migration Testing - SQL Optimization with Joe Bot - Questions and answers

Where to start

• DBLab tutorial for any PostgreSQL database
• DBLab tutorial for Amazon RDS
• How to install DBLab SE using Postgres.ai Console
• How to install DBLab SE using AWS Marketplace

Case studies

• GitLab: How GitLab iterates on SQL performance optimization workflow to reduce downtime risks

Features

• Speed & scale
  • Blazing-fast cloning of PostgreSQL databases – clone in seconds, irrespective of database size
  • Theoretical max of snapshots/clones: 2⁶⁴ (ZFS, default)
  • Maximum size of PostgreSQL data directory: 256 quadrillion zebibytes, or 2¹²⁸ bytes (ZFS, default)
• Support & technologies
  • Supported PostgreSQL versions: 10–18
  • Thin cloning (CoW) technologies: ZFS and LVM
  • UI for manual tasks and API & CLI for automation
  • Packaged in Docker containers for all components
• PostgreSQL containers
  • Popular extensions including contrib modules, pgvector, HypoPG and many others (docs)
  • Customization capabilities for containers (docs)
  • Docker container and PostgreSQL configuration parameters in the DBLab config
• Source database requirements
  • Location flexibility: self-managed PostgreSQL, AWS RDS, GCP Cloud SQL, Azure, etc.—no source adjustments needed.
  • No ZFS or Docker requirements for source databases
• Data provisioning & retrieval
  • Physical (pg_basebackup, WAL-G, pgBackRest) and logical (dump/restore) provisioning
  • Partial data retrieval in logical mode (specific databases/tables)
  • Database renaming during snapshot creation (databaseRename option)
  • Continuous update in physical mode
  • Periodic full refresh in logical mode without downtime
• Recovery & management
  • Fast Point in Time Recovery (PITR) for physical mode
  • Auto-deletion of unused clones
  • Snapshot retention policies in DBLab configuration
• Clones
  • "Deletion protection" for preventing clone deletion
  • Persistent clones withstand DBLab restarts
  • "Reset" command for data version switching
  • Resource quotas: CPU, RAM
• Monitoring & security
  • /healthz API endpoint (no auth), extended /status endpoint (API docs)
  • Prometheus metrics endpoint (/metrics) for monitoring
  • Netdata module for insights

How to contribute

Support us on GitHub/GitLab

The simplest way to show your support is by giving us a star on GitHub or GitLab! ⭐

Spread the word

• Tweet about DBLab and mention @Database_Lab.
• Share a link to this repository on your favorite social media platform.

Share your experience

If DBLab has been a vital tool for you, tell the world about your journey. Use the logo from the ./assets folder for a visual touch. Whether it's in documents, presentations, applications, or on your website, let everyone know you trust and use DBLab.

HTML snippet for lighter backgrounds:

<a href="http://databaselab.io">
  <img width="400" src="https://postgres.ai/assets/powered-by-dle-for-light-background.svg" />
</a>

For darker backgrounds:

<a href="http://databaselab.io">
  <img width="400" src="https://postgres.ai/assets/powered-by-dle-for-dark-background.svg" />
</a>

Propose an idea or report a bug

For proposals, bug reports, and participation in development, see our Contributing Guide.

Reference guides

• DBLab components
• Client CLI reference
• DBLab API reference
• DBLab configuration reference

How-to guides

• How to install and initialize Database Lab CLI
• How to manage DBLab
• How to work with clones
• How to work with branches
• How to integrate DBLab with GitHub Actions – TBD
• How to integrate DBLab with GitLab CI/CD – TBD

You can find more in the "How-to guides" section of the documentation.

Miscellaneous

• DBLab Docker images
• Extended Docker images for PostgreSQL (with plenty of extensions)
• SQL Optimization chatbot (Joe Bot)
• DB Migration Checker

License

The DBLab source code is licensed under the OSI-approved open source license Apache 2.0.

Reach out to the Postgres.ai team if you want a trial or commercial license that does not contain the GPL clauses: Contact page.

Community & Support

• Database Lab Engine Community Covenant Code of Conduct
• Where to get help: Contact page.
• Community Slack
• If you need to report a security issue, follow the instructions in Database Lab Engine Security Guidelines.

Many thanks to our amazing contributors!

<a href = "https://github.com/postgres-ai/d