libmdbx

libmdbx is an extremely fast, compact, powerful, embedded, transactional key-value database, with Apache 2.0 license. libmdbx has a specific set of properties and capabilities, focused on creating unique lightweight solutions.

1. Allows a swarm of multi-threaded processes to ACIDly read and update several key-value maps and multimaps in a locally-shared database.

2. Provides extraordinary performance, minimal overhead through Memory-Mapping and Olog(N) operations costs by virtue of B+tree.

3. Requires no maintenance and no crash recovery since it doesn't use WAL, but that might be a caveat for write-intensive workloads with durability requirements.

4. Enforces serializability for writers just by single mutex and affords wait-free for parallel readers without atomic/interlocked operations, while writing and reading transactions do not block each other.

5. Guarantee data integrity after crash unless this was explicitly neglected in favour of write performance.

6. Supports Linux, Windows, MacOS, Harmony, Android, iOS, FreeBSD, DragonFly, Solaris, OpenSolaris, OpenIndiana, NetBSD, OpenBSD and other systems compliant with POSIX.1-2008.

7. Compact and friendly for fully embedding. Just a few flat source code files. Here no internal threads nor server process(es), but implements core part of the Berkeley DB API with many powerful extensions.

Historically, libmdbx is a deeply revised and extended descendant of the legendary Lightning Memory-Mapped Database. libmdbx inherits all benefits from LMDB, but resolves some issues and adds a large set of improvements.

Please refer to the online official libmdbx documentation site with C API description and pay attention to the C++ API. Donations are welcome to ETH 0xD104d8f8B2dC312aaD74899F83EBf3EEBDC1EA3A, BTC bc1qzvl9uegf2ea6cwlytnanrscyv8snwsvrc0xfsu, SOL FTCTgbHajoLVZGr8aEFWMzx3NDMyS5wXJgfeMTmJznRi. Всё будет хорошо!

Telegram Group archive: 1, 2, 3, 4, 5, 6, 7.

The Turnpoint

in English

To get acquainted with important changes and plans, we recommend reading the compact presentation "libmdbx: successes, obstacles, goals and roadmap", which contains important explanations in the form of embedded comments.

For ease of use and to eliminate potential limitations in both distribution and obstacles in technology development, libmdbx is distributed as an amalgamated source code starting at the end of 2025. The source code of the tests, as well as the internal documentation, will be available only to the team directly involved in the development. The new libmdbx development strategy is presented, the essence of which is the continuous movement towards the MithrilDB. In this regard, some of the information provided below and in other parts of the documentation may be inaccurate or inapplicable. We will try to resolve all discrepancies as quickly as possible.

The libmdbx code will forever remain open and with high-quality free support, as far as the life circumstances of the project participants allow. However, support will be provided only for officially published versions of the code. As an identity criterion, the git tree hash must match the signed commit in the libmdbx public repository. For all other cases, paid support will be offered. We will also insist and enforce that all derivative versions comply with the license requirements, including the explicit presence of a notice stating that such derivative modified code originates from libdmbx, but is no longer original and supported, and is not subject to any quality guarantees from libmdbx.

на Русском (мой родной язык)

Для знакомства с важными изменениями и планами, рекомендуем посмотреть компактную презентацию "libmdbx: успехи, препятствия, цели и план развития", в которой важные пояснения даны в виде встроенных комментариев.

Для удобства использования и устранения потенциальных ограничений как в распространении, так и препятствий в разработке технологий, начиная с конца 2025 года libmdbx распространяется в виде амальгамированного исходного кода. Исходный код тестов, как и внутренняя документация будет доступна только команде непосредственно занимающейся разработкой. Также представлена новая стратегия развития libmdbx, суть которой в непрерывном движении к MithrilDB. В связи с этим часть информации изложенной ниже и в других частях документации может быть неточной или неприменимой. Мы постараемся устранить все несоответствия по-быстрее.

Код libmdbx навсегда останется открытым и с качественной бесплатной поддержкой, насколько это позволят жизненные обстоятельства участников проекта. Однако, поддержка будет оказываться только для официально публикуемых версий кода. В качестве критерия идентичности требуется совпадение git tree hash с подписанным коммитом в публичном репозитории libmdbx. Для всех остальных случаев будет предлагаться платная поддержка. Также мы будем настаивать и добиваться выполнения всеми производными версиями требований лицензии, включая явное присутствие уведомления о том, что такой производный модифицированный код происходит от libdmbx, но уже не является оригинальным и поддерживаемым, и на него не распространяются какие-либо гарантии качества libmdbx.

MithrilDB and Future

The next version is under non-public development and will be released as MithrilDB and libmithrildb for libraries & packages. Admittedly mythical Mithril is resembling silver but being stronger and lighter than steel. Therefore MithrilDB is a rightly relevant name.

MithrilDB is radically different from libmdbx by the new database format and API based on C++20. The goal of this revolution is to provide a clearer and robust API, add more features and new valuable properties of the database. All fundamental architectural problems of libmdbx/LMDB have been solved there, but now the active development has been suspended for top-three reasons:

1. For now libmdbx mostly enough and I’m busy for scalability.
2. Waiting for fresh Elbrus CPU of e2k architecture, especially with hardware acceleration of Streebog and Kuznyechik, which are required for Merkle tree, etc.
3. The expectation of needs and opportunities due to the wide use of NVDIMM (aka persistent memory), modern NVMe and Ангара.

However, MithrilDB will not be available for countries unfriendly to Russia (i.e. acceded the sanctions, devil adepts and/or NATO). But it is not yet known whether such restriction will be implemented only through a license and support, either the source code will not be open at all. Basically I am not inclined to allow my work to contribute to the profit that goes to weapons that kill my relatives and friends. NO OPTIONS.

Nonetheless, I try not to make any promises regarding MithrilDB until release.

Contrary to MithrilDB, libmdbx will forever free and open source. Moreover with high-quality support whenever possible. Tu deviens responsible pour toujours de ce que tu as apprivois. So I will continue to comply with the original open license and the principles of constructive cooperation, in spite of outright Github sabotage and sanctions. I will also try to keep (not drop) Windows support, despite it is an unused obsolete technology for us.

Table of Contents

• Characteristics
  • Features
  • Limitations
  • Gotchas
  • Comparison with other databases
  • Improvements beyond LMDB
  • History & Acknowledgments
• Usage
  • Building and Testing
  • API description
  • Bindings
• Performance comparison
  • Integral performance
  • Read scalability
  • Sync-write mode
  • Lazy-write mode
  • Async-write mode
  • Cost comparison

Characteristics

Features

• Key-value data model, keys are always sorted.

• Fully ACID-compliant, through to MVCC and CoW.

• Multiple key-value tables/sub-databases within a single datafile.

• Range lookups, including range query estimation.

• Efficient support for short fixed length keys, including native 32/64-bit integers.

• Ultra-efficient support for multimaps. Multi-values sorted, searchable and iterable. Keys stored without duplication.

• Data is memory-mapped and accessible directly/zero-copy. In-memory fullscan of database records is extremely-fast.

• Transactions for readers and writers, ones do not block others.

• No transaction conflicts nor deadlocks since writes are strongly serialized.

• Readers are non-blocking, notwithstanding snapshot isolation.

• Reads scale linearly across CPUs.

• Both explicit defragmentation and continuous zero-overhead database compactification.

• Automatic on-the-fly database size adjustment.

• Olog(N) cost of lookup, insert, update, and delete operations by virtue of B+ tree characteristics.

• Improbably fast and robust get(key) operations accelerated by shareable lockfree cache.

• Online hot backup.

• Append operation for efficient bulk insertion of pre-sorted data.

• No WAL nor any transaction journal. No crash recovery needed. No maintenance is required.

• Flexible transaction API with support for nested transactions.

• Customizable database page size.

Limitations

• Page size: a power of 2, minimum 256 (mostly for testing), maximum 65536 bytes, default 4096 bytes.
• Key size: minimum 0, maximum ≈½ pagesize (2022 bytes for default 4K pagesize, 32742 bytes for 64K pagesize).
• Value size: minimum 0, maximum 2146435072 (0x7FF00000) bytes for maps, ≈½ pagesize for multimaps (2022 bytes for default 4K pagesize, 32742 bytes for 64K pagesize).
• Write transaction size: up to 1327217884 pages (4.944272 TiB for default 4K pagesize, 79.108351 TiB for 64K pagesize).
• Database size: up to 2147483648 pages (≈8.0 TiB for default 4K pagesize, ≈128.0 TiB for 64K pagesize).
• Maximum tables/sub-databases: 32765.

Gotchas

• There cannot be more than one writer at a time, i.e. no more than one write transaction at a time.

• libmdbx is based on B+ tree, so access to database pages is mostly random. Thus SSDs provide a significant performance boost over spinning disks for large databases.

• libmdbx uses shadow paging instead of WAL. Thus syncing data to disk might be a bottleneck for write intensive workload.

• libmdbx uses copy-on-write for snapshot isolation during updates, but read transactions prevents recy