Class MmapCache

salt/utils/mmap_cache.py:148–1850

A memory-mapped hash table backed by an index file, a heap file, and a roster file. **Index file (``path``)** Slot 0 is a *header* storing three uint64 counters: * ``occupied_count`` — number of live (OCCUPIED) data slots * ``deleted_count`` — number of soft-deleted (DEL

class MmapCache:
    """
    A memory-mapped hash table backed by an index file, a heap file, and a
    roster file.

    **Index file (``path``)**

    Slot 0 is a *header* storing three uint64 counters:

    * ``occupied_count`` — number of live (OCCUPIED) data slots
    * ``deleted_count``  — number of soft-deleted (DELETED) data slots
    * ``high_water_mark`` — highest data-slot index ever written

    Data slots occupy positions ``1 … size-1``.  Each data slot stores a
    null-padded key plus a pointer (offset + length) into the heap file and
    an mtime timestamp (nanoseconds).

    **Heap file (``path + ".heap"``)**

    A flat binary append-log for variable-size values.  Deleted or superseded
    heap regions are reclaimed by ``atomic_rebuild``.

    **Roster file (``path + ".roster"``)**

    A packed array of ``uint32`` slot indices, one per live (OCCUPIED) entry.
    ``list_items()`` reads this file directly instead of scanning the full
    index, making it O(occupied) regardless of table size or fill factor.
    The roster is kept in sync by ``put()`` and ``delete()`` under the write
    lock, and rebuilt atomically by ``atomic_rebuild``.

    **Performance summary**

    * ``get`` / ``get_mtime`` / ``contains``: O(1) average (open-addressing)
    * ``put`` / ``delete``: O(1) average + one roster file append/rewrite
    * ``list`` / ``list_items``: O(occupied) — independent of table size
    * ``get_stats`` occupied+deleted counters: O(1) from header
    * ``atomic_rebuild``: O(items) to repopulate

    **Key length**

    Keys are encoded to bytes and silently truncated to ``key_size`` bytes —
    two logical keys that share the same first ``key_size`` bytes will collide
    on the same slot. Callers that may use long or user-supplied keys must
    either size ``key_size`` accordingly or pre-hash the keys themselves.
    """

    def __init__(
        self,
        path,
        size=1_000_000,
        slot_size=96,
        key_size=64,
        heap_path=None,
        staleness_check_interval=0.25,
        verify_checksums=True,
        max_segment_bytes=DEFAULT_MAX_SEGMENT_BYTES,
    ):
        _ensure_xxhash()