Memberlist

A highly customable, adaptable, runtime agnostic and WASM/WASI friendly Gossip protocol which helps manage cluster membership and member failure detection.

Port and improve HashiCorp's memberlist to Rust.

[][Github-url] [][CI-url] [][codecov-url]

[][doc-url] [][crates-url] [][crates-url]

English | 简体中文

Introduction

memberlist is a rust crate that manages cluster membership and member failure detection using a gossip based protocol.

The use cases for such a library are far-reaching: all distributed systems require membership, and memberlist is a re-usable solution to managing cluster membership and node failure detection.

memberlist is eventually consistent but converges quickly on average. The speed at which it converges can be heavily tuned via various knobs on the protocol. Node failures are detected and network partitions are partially tolerated by attempting to communicate to potentially dead nodes through multiple routes.

memberlist is WASM/WASI friendly, all crates can be compiled to wasm-wasi and wasm-unknown-unknown (need to configure the crate features).

Design

Unlike the original Go implementation, Rust's memberlist use highly generic and layered architecture, users can easily implement a component by themselves and plug it to the memberlist. Users can even custom their own Id and Address.

Here are the layers:

• Transport Layer

By default, Rust's memberlist provides two kinds of transport -- QuicTransport and NetTransport.

• Runtime Layer

  Async runtime agnostic are provided by agnostic's Runtime trait, tokio, async-std and smol are supported by default. Users can implement their own Runtime and plug it into the memberlist.

• Address Resolver Layer

  The address resolver layer is supported by nodecraft's AddressResolver trait.

• Serialize/Deserilize Layer

  By default, Rust's memberlist is using length-prefix encoding (Lpe) to serialize/deserialize messages to bytes or visa-vise. The implemention of Lpe tries the best to avoid reallocating when doing the serialize/deserialize.

  But, users can use any other serialize/deserialize framework by implementing Wire trait.

• NetTransport

  Builtin stream layers for NetTransport:

  • Tcp: based on TCP and UDP
  • Tls: based on rustls and UDP

• QuicTransport

  QUIC transport is an experimental transport implementation, it is well tested but still experimental.

  Builtin stream layers for QuicTransport: - Quinn: based on quinn

Users can still implement their own stream layer for different kinds of transport implementations.

• Delegate Layer

This layer is used as a reactor for different kinds of messages.

• Delegate

  Delegate is the trait that clients must implement if they want to hook into the gossip layer of Memberlist. All the methods must be thread-safe, as they can and generally will be called concurrently.

  Here are the sub delegate traits:

  • AliveDelegate

  Used to involve a client in processing a node "alive" message. When a node joins, either through a packet gossip or promised push/pull, we update the state of that node via an alive message. This can be used to filter a node out and prevent it from being considered a peer using application specific logic.

  • ConflictDelegate

  Used to inform a client that a node has attempted to join which would result in a name conflict. This happens if two clients are configured with the same name but different addresses.

  • EventDelegate

  A simpler delegate that is used only to receive notifications about members joining and leaving. The methods in this delegate may be called by multiple threads, but never concurrently. This allows you to reason about ordering.

  • MergeDelegate

  Used to involve a client in a potential cluster merge operation. Namely, when a node does a promised push/pull (as part of a join), the delegate is involved and allowed to cancel the join based on custom logic. The merge delegate is NOT invoked as part of the push-pull anti-entropy.

  • NodeDelegate

  Used to manage node related events. e.g. metadata

  • PingDelegate

  Used to notify an observer how long it took for a ping message to complete a round trip. It can a