MCPcopy Create free account
hub / github.com/NodeDB-Lab/nodedb / spawn_wal_catchup_task

Function spawn_wal_catchup_task

nodedb/src/control/wal_catchup.rs:37–68  ·  view source on GitHub ↗

Spawn the WAL catch-up background task. Runs on the Tokio runtime (Control Plane). Periodically reads unflushed WAL TimeseriesBatch records and dispatches them to the Data Plane. `initial_lsn` should be `wal.next_lsn()` after startup WAL replay — everything before that has already been replayed.

(
    shared: Arc<SharedState>,
    initial_lsn: Lsn,
    mut shutdown: tokio::sync::watch::Receiver<bool>,
)

Source from the content-addressed store, hash-verified

35/// `initial_lsn` should be `wal.next_lsn()` after startup WAL replay —
36/// everything before that has already been replayed.
37pub fn spawn_wal_catchup_task(
38 shared: Arc<SharedState>,
39 initial_lsn: Lsn,
40 mut shutdown: tokio::sync::watch::Receiver<bool>,
41) {
42 shared
43 .wal_catchup_lsn
44 .store(initial_lsn.as_u64(), Ordering::Release);
45
46 tokio::spawn(async move {
47 // Adaptive interval: 500ms default, tighten when catching up, relax when idle.
48 let mut interval_ms: u64 = 500;
49
50 loop {
51 tokio::select! {
52 _ = tokio::time::sleep(std::time::Duration::from_millis(interval_ms)) => {
53 let result = run_catchup_cycle(&shared).await;
54 interval_ms = match result {
55 CatchupResult::HasMore => 100, // rapid drain
56 CatchupResult::Dispatched => 250, // active, normal pace
57 CatchupResult::Backpressured => 200, // retry soon
58 CatchupResult::Idle => 2000, // nothing to do
59 };
60 }
61 _ = shutdown.changed() => {
62 info!("WAL catch-up task shutting down");
63 break;
64 }
65 }
66 }
67 });
68}
69
70/// Result of a single catch-up cycle.
71enum CatchupResult {

Calls 3

spawnFunction · 0.50
storeMethod · 0.45
as_u64Method · 0.45