Add's a new node to the HNSW graph, connecting it to existing nodes based on proximity

Close closes the underlying file of the DiskStore

Quantize compresses a float32 vector into an int8 vector

DistQuantize calculates the approximate squared Euclidean distance between two vectors natively in int8. This is much faster as integer math is cheape

Dequantize unpacks int8 vector back to float32 for precise math

Join handles join requests and returns a shard for the specific ID to be used by a new node for joining a cluster.

searchLayer finds the closest node to query in a specific layer starting from entry point

Apply applies a Raft Log Entry to the FSM.

Write appends data to the disk and returns its FileLocation

Generic Test Runner to handle Concurrency and Timing

Helper to send HTTP requests

NewDiskStore initializes a new DiskStore at the given path.

Return a new HNSW Index Tree

Initializes arena with a pre allocated capacity

Read retrieves data from the disk based on the provided FileLocation

Returns the total number of vectors stored

randomLevel generates a level for a new node (Geometric Distribution)

Inserts a vector in the arena and returns its global index

Delete handles delete requests and flags a vector with tombstone for deletion

Delete marks a node as deleted using a tombstone. Thne actual node remains intact to preserve structure but is ignored in search results.

Retrieves a vector by its global index

Insert handles insert requests and adds a new vector record to the cluster

Search handles search requests and returns top K similar vectors from the database

Search finds and returns the k closest nodes to the query vector using the HNSW algorithm.

Euclidean Distance (Squared) - Faster than Cosine for HNSW usually TODO: Use SIMD for pure math performance