Get a recommended combine config. Argument: num_ranks: the number of ranks. Returns: config: the re

Get a recommended dispatch config. Argument: num_ranks: the number of ranks. Returns: config: the r

Get ranks that were killed at current phase.

Get the key-value cache size for the GPT-OSS model (per token). The KV cache is based on the Grouped Query Attention mechanism:

Get the key-value cache size for the Llama 3.1 model (per token). Returns: int: The size of the key-value cache, current

Get the key-value cache size for the DeepSeek-R1 model. Returns: int: The size of the key-value cache, currently hardcod

Get a minimum buffer size requirement for low-latency mode. The size calculation will be done with BF16. Arguments: num_

Get the maximum participating rank index

Get system memory information from /proc/meminfo

Get ranks to connect to at current phase (only the new ones).

Returns a specific notification for a given agent as raw bytes

Gets the parameters if this response has a value

Backward-compatible alias for get_low_latency_buffer_size_hint.

Get ranks to remove at current phase.

auditwheel repair renames all libs to include a hash of the library name e.g. "nixl.libs/libboost_atomic-fb1368c6.so.1.66.0" Extract mapp

Get string representation of telemetry event type enum

Get the buffer version

Handle remote memory and storage transfers as target.

@brief Queries whether there is any NIC with PCIe bus info. */

Multi-request completion tracking methods

Invalidate this agent's metadata in etcd This signals to other agents that this agent's metadata is no longer valid. # Arguments `opt_args` - Option

Invalidates a remote metadata for this agent

* Validate parameters for prepXfer operation. * Ensures operation type, memory types, and descriptor counts are valid. * * @param operation Transfe

Returns an iterator over the parameter key-value pairs

Load remote metadata, if supported.

Log completion percentage at regular intervals (every log_percent_step percent)

A low-latency implementation for combining tokens (reduce **with weights**) with NIXL device API. This kernel requires all the ranks

A low-latency implementation for dispatching with NIXL device API. This kernel requires all the ranks (no matter intranode or interno

Args: - prefill_gpus: List of GPUs ranks that are used for prefill - rail_optimized: Whether to reorder the decode workers to mat

Main function

Advance to next phase.

It is unsafe to move nixlAgent object */

Convert nixl_xfer_dlist_t to vector of xferBenchIOV