Struct InFlightTrackingClient

core/services/nodes/inflight.go:30–40

InFlightTrackingClient wraps a grpc.Backend and tracks active inference requests in the NodeRegistry. This allows the router's eviction logic to know which models are actively serving and should not be unloaded. Per-replica: a single tracker instance is bound to (nodeID, modelName, replicaIndex). T

// until it is wrapped with track() - so a new inference path can't be added
// without an in-flight accounting decision.
type InFlightTrackingClient struct {
	grpc.ControlBackend                       // passthrough for control-plane / streaming-constructor methods
	inner               grpc.InferenceBackend // tracked inference methods delegate here
	registry            InFlightTracker
	nodeID              string
	modelName           string
	replicaIndex        int

	firstOnce       sync.Once // guards onFirstComplete
	onFirstComplete func()    // called once after the first tracked inference call completes
}

// Compile-time contract: *InFlightTrackingClient must implement the FULL backend
// surface. Because it embeds only ControlBackend, this fails to compile if any