MCPcopy Create free account
hub / github.com/InferCore/InferCore / warnPerReplicaRateLimit

Function warnPerReplicaRateLimit

internal/server/server.go:888–905  ·  view source on GitHub ↗

warnPerReplicaRateLimit logs a startup advisory when any tenant has a non-zero rate_limit_rps. The in-memory rate limiter counts within a single process; each replica enforces its own window independently. In a multi-replica deployment the effective global limit is N × rate_limit_rps. To approximate

(cfg *config.Config)

Source from the content-addressed store, hash-verified

886// independently. In a multi-replica deployment the effective global limit is N × rate_limit_rps.
887// To approximate a global cap, set rate_limit_rps = desired_global_rps / replica_count.
888func warnPerReplicaRateLimit(cfg *config.Config) {
889 var limited []string
890 for _, t := range cfg.Tenants {
891 if t.RateLimitRPS > 0 {
892 limited = append(limited, t.ID)
893 }
894 }
895 if len(limited) == 0 {
896 return
897 }
898 log.Printf(
899 "event=rate_limit_per_replica tenants=%v "+
900 "note=\"rate_limit_rps is enforced per-replica (in-memory); "+
901 "in a multi-replica deployment the effective global limit is N×rate_limit_rps. "+
902 "To approximate a global cap, divide the desired RPS by your replica count.\"",
903 limited,
904 )
905}

Callers 1

NewWithDependenciesFunction · 0.85

Calls

no outgoing calls

Tested by

no test coverage detected