MCPcopy
hub / github.com/mudler/LocalAI / tokensFromContext

Function tokensFromContext

core/http/middleware/usage.go:155–176  ·  view source on GitHub ↗

tokensFromContext returns canonical token counts stamped by a handler via middleware.StampUsage. Returns ok=false when no stamp is present — the caller then tries the body-parse fallback. A model name without token counts is not considered "stamped" because a record with zero tokens looks the same

(c echo.Context)

Source from the content-addressed store, hash-verified

153// record with zero tokens looks the same as a never-recorded request to
154// later analytics; the second condition is what gates ok.
155func tokensFromContext(c echo.Context) (model string, prompt, completion, total int64, ok bool) {
156 if v, found := c.Get(ContextKeyResponseModel).(string); found {
157 model = v
158 }
159 pPresent := false
160 cPresent := false
161 if v, found := c.Get(ContextKeyPromptTokens).(int64); found {
162 prompt = v
163 pPresent = true
164 }
165 if v, found := c.Get(ContextKeyCompletionTokens).(int64); found {
166 completion = v
167 cPresent = true
168 }
169 if v, found := c.Get(ContextKeyTotalTokens).(int64); found {
170 total = v
171 } else {
172 total = prompt + completion
173 }
174 ok = pPresent || cPresent
175 return
176}
177
178// tokensFromBody covers the passthrough-proxy / foreign-endpoint case
179// where no handler stamps the context. Returns ok=false on any parse

Callers 1

UsageMiddlewareFunction · 0.85

Calls 1

GetMethod · 0.65

Tested by

no test coverage detected