Budget subtracts the fixed reservations from the total context size, then leaves a headroom margin (see budgetHeadroomDivisor) so a char/4 undercount can't push the real prompt past the declared window.
(ctxSize int)
| 393 | break |
| 394 | } |
| 395 | } |
| 396 | if dangling { |
| 397 | continue |
| 398 | } |
| 399 | } |
| 400 | out = append(out, m) |
| 401 | } |
| 402 | return out |
| 403 | } |
| 404 |