You'll noticeTier: how much it should matter to you
3Useful: my rating, 1 to 5
2Signal: worth watching, 1 to 5
PerformanceArea: what it touches
Internal ChangesKind: in v2.1.232,
Internal ChangesSection of the release
What
Preparing the message list for a request now caches the previous input and output and reuses the unchanged prefix and suffix when the conversation grows, rather than rebuilding everything each turn.
Details
Only streaming assistant messages whose stop reason is still null are re-walked.
That pending set is pruned once it exceeds a threshold constant.
Evidence
PRUNE_PENDING_AT
Strings lifted out of the shipped bundle, so the claim above can be checked against them.
