What
Cache breakpoints are now applied to the last two messages in a conversation instead of only the final message, potentially improving cache hit rates with the Anthropic API.
Details
- Previously, only the very last message in a conversation received a cache breakpoint marker
- Now the last two messages both receive cache breakpoints, increasing the likelihood of cache reuse when new messages are appended
- The
skipCacheWriteflag now only suppresses the breakpoint on the final message, leaving the second-to-last message always cached - This is an internal optimization; users may experience marginally faster response times due to improved cache utilization, but no action is required
Evidence
Cache breakpoint logic in message preparation (search for "tengu_api_cache_breakpoints") — K$z() at line ~519758
Strings lifted out of the shipped bundle, so the claim above can be checked against them.