reduce-latency changedtest-and-evaluate/strengthen-guardrails/reduce-latency
Nearest release: v2.1.312, published under an hour before this site recorded the change. Shown because the two are within 24 hours of each other. Nothing here says the release caused the edit.
Recorded here
Lines+7added
Lines−7removed
From line
7
where the diff opens
First seen
14 Aug 2026
this site's first read of the page
Recorded edits9to this page, all time
The whole hunk
from line 7, old and new numbered
/
from line 7
77Latency refers to the time it takes for the model to process a prompt and generate an output. Latency can be influenced by various factors, such as the size of the model, the complexity of the prompt, and the underlying infrastructure supporting the model and point of interaction.
88
99<Note>
10 It's always better to first engineer a prompt that works well without model or prompt constraints, and then try latency reduction strategies afterward. Trying to reduce latency prematurely might prevent you from discovering what top performance looks like.
10 It’s always better to first engineer a prompt that works well without model or prompt constraints, and then try latency reduction strategies afterward. Trying to reduce latency prematurely might prevent you from discovering what top performance looks like.
1111</Note>
1212
1313***
from line 16
1616
1717When discussing latency, you might come across several terms and measurements:
1818
19* **Baseline latency:** This is the time taken by the model to process the prompt and generate the response, without considering the input and output tokens per second. It provides a general idea of the model's speed.
20* **Time to first token (TTFT):** This metric measures the time it takes for the model to generate the first token of the response, from when the prompt was sent. It's particularly relevant when you're using streaming (more on that later) and want to provide a responsive experience to your users.
19* **Baseline latency:** This is the time taken by the model to process the prompt and generate the response, without considering the input and output tokens per second. It provides a general idea of the model’s speed.
20* **Time to first token (TTFT):** This metric measures the time it takes for the model to generate the first token of the response, from when the prompt was sent. It’s particularly relevant when you’re using streaming (more on that later) and want to provide a responsive experience to your users.
2121
2222For a more in-depth understanding of these terms, see the [glossary](https://platform.claude.com/docs/en/about-claude/glossary).
2323
from line 226
226226
227227### 3. Stream responses
228228
229Streaming is a feature that allows the model to start sending back its response before the full output is complete. This can significantly improve the perceived responsiveness of your application, as users can see the model's output in real time.
229Streaming is a feature that allows the model to start sending back its response before the full output is complete. This can significantly improve the perceived responsiveness of your application, as users can see the model’s output in real time.
230230
231With streaming enabled, you can process the model's output as it arrives, updating your user interface or performing other tasks in parallel.
231With streaming enabled, you can process the model’s output as it arrives, updating your user interface or performing other tasks in parallel.
232232
233233Visit [Streaming messages](https://platform.claude.com/docs/en/build-with-claude/streaming) to learn about how you can implement streaming for your use case.
234234
from line 238
238238
239239<CardGroup cols={2}>
240240 <Card title="Reduce hallucinations" icon="shield" href="https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/reduce-hallucinations">
241 Minimize hallucinations in Claude's outputs by allowing uncertainty, grounding responses in direct quotes, and verifying claims with citations.
241 Minimize hallucinations in Claude’s outputs by allowing uncertainty, grounding responses in direct quotes, and verifying claims with citations.
242242 </Card>
243243
244244 <Card title="Streaming messages" icon="bolt" href="https://platform.claude.com/docs/en/build-with-claude/streaming">
No line in this hunk matches that.