Roll out an LLM gateway for your organization changedllm-gateway-rollout
Nearest release: v2.1.292, published an hour before upstream edited the page. Shown because the two are within 24 hours of each other. Nothing here says the release caused the edit.
Upstream edited this page at 6 Oct 2026 18:41 UTC, give or take a minute or two: the time comes from Anthropic’s own sitemap rather than from a commit. This site recorded the change at 20 Oct 2026 18:37 UTC.
Upstream edited
Recorded here
Lines+76added
Lines−84removed
From line
4
where the diff opens
First seen
14 Aug 2026
this site's first read of the page
Recorded edits13to this page, all time
The whole hunk
from line 4, old and new numbered
/
from line 4
44
55This page walks an administrator through rolling out an LLM gateway for Claude Code. It assumes you have a gateway product deployed that meets the [gateway requirements](#gateway-requirements). Deploying or operating any specific product isn't covered here; deploy yours following its vendor's documentation.
66
7<Note>
8 * To connect Claude Code on your own machine to an existing gateway, see [Connect Claude Code to an LLM gateway](/docs/en/llm-gateway-connect)
9 * For what Claude Code sends to a gateway and what to forward, see the [gateway compatibility guide](/docs/en/llm-gateway-protocol)
10</Note>
7> [!NOTE]
8>
9> * To connect Claude Code on your own machine to an existing gateway, see [Connect Claude Code to an LLM gateway](/docs/en/llm-gateway-connect)
10> * For what Claude Code sends to a gateway and what to forward, see the [gateway compatibility guide](/docs/en/llm-gateway-protocol)
1111
1212## Prerequisites
1313
from line 58
5858* `<gateway-key>` is whatever credential lets you call the gateway right now: an administrative key, a test key, or your own developer key if you've already issued one. Not every gateway product has a separate admin credential; if yours doesn't, issue yourself a developer key in [Issue developer credentials](#issue-developer-credentials) first
5959* `model` is a Claude model name your gateway is configured to route. The example uses `claude-sonnet-4-6`; substitute a name you've configured
6060
61<Tabs>
62 <Tab title="Bash or Zsh">
63 ```bash theme={null}
64 curl -X POST "https://llm-gateway.example.com/v1/messages" \
65 -H "Authorization: Bearer <gateway-key>" \
66 -H "anthropic-version: 2023-06-01" \
67 -H "content-type: application/json" \
68 -d '{"model": "claude-sonnet-4-6", "max_tokens": 1, "messages": [{"role": "user", "content": "."}]}'
69 ```
70 </Tab>
61**Bash or Zsh**
7162
72 <Tab title="PowerShell">
73 ```powershell theme={null}
74 Invoke-RestMethod -Method Post -Uri "https://llm-gateway.example.com/v1/messages" `
75 -Headers @{ "Authorization" = "Bearer <gateway-key>"; "anthropic-version" = "2023-06-01" } `
76 -ContentType "application/json" `
77 -Body '{"model": "claude-sonnet-4-6", "max_tokens": 1, "messages": [{"role": "user", "content": "."}]}'
78 ```
79 </Tab>
80</Tabs>
63```bash theme={null}
64curl -X POST "https://llm-gateway.example.com/v1/messages" \
65 -H "Authorization: Bearer <gateway-key>" \
66 -H "anthropic-version: 2023-06-01" \
67 -H "content-type: application/json" \
68 -d '{"model": "claude-sonnet-4-6", "max_tokens": 1, "messages": [{"role": "user", "content": "."}]}'
69```
8170
71**PowerShell**
72
73```powershell theme={null}
74Invoke-RestMethod -Method Post -Uri "https://llm-gateway.example.com/v1/messages" `
75 -Headers @{ "Authorization" = "Bearer <gateway-key>"; "anthropic-version" = "2023-06-01" } `
76 -ContentType "application/json" `
77 -Body '{"model": "claude-sonnet-4-6", "max_tokens": 1, "messages": [{"role": "user", "content": "."}]}'
78```
79
8280**Checkpoint**: a `200` with a `content` field means the gateway reached the provider with that model name. A `404` means that name isn't routed at the gateway; a `401` from the provider means the gateway's provider credential is wrong.
8381
8482Repeat the request once per Claude model name in your gateway's routing configuration. A name the gateway doesn't route returns `404` to any developer who selects it, so test every name before rollout.
8583
86<Note>
87 Avoid serving the gateway behind a redirect. A redirect can drop the request body or strip the credential header on inference requests, and [model discovery](/docs/en/llm-gateway-protocol#model-discovery) treats any redirect as a failure so the credential cannot leak to a redirect target.
88</Note>
84> [!NOTE]
85>
86> Avoid serving the gateway behind a redirect. A redirect can drop the request body or strip the credential header on inference requests, and [model discovery](/docs/en/llm-gateway-protocol#model-discovery) treats any redirect as a failure so the credential cannot leak to a redirect target.
8987
9088### Issue developer credentials
9189
from line 91
9391
9492Confirm a freshly issued key works against the gateway with the same request as [Confirm the gateway routes your models](#confirm-the-gateway-routes-your-models), replacing `<gateway-key>` with the new `<developer-key>`:
9593
96<Tabs>
97 <Tab title="Bash or Zsh">
98 ```bash theme={null}
99 curl -X POST "https://llm-gateway.example.com/v1/messages" \
100 -H "Authorization: Bearer <developer-key>" \
101 -H "anthropic-version: 2023-06-01" \
102 -H "content-type: application/json" \
103 -d '{"model": "claude-sonnet-4-6", "max_tokens": 1, "messages": [{"role": "user", "content": "."}]}'
104 ```
105 </Tab>
94**Bash or Zsh**
10695
107 <Tab title="PowerShell">
108 ```powershell theme={null}
109 Invoke-RestMethod -Method Post -Uri "https://llm-gateway.example.com/v1/messages" `
110 -Headers @{ "Authorization" = "Bearer <developer-key>"; "anthropic-version" = "2023-06-01" } `
111 -ContentType "application/json" `
112 -Body '{"model": "claude-sonnet-4-6", "max_tokens": 1, "messages": [{"role": "user", "content": "."}]}'
113 ```
114 </Tab>
115</Tabs>
96```bash theme={null}
97curl -X POST "https://llm-gateway.example.com/v1/messages" \
98 -H "Authorization: Bearer <developer-key>" \
99 -H "anthropic-version: 2023-06-01" \
100 -H "content-type: application/json" \
101 -d '{"model": "claude-sonnet-4-6", "max_tokens": 1, "messages": [{"role": "user", "content": "."}]}'
102```
116103
104**PowerShell**
105
106```powershell theme={null}
107Invoke-RestMethod -Method Post -Uri "https://llm-gateway.example.com/v1/messages" `
108 -Headers @{ "Authorization" = "Bearer <developer-key>"; "anthropic-version" = "2023-06-01" } `
109 -ContentType "application/json" `
110 -Body '{"model": "claude-sonnet-4-6", "max_tokens": 1, "messages": [{"role": "user", "content": "."}]}'
111```
112
117113**Checkpoint**: a `200` with a `content` field means the developer key reaches the gateway and the gateway forwards it. A `401` here, when [the previous step](#confirm-the-gateway-routes-your-models) succeeded, means the developer key is wrong or hasn't taken effect at the gateway yet.
118114
119115Issuing one key per developer rather than a shared key is what makes per-developer usage attribution and individual offboarding work. The environment variable that holds the key depends on which header the gateway reads. For a gateway that checks credentials in the `Authorization: Bearer` header, developers set their key in `ANTHROPIC_AUTH_TOKEN`. For a gateway that reads keys from the `x-api-key` header, developers set `ANTHROPIC_API_KEY` instead; the [credential table](/docs/en/llm-gateway-connect#set-the-credential-variable) covers the mapping.
from line 118
122118
123119Run Claude Code through the gateway yourself before distributing anything, using the same configuration the rollout will deliver fleet-wide. Type these directly in a terminal, not in a `.env` or settings file; they last only for this terminal session, so closing it returns your machine to its normal configuration. Use `ANTHROPIC_API_KEY` instead of `ANTHROPIC_AUTH_TOKEN` if your gateway reads the `x-api-key` header:
124120
125<Tabs>
126 <Tab title="Bash or Zsh">
127 ```bash theme={null}
128 export ANTHROPIC_BASE_URL=https://llm-gateway.example.com
129 export ANTHROPIC_AUTH_TOKEN="<developer-key>"
130 ```
131 </Tab>
121**Bash or Zsh**
132122
133 <Tab title="PowerShell">
134 ```powershell theme={null}
135 $env:ANTHROPIC_BASE_URL = "https://llm-gateway.example.com"
136 $env:ANTHROPIC_AUTH_TOKEN = "<developer-key>"
137 ```
138 </Tab>
139</Tabs>
123```bash theme={null}
124export ANTHROPIC_BASE_URL=https://llm-gateway.example.com
125export ANTHROPIC_AUTH_TOKEN="<developer-key>"
126```
140127
128**PowerShell**
129
130```powershell theme={null}
131$env:ANTHROPIC_BASE_URL = "https://llm-gateway.example.com"
132$env:ANTHROPIC_AUTH_TOKEN = "<developer-key>"
133```
134
141135Then send a one-shot prompt through the gateway:
142136
143137```bash theme={null}
from line 220
226220
227221Confirm everything works from a developer machine, not the gateway host, so the test covers the network path developers use. Send a streaming request, which checks the endpoint, streaming pass-through, and model routing at once:
228222
229<Tabs>
230 <Tab title="Bash or Zsh">
231 ```bash theme={null}
232 curl -N -X POST "https://llm-gateway.example.com/v1/messages" \
233 -H "Authorization: Bearer <developer-key>" \
234 -H "anthropic-version: 2023-06-01" \
235 -H "content-type: application/json" \
236 -d '{"model": "claude-sonnet-4-6", "max_tokens": 16, "stream": true, "messages": [{"role": "user", "content": "count to 3"}]}'
237 ```
238 </Tab>
223**Bash or Zsh**
239224
240 <Tab title="PowerShell">
241 ```powershell theme={null}
242 $body = '{"model": "claude-sonnet-4-6", "max_tokens": 16, "stream": true, "messages": [{"role": "user", "content": "count to 3"}]}'
243 $body | curl.exe -N -X POST "https://llm-gateway.example.com/v1/messages" `
244 -H "Authorization: Bearer <developer-key>" `
245 -H "anthropic-version: 2023-06-01" `
246 -H "content-type: application/json" `
247 --data-binary '@-'
248 ```
249 </Tab>
250</Tabs>
225```bash theme={null}
226curl -N -X POST "https://llm-gateway.example.com/v1/messages" \
227 -H "Authorization: Bearer <developer-key>" \
228 -H "anthropic-version: 2023-06-01" \
229 -H "content-type: application/json" \
230 -d '{"model": "claude-sonnet-4-6", "max_tokens": 16, "stream": true, "messages": [{"role": "user", "content": "count to 3"}]}'
231```
232
233**PowerShell**
234
235```powershell theme={null}
236$body = '{"model": "claude-sonnet-4-6", "max_tokens": 16, "stream": true, "messages": [{"role": "user", "content": "count to 3"}]}'
237$body | curl.exe -N -X POST "https://llm-gateway.example.com/v1/messages" `
238 -H "Authorization: Bearer <developer-key>" `
239 -H "anthropic-version: 2023-06-01" `
240 -H "content-type: application/json" `
241 --data-binary '@-'
242```
251243
252244You should see `data:` lines arrive incrementally. The whole response arriving at once after a pause means the gateway is buffering, which stalls Claude Code; a `404` means the model name isn't routed. Repeat per model name.
253245
No line in this hunk matches that.