vision-coordinates changedbuild-with-claude/vision-coordinates
Nearest release: v2.1.312, published under an hour before this site recorded the change. Shown because the two are within 24 hours of each other. Nothing here says the release caused the edit.
Recorded here
Lines+26added
Lines−26removed
From line
6
where the diff opens
First seen
14 Aug 2026
this site's first read of the page
Recorded edits11to this page, all time
The whole hunk
from line 6, old and new numbered
/
from line 6
66
77Claude can locate and label regions of an image (for example, returning bounding boxes for tables, form fields, chart elements, or UI components). This guide covers how Claude resizes images before processing them and how to work with the pixel coordinates it returns, so that boxes and points line up with your original image.
88
9You'll need this for OCR pipelines, form extraction, chart parsing, UI element location, and any task where you act on a specific region of an image. For sending images, supported formats, and per-model resolution limits, see [Vision](https://platform.claude.com/docs/en/build-with-claude/vision).
9You’ll need this for OCR pipelines, form extraction, chart parsing, UI element location, and any task where you act on a specific region of an image. For sending images, supported formats, and per-model resolution limits, see [Vision](https://platform.claude.com/docs/en/build-with-claude/vision).
1010
1111<Note>
12 **Claude works best with absolute pixel coordinates.** Ask for them explicitly in your prompt. For example: *"Return the bounding box of each table as `[x1, y1, x2, y2]` (top-left and bottom-right corners) in pixel coordinates."* Claude does not work well when you ask for normalized coordinates, for example: *"Return bounding box coordinates between `0` and `1000`."* Always ask for pixel coordinates and normalize in your own code if you need to. To get coordinates as machine-readable JSON instead of prose, define a schema with [structured outputs](https://platform.claude.com/docs/en/build-with-claude/structured-outputs), for example an object with an `[x1, y1, x2, y2]` array for each detected element.
12 **Claude works best with absolute pixel coordinates.** Ask for them explicitly in your prompt. For example: *“Return the bounding box of each table as `[x1, y1, x2, y2]` (top-left and bottom-right corners) in pixel coordinates.”* Claude does not work well when you ask for normalized coordinates, for example: *“Return bounding box coordinates between `0` and `1000`.”* Always ask for pixel coordinates and normalize in your own code if you need to. To get coordinates as machine-readable JSON instead of prose, define a schema with [structured outputs](https://platform.claude.com/docs/en/build-with-claude/structured-outputs), for example an object with an `[x1, y1, x2, y2]` array for each detected element.
1313</Note>
1414
15Coordinates follow the standard image convention: the origin `(0, 0)` is the top-left corner of the image, with x increasing to the right and y increasing downward. The coordinates Claude returns are pixel positions in the image Claude processes: your image after Claude resizes it to fit the model's native resolution (see [How Claude resizes and pads images](https://platform.claude.com/docs/en/build-with-claude/vision-coordinates#how-claude-resizes-and-pads-images)). To get coordinates you can use directly, either pre-resize your image so the coordinates map one-to-one onto the image you have (see [Resize your image before uploading](https://platform.claude.com/docs/en/build-with-claude/vision-coordinates#resize-your-image-before-uploading)), or rescale the coordinates Claude returns (see [Rescale coordinates when you cannot pre-resize](https://platform.claude.com/docs/en/build-with-claude/vision-coordinates#rescale-coordinates-when-you-cannot-pre-resize)).
15Coordinates follow the standard image convention: the origin `(0, 0)` is the top-left corner of the image, with x increasing to the right and y increasing downward. The coordinates Claude returns are pixel positions in the image Claude processes: your image after Claude resizes it to fit the model’s native resolution (see [How Claude resizes and pads images](https://platform.claude.com/docs/en/build-with-claude/vision-coordinates#how-claude-resizes-and-pads-images)). To get coordinates you can use directly, either pre-resize your image so the coordinates map one-to-one onto the image you have (see [Resize your image before uploading](https://platform.claude.com/docs/en/build-with-claude/vision-coordinates#resize-your-image-before-uploading)), or rescale the coordinates Claude returns (see [Rescale coordinates when you cannot pre-resize](https://platform.claude.com/docs/en/build-with-claude/vision-coordinates#rescale-coordinates-when-you-cannot-pre-resize)).
1616
1717<Note>
18 Claude's spatial reasoning has limits (see [Limitations](https://platform.claude.com/docs/en/build-with-claude/vision#limitations)). Coordinate accuracy is best when you state the expected coordinate format in your prompt and spot-check results visually before processing at scale. Small elements lose precision when an image is downscaled: for fine targets, crop the region of interest and send the crop (offset returned coordinates by the crop origin), or use a high-resolution-tier model. For [PDF support](https://platform.claude.com/docs/en/build-with-claude/pdf-support), pages are rasterized to images server-side at dimensions you don't control, so the returned coordinates can't be reliably mapped back onto the page. To work with coordinates on PDF content, rasterize the pages to images yourself and use the pre-resize approach.
18 Claude’s spatial reasoning has limits (see [Limitations](https://platform.claude.com/docs/en/build-with-claude/vision#limitations)). Coordinate accuracy is best when you state the expected coordinate format in your prompt and spot-check results visually before processing at scale. Small elements lose precision when an image is downscaled: for fine targets, crop the region of interest and send the crop (offset returned coordinates by the crop origin), or use a high-resolution-tier model. For [PDF support](https://platform.claude.com/docs/en/build-with-claude/pdf-support), pages are rasterized to images server-side at dimensions you don’t control, so the returned coordinates can’t be reliably mapped back onto the page. To work with coordinates on PDF content, rasterize the pages to images yourself and use the pre-resize approach.
1919</Note>
2020
2121## How Claude resizes and pads images
2222
23Claude finds the largest aspect-preserving size that satisfies both of the model's image limits:
23Claude finds the largest aspect-preserving size that satisfies both of the model’s image limits:
2424
25251. **Edge limit:** neither side exceeds the maximum edge length (1,568 px on the standard tier, 2,576 px on the high-resolution tier).
262. **Visual token limit:** the image's token cost `⌈width / 28⌉ × ⌈height / 28⌉` does not exceed the model's visual token budget (1,568 tokens on the standard tier, 4,784 on the high-resolution tier).
262. **Visual token limit:** the image’s token cost `⌈width / 28⌉ × ⌈height / 28⌉` does not exceed the model’s visual token budget (1,568 tokens on the standard tier, 4,784 on the high-resolution tier).
2727
2828See [Resolution and token cost](https://platform.claude.com/docs/en/build-with-claude/vision#evaluate-image-size) for which models are in which tier.
2929
from line 32
3232The token limit can also trigger a resize when neither side exceeds the edge limit. Overlooking this is the most common cause of misaligned coordinates. For example, an A4 page scanned at 130 DPI is 1075×1520 pixels: both sides are under 1,568 px, but it costs `39 × 55 = 2145` visual tokens, so Claude resizes it to 924×1307.
3333
3434<Note>
35 This example assumes a model on the standard resolution tier. A high-resolution-tier model doesn't resize the same scan: 2,145 tokens is within its 4,784-token budget, so the coordinates it returns map directly onto the 1075×1520 original. Model tiers are listed in [Resolution and token cost](https://platform.claude.com/docs/en/build-with-claude/vision#evaluate-image-size).
35 This example assumes a model on the standard resolution tier. A high-resolution-tier model doesn’t resize the same scan: 2,145 tokens is within its 4,784-token budget, so the coordinates it returns map directly onto the 1075×1520 original. Model tiers are listed in [Resolution and token cost](https://platform.claude.com/docs/en/build-with-claude/vision#evaluate-image-size).
3636</Note>
3737
3838Claude then pads every image, resized or not, up to the next multiple of 28 pixels on the bottom and right edges (924×1307 becomes 924×1316 in the example). The padding contains no content: Claude perceives the padded image, but the page content only ever occupies the un-padded resized region. **Always normalize or rescale by the resized dimensions, not the padded dimensions**; dividing by the padded dimensions scales every coordinate by a small amount.
from line 436
436436 ```
437437</CodeGroup>
438438
4391. Resize the image to the dimensions returned by the resize helper. If the image already fits within the model's limits, the helper returns its dimensions unchanged and no resize is needed.
4402. [Send the resized image](https://platform.claude.com/docs/en/build-with-claude/vision#send-images-to-claude) to the API. Don't pad it yourself. Claude handles padding, and padding doesn't shift the coordinate origin.
4413. In your prompt, ask explicitly for pixel coordinates. For example: *"Return the click point for the Submit button as `[x, y]` in pixel coordinates."*
4424. Use the returned coordinates directly against the image you sent. If you need normalized coordinates, divide by the dimensions of the image you sent, not by the original image's dimensions and not by the padded dimensions.
4391. Resize the image to the dimensions returned by the resize helper. If the image already fits within the model’s limits, the helper returns its dimensions unchanged and no resize is needed.
4402. [Send the resized image](https://platform.claude.com/docs/en/build-with-claude/vision#send-images-to-claude) to the API. Don’t pad it yourself. Claude handles padding, and padding doesn’t shift the coordinate origin.
4413. In your prompt, ask explicitly for pixel coordinates. For example: *“Return the click point for the Submit button as `[x, y]` in pixel coordinates.”*
4424. Use the returned coordinates directly against the image you sent. If you need normalized coordinates, divide by the dimensions of the image you sent, not by the original image’s dimensions and not by the padded dimensions.
443443
444444<Note>
445 The [Token counting](https://platform.claude.com/docs/en/build-with-claude/token-counting) endpoint estimates an image's token cost from its dimensions without fully processing it, so a successful count doesn't mean the image is within the Messages API's [request limits](https://platform.claude.com/docs/en/build-with-claude/vision#request-limits). An image can count successfully and still be rejected when you send it.
445 The [Token counting](https://platform.claude.com/docs/en/build-with-claude/token-counting) endpoint estimates an image’s token cost from its dimensions without fully processing it, so a successful count doesn’t mean the image is within the Messages API’s [request limits](https://platform.claude.com/docs/en/build-with-claude/vision#request-limits). An image can count successfully and still be rejected when you send it.
446446</Note>
447447
448448## Turn resizing into an error with `transformations`
from line 457
457457}
458458```
459459
460A request whose marked image (any block that sets `"oversized_image": "error"`) would be resized is rejected with a 400 `invalid_request_error` naming the image's dimensions and the largest dimensions that fit. Whether an image triggers the rejection depends on the [limits of every model the request names](https://platform.claude.com/docs/en/build-with-claude/vision-coordinates#how-claude-resizes-and-pads-images): the following 1920×1080 example is rejected by a standard-tier model but fits within the high-resolution tier:
460A request whose marked image (any block that sets `"oversized_image": "error"`) would be resized is rejected with a 400 `invalid_request_error` naming the image’s dimensions and the largest dimensions that fit. Whether an image triggers the rejection depends on the [limits of every model the request names](https://platform.claude.com/docs/en/build-with-claude/vision-coordinates#how-claude-resizes-and-pads-images): the following 1920×1080 example is rejected by a standard-tier model but fits within the high-resolution tier:
461461
462462```text wrap
463463messages.0.content.0: image dimensions 1920x1080 exceed the maximum image size of a model named on this request and would be downsized to 1456x819; scale the image to at most 1456x819 or set the image's oversized_image setting to "downsize"
464464```
465465
466Rescale to the reported target and resend: the target is the largest size, at your image's aspect ratio, that every model named on the request accepts. How marked images interact with the [server-side fallback beta](https://platform.claude.com/docs/en/build-with-claude/refusals-and-fallback#server-side-fallback) is described with that feature; in every mode, a marked image is never served resized.
466Rescale to the reported target and resend: the target is the largest size, at your image’s aspect ratio, that every model named on the request accepts. How marked images interact with the [server-side fallback beta](https://platform.claude.com/docs/en/build-with-claude/refusals-and-fallback#server-side-fallback) is described with that feature; in every mode, a marked image is never served resized.
467467
468Each image has its own setting. `"oversized_image": "downsize"` (the default when the field is omitted) keeps automatic resizing as described on this page. Each image block is checked only against its own setting, so one request can mix images whose dimensions matter (a screenshot you'll click) with images where resizing is harmless (a logo). What the setting does and does not change:
468Each image has its own setting. `"oversized_image": "downsize"` (the default when the field is omitted) keeps automatic resizing as described on this page. Each image block is checked only against its own setting, so one request can mix images whose dimensions matter (a screenshot you’ll click) with images where resizing is harmless (a logo). What the setting does and does not change:
469469
470470* [Padding](https://platform.claude.com/docs/en/build-with-claude/vision-coordinates#how-claude-resizes-and-pads-images) (which never discards content), format conversion, and orientation correction proceed as usual.
471471* The hard limits (8,000 px on the longest side, and the stricter per-image limit on [many-image requests](https://platform.claude.com/docs/en/build-with-claude/vision#request-limits)) are separate rejections; this setting never lets an image past them.
472* Images supplied by URL or file ID are checked once their bytes have been fetched; those rejections carry the same message without the leading position, so they don't identify which image failed; only embedded base64 images are named by position in the error.
473* [PDF pages](https://platform.claude.com/docs/en/build-with-claude/pdf-support) are rasterized server-side at dimensions you don't control; the `document` block does not accept the field (an image block nested inside a document's content accepts it like any other).
474* A marked image whose dimensions cannot be determined is rejected rather than passed through: that rejection reports that the image's source dimensions could not be determined, not the resize message quoted earlier. No image that sets `"error"` reaches the model resized.
472* Images supplied by URL or file ID are checked once their bytes have been fetched; those rejections carry the same message without the leading position, so they don’t identify which image failed; only embedded base64 images are named by position in the error.
473* [PDF pages](https://platform.claude.com/docs/en/build-with-claude/pdf-support) are rasterized server-side at dimensions you don’t control; the `document` block does not accept the field (an image block nested inside a document’s content accepts it like any other).
474* A marked image whose dimensions cannot be determined is rejected rather than passed through: that rejection reports that the image’s source dimensions could not be determined, not the resize message quoted earlier. No image that sets `"error"` reaches the model resized.
475475
476476The [Token counting](https://platform.claude.com/docs/en/build-with-claude/token-counting) endpoint honors `transformations` too, rejecting an embedded image exactly as the Messages API would, so you can check whether an embedded image fits without being resized, before running inference. Counting rejects images supplied by URL or file ID rather than fetching them, so a marked image from those sources is checked only at Messages time.
477477
478478## Rescale coordinates when you cannot pre-resize
479479
480If you cannot pre-resize (for example, when the image comes from an upstream system you can't modify), use the resize helper from [Resize your image before uploading](https://platform.claude.com/docs/en/build-with-claude/vision-coordinates#resize-your-image-before-uploading) to recover the dimensions of the image Claude processed, then map the coordinates Claude returns into normalized coordinates or back onto your original image. Unless an image opts into an [oversized-image error](https://platform.claude.com/docs/en/build-with-claude/vision-coordinates#oversized-image-error) instead, Claude resizes oversized images rather than rejecting them, up to the API's [request limits](https://platform.claude.com/docs/en/build-with-claude/vision#request-limits). Beyond those limits the request fails with a validation error instead. Pass the tier limits that match the model you called: the wrong tier's limits recover the wrong resized dimensions and silently shift every coordinate. This approach requires knowing the pixel dimensions of the image you uploaded, so it does not apply to PDF uploads.
480If you cannot pre-resize (for example, when the image comes from an upstream system you can’t modify), use the resize helper from [Resize your image before uploading](https://platform.claude.com/docs/en/build-with-claude/vision-coordinates#resize-your-image-before-uploading) to recover the dimensions of the image Claude processed, then map the coordinates Claude returns into normalized coordinates or back onto your original image. Unless an image opts into an [oversized-image error](https://platform.claude.com/docs/en/build-with-claude/vision-coordinates#oversized-image-error) instead, Claude resizes oversized images rather than rejecting them, up to the API’s [request limits](https://platform.claude.com/docs/en/build-with-claude/vision#request-limits). Beyond those limits the request fails with a validation error instead. Pass the tier limits that match the model you called: the wrong tier’s limits recover the wrong resized dimensions and silently shift every coordinate. This approach requires knowing the pixel dimensions of the image you uploaded, so it does not apply to PDF uploads.
481481
482Screenshots and zoom images that you return to the [computer use](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool#handle-coordinate-scaling-for-higher-resolutions) and [browser use](https://platform.claude.com/docs/en/agents-and-tools/tool-use/browser-use-tool#targets-and-coordinates) toolsets are an exception to automatic resizing. The API rejects a `tool_result` image that exceeds the model's limits with a validation error instead of resizing it. Resize those images in your application before returning them, then scale the coordinates Claude returns back to your screen's dimensions.
482Screenshots and zoom images that you return to the [computer use](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool#handle-coordinate-scaling-for-higher-resolutions) and [browser use](https://platform.claude.com/docs/en/agents-and-tools/tool-use/browser-use-tool#targets-and-coordinates) toolsets are an exception to automatic resizing. The API rejects a `tool_result` image that exceeds the model’s limits with a validation error instead of resizing it. Resize those images in your application before returning them, then scale the coordinates Claude returns back to your screen’s dimensions.
483483
484484<CodeGroup exclude="shell">
485485 <CodeGroupItem language="cURL">
from line 649
649649 ```
650650</CodeGroup>
651651
652Padding is applied only to the bottom and right edges, so the origin doesn't shift and a per-axis linear rescale is sufficient. Clamp returned coordinates to the resized dimensions before rescaling, so a point slightly outside the image can't map outside your original.
652Padding is applied only to the bottom and right edges, so the origin doesn’t shift and a per-axis linear rescale is sufficient. Clamp returned coordinates to the resized dimensions before rescaling, so a point slightly outside the image can’t map outside your original.
653653
654The relative coordinates multiply against whatever surface you act on: the original image, a full-resolution scan, or a screen. When you act on a screen and screenshot pixels differ from logical coordinates (HiDPI displays), also divide by the display scale factor. The [Computer use tool's scaling guidance](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool#handle-coordinate-scaling-for-higher-resolutions) covers that pattern.
654The relative coordinates multiply against whatever surface you act on: the original image, a full-resolution scan, or a screen. When you act on a screen and screenshot pixels differ from logical coordinates (HiDPI displays), also divide by the display scale factor. The [Computer use tool’s scaling guidance](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool#handle-coordinate-scaling-for-higher-resolutions) covers that pattern.
655655
656656## Next steps
657657
658658<CardGroup cols={2}>
659659 <Card title="Agent Skills" icon="stack" href="https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview">
660 Agent Skills are modular capabilities that extend Claude's functionality. Each Skill packages instructions, metadata, and optional resources (scripts, templates) that Claude uses automatically when relevant.
660 Agent Skills are modular capabilities that extend Claude’s functionality. Each Skill packages instructions, metadata, and optional resources (scripts, templates) that Claude uses automatically when relevant.
661661 </Card>
662662
663663 <Card title="Computer use tool" icon="computer" href="https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool">
No line in this hunk matches that.