Field scan · 2026-09-29
E-commerce
A first-hand scan of 16 sites and 12 repositories in e-commerce, measured on 2026-09-29. 3 of 16 sites name an AI crawler in robots.txt, 8 challenged or refused a scripted request, and 0 answered with a payment demand.
16 sites · 12 repositories · 9 findings · Markdown
Figures
Counts are out of every site scanned. A site that could not be read counts as not having the feature, so these are floors. The limits of the scan say which sites could not be read.
Findings
Measured means read first-hand, with the evidence kept. Inferred means a conclusion drawn from what was measured, and never above medium confidence.
- Measured · highThe first surface in four sectors and 55 sites where an agent can find a machine-readable way to buy. Allbirds, a Shopify storefront, serves a Universal Commerce Protocol manifest at /.well-known/ucp. It declares cart, checkout, order and catalog capabilities, a live endpoint, and payment handlers. Asked without any credential what it offers, the endpoint listed 13 tools, among them create_checkout and complete_checkout. No purchase was attempted.
- Inferred · mediumThe manifest looks like a platform default, not one merchant's project. Of three further Shopify storefronts checked for the same file, Skims and Brooklinen serve it at nearly the same size and version, and Gymshark does not. If so, the agent-ready part of e-commerce is whatever runs on Shopify, and merchants got it without building it.
- Measured · highAgent checkout is arriving on card rails. The manifest's payment handlers are Google Pay, Shop Pay and card. No site in this scan returned HTTP 402 or an x402 challenge. Across four sectors the only 402 seen named no price, and no x402 challenge has been seen at all. The way in for agents is a protocol wrapped around existing payments, not a new way to pay.
- Measured · highRetail is split. Walmart, Target, Costco, Instacart, Allbirds, Shopify, Stripe, PayPal served all four user agents the page. Amazon answered every one with a script-only interstitial, Wayfair and DoorDash with a challenge, and The Home Depot with a refusal. Best Buy returned no HTTP response to any request, robots.txt included.
- Measured · higheBay states its position on agents in words. Its robots.txt carries a Robots and Agent Policy in comments: checkouts are strictly for human users, and buy-for-me agents, LLM-driven bots and any flow that places orders without human review are strictly prohibited. Approved integrations are sent to its API. It is the only site in this scan to address buying agents by name, and it does so in a place no crawler is obliged to read.
- Measured · highAmazon's robots.txt blocks 22 of the 27 tracked AI crawler tokens outright, the most in the sector. Most retailers name none: 11 of 14 readable robots.txt files have no AI rule. Retail relies on bot defences at the edge, where news publishers wrote policy.
- Measured · highEtsy publishes an llms.txt titled as its official reference for AI assistants, and answers GPTBot, ClaudeBot and PerplexityBot with a challenge at the edge while serving a Chrome user agent. 6 of 14 sites publish llms.txt, against none in news. E-commerce wants to be described to agents and has not decided to let their crawlers in.
- Measured · highCommerce platforms are being written with agents at more than twice the rate measured in developer tooling (4.8%) and news publishing (3.3%). Across eight platforms and 6,255 commits in 90 days, 807 (12.9%) carry an AI-agent co-author trailer. Saleor leads at 51 of 131 (39%), then WooCommerce at 331 of 1,435 (23%) and Spree at 331 of 1,748 (19%). Magento has none in 948 commits.
- Measured · highOf the four agent-payment protocols, x402 is the most active and the only one with agent co-authored commits: 31 of 287 (11%). The Universal Commerce Protocol has 88 commits and none co-authored. The Agentic Commerce Protocol has one commit in the window and AP2 has none. The protocol with the most deployment found in this scan is not the one with the most development.
Sites scanned
AliExpress, Allbirds (Shopify storefront), Amazon, Best Buy, Costco, DoorDash, eBay, Etsy, The Home Depot, Instacart, PayPal, Shopify, Stripe, Target, Walmart, Wayfair.
Repositories scanned
Agentic Commerce Protocol, Agent Payments Protocol (AP2), Magento, Medusa, PrestaShop, Saleor, Shopify Hydrogen, Spree, Universal Commerce Protocol, Vendure, WooCommerce, x402.
Method
Web: for each of 16 sites, fetched /robots.txt and /llms.txt once with a Chrome user agent, the home page once each with Chrome, GPTBot, ClaudeBot and PerplexityBot user agents, and four well-known paths where a site may publish a manifest for agents (/.well-known/ucp, agent-card.json, agent.json, mcp.json), 1.5 seconds apart, from one US address using a scripted HTTP client (no JavaScript, no cookies). Requests that returned no HTTP response or a body that could not be decoded were repeated once with a second client (curl). Each response was classed as served, challenged, refused, payment required, or reset; a success status carrying only a script-only interstitial counts as challenged. robots.txt was parsed into user-agent groups and checked against 27 tracked AI crawler tokens, and its comments were read for statements addressed to agents. Where a commerce manifest was found, the scanner read it, sent one read-only tools/list call with no credential to the endpoint it declares, and requested the same path on three other storefronts of the same platform. No cart, checkout or order was created on any site. Repositories: for each of 12 repositories, read every commit on the default branch from 2026-06-30 to 2026-09-28 through the GitHub commits API; a commit counts as AI co-authored when a Co-authored-by trailer carries an agent's name or an address an agent signs with, or the message carries an agent's generated-with marker. A person's employer domain does not count.
Limits of this scan
Read these before quoting a figure.
- The web client sent a Chrome user agent string but is not a browser: it runs no JavaScript and has a different TLS fingerprint. Retail bot defences are tuned to exactly this, so 'challenged' and 'refused' describe what a script met, not what a shopper meets.
- AI user agents were sent from an ordinary address, not from the operators' published ranges, and unsigned. A site that verifies crawlers by address or signature may treat the real crawler differently.
- Only the home page was requested. Product, search and checkout pages may be treated differently, and those are the pages a shopping agent uses.
- A commerce manifest shows that a way to transact is declared and that its endpoint answers. Whether an agent can complete a purchase through it was not tested, on purpose.
- One storefront was in the main scan and three more were checked for one file. Four storefronts cannot establish what a platform does by default, which is why that finding is an inference.
- The well-known paths checked are the ones this scanner knows. A site may publish an agent manifest elsewhere.
- bestbuy.com returned no HTTP response to any request, including robots.txt, and doordash.com answered every request with a challenge. Their robots.txt, llms.txt and manifests are unmeasured, not absent.
- aliexpress.com gave inconsistent readings across requests and is recorded at medium confidence.
- robots.txt comments are read as statements of policy. They bind no one, and a policy stated only there may differ from the site's terms of service.
- AP2 has no commits in the window, so no share is recorded for it. The Agentic Commerce Protocol has one, and Shopify Hydrogen 34; their shares are at medium confidence where the count is below 30.
- coinbase/x402 is now a fork. The project is developed at x402-foundation/x402, which is the repository scanned.
- Co-author trailers are voluntary and undercount agent-written code where contributors strip them.
- One reading per site on one day, with a second reading only where the first failed.
How to cite
Overwing Atlas. "E-commerce: field scan of 16 sites and 12 repositories." Measured 2026-09-29. https://overwing.ai/atlas/scans/ecommerce-2026-09
Quote a figure with its date and its denominator. These are single readings on one day, and policies change. Headline findings are free to quote with attribution. Per-target rows are in the sector-scans dataset, under the Atlas data license.