Skip to main content

The bots crawling Shopify's agentic stack.

Two months of edge logs from a Shopify research storefront. Microsoft Copilot is the only organic agentic verifier polling on its own, hitting /.well-known/ucp from first-party Azure week over week. The common LLM crawlers (GPTBot, ClaudeBot, Bingbot, Googlebot) keep fetching the Shopify-canonical /pages/llms-txt and ignore every newer agent-discovery file.

Available in: Deutsch | Español | Français | हिन्दी | Italiano | Bahasa Melayu | Nederlands | Polski | Português | Türkçe | Українська
Tony Castillo

Tony Castillo

← All research

Key takeaways
  1. Microsoft Copilot is polling for the UCP manifest on its own.
    Microsoft-Commerce-Platform polls /.well-known/ucp roughly weekly from Microsoft Azure.
    Microsoft-Commerce-Platform is Microsoft Copilot's UCP ingestion pipeline.
  2. AI bots aren't using the newer agent-discovery files yet. None of the common LLM crawlers (GPTBot, ClaudeBot, Bingbot, Googlebot) have hit any of the newer agent-discovery paths yet: /llms-full.txt, /agents.md, or /sitemap_agentic_discovery.xml. They all keep fetching the Shopify-canonical /pages/llms-txt instead.
  3. Google-Extended has not visited the storefront once. Google's stated AI-training opt-out bot logged zero hits in the trailing 60 days, on any path. If Gemini is pulling Shopify product data into its training pipeline at all, it isn't doing it under this UA. The Google bots that did show up (Googlebot, GoogleOther) are search and shopping crawlers, not the training-data bot.

Who's there, and what they go for.

Total LLM-bot fetches across the window, alongside a bot-by-endpoint heatmap. GPTBot, ClaudeBot, and Bingbot double-fetch /pages/llms-txt and /a/llms but ignore the new spec files. Only Microsoft-Commerce-Platform polls /.well-known/ucp on its own.

60-day total fetches per operator

All LLM-discovery resources combined
BingbotLLM
93
GPTBotLLM
71
ClaudeBotLLM
28
AmazonbotLLM
13
GoogleOtherLLM
5
OAI-SearchBotLLM
4
Meta-ExternalLLM
3
GooglebotLLM
3
MS-CommerceVRF
3
ApplebotLLM
2
Google-ExtendedLLM
0

Bot × endpoint, 60 days

Empty cells = zero hits. /[locale]/llms-txt aggregates locale-prefixed variants Shopify auto-generates for every enabled region, e.g. /en-gb/pages/llms-txt.
  /pages/llms-txt /a/llms /[locale]/llms-txt /llms.txt /llms-full.txt /agents.md /sitemap_agentic /.well-known/ucp
BingbotLLM 50 43 ······
GPTBotLLM 34 31 6 ·····
ClaudeBotLLM 14 14 ······
AmazonbotLLM 13 ·······
GoogleOtherLLM 1 · 4 ·····
OAI-SearchBotLLM 4 ·······
GooglebotLLM 2 · 1 ·····
Meta-ExternalLLM 2 1 ······
MS-CommerceVRF ······· 3
ApplebotLLM 1 1 ······
Google-ExtendedLLM ········

The one organic agentic verifier we caught: Microsoft Copilot.

Microsoft-Commerce-Platform polled /.well-known/ucp on a roughly weekly cadence from Microsoft's first-party Azure space. The IP, ASN, PTR records, and naming convention all match Microsoft directly.

Microsoft-Commerce -Platform
User-Agent: Microsoft-Commerce-Platform/1.0
IPs
Three IPs in 4.149.255.0/24
ASN
AS8075 · Microsoft · 1st-party Azure
Geo
Washington · Seattle / Redmond
CF colo
SEA
Reverse DNS
Microsoft-managed PTRs
Total reqs
3 · only /.well-known/ucp
Cadence
~weekly
Behavioural evidence
  • Microsoft's first-party Azure ASN (AS8075).
  • Three IPs in contiguous 4.149.255.0/24. Production poller pattern.
  • Touches only /.well-known/ucp. No readiness sweep.
  • Roughly weekly cadence consistent with capability-refresh polling.
  • UA names "Commerce-Platform", matching Microsoft's own product line.

Microsoft Copilot's UCP ingestion pipeline. Microsoft adopted UCP at NRF 2026 and ships UCP in Copilot Checkout with Shopify, PayPal, and Stripe. The weekly poll from first-party Azure is the production capability-refresh, not a one-shot probe.