Skip to main content

OpenAI is sending its bots out to fetch far fewer pages.We validated the signal. Now we're hypothesizing the reason. Your audience, meanwhile, didn't go anywhere.

OpenAI's crawlers are fetching far fewer pages, and if your AI bot numbers collapsed around August 20, 2026, it wasn't your site. Across August 19-21, page fetches from OpenAI's recognized crawlers dropped by roughly three quarters on every property we measured, and stayed down. Over the same days, the people arriving from ChatGPT moved by single digits. Tools that read your logs for AI bot activity showed the whole drop, and none of it is a penalty on your site. This covers what changed, why it happened, how we ruled out a renamed crawler, and how to read your own numbers now that fetches and arrivals have separated.

Two clay pathways leading to a storefront door: on the left, three robot couriers toppled on their sides with their papers dropped around them, and on the right, an unbroken queue of people still walking in
The short version
  1. Crawler fetches dropped about 74%. The people arriving from ChatGPT moved by single digits. Same days, same properties. Across August 19-21, 2026, page fetches from OpenAI's recognized crawlers fell to roughly a quarter of what they had been and stayed there for the eighteen days we watched. Arrivals from ChatGPT over those same days moved by an amount this series moves anyway, swinging 8.7% on average between ordinary periods earlier in the summer. The two numbers had tracked each other for months, and in one weekend they came apart.
  2. Bot-fetch tools showed the cliff. Prompt trackers and arrival tools didn't, and couldn't. A tool reading your CDN's bot analytics or your server logs counts crawler requests, so it caught the whole drop. A prompt tracker asks assistants questions and records whether you get named, so a change in fetching can't reach it. An arrivals report counts people, and those people kept coming. Three products, three quantities, one of them moved.
  3. The drop is real. It isn't a reporting glitch. It reproduces on separate properties running different codebases, it shows up in raw edge logs with no classifier in the path, and the daily log volume itself never budged. A second property, measured independently, shows a fall of about 75%.
  4. It isn't the scraped-mirror cleanup from the week before either. That was a separate event on August 14, done by the 16th. Fetches touching mirror copies were under 2% of the total on both sides of the change, and their share went up slightly afterward rather than down. Two unrelated things landed within a week of each other.
  5. The cause looks like less query fanout, not harder caching. Fetches per page fell 47% and the number of distinct pages fetched fell 53%, together. Caching harder would shrink the frequency while leaving the set of pages roughly intact. Reaching fewer sources per question shrinks both, which is what happened. Trade press attributes the change to a cut in ChatGPT Search's query fanout, and there is no first-party announcement, so the named cause is somebody else's reporting rather than an established fact.
  6. A drop in a log-based bot report isn't a penalty. Tools that read your logs for AI crawler activity, Profound and Searchable among them, count how often OpenAI's crawlers asked for your pages. The fall they showed in August reflects a change in how often OpenAI asks, not a judgment about your site. It landed on every property we measured inside the same three days, and a far larger site was reported down 86% the same week.
  7. Nothing was renamed, and that was worth checking before printing any of this. A relabeled crawler would have produced identical evidence, so the check had to avoid names. Counting requests from OpenAI's published crawler IP ranges instead of by user agent, the drop is still there at 63%, with no unfamiliar user agent among them. The traffic genuinely isn't arriving.

You opened your AI bot report at the start of September and there’s a cliff in it. Around August 20 a line that had climbed all year drops by three quarters and never comes back. Maybe it reached you through your CDN’s bot analytics, maybe through a log-based visibility tool, maybe in a dashboard your agency sends over.

You’re not alone, and it isn’t your site. The same cliff landed on every property we measured, on the same three days, and a second one we checked independently shows it too.

Your dashboard is only showing you half of what happened, though. Over those same days the number of people arriving from ChatGPT moved by single digits. Nobody stopped coming. Two numbers that had tracked each other for months came apart in a weekend, and the tools counting crawler activity show you only the one that fell.

The rest of this is what changed, why it happened, and how to read your numbers now that the two have separated.

What changed

Across August 19-21, 2026, page fetches from OpenAI’s recognized crawlers dropped to roughly a quarter of what they had been and stayed there. Every property we analyze shows it. A second one, measured independently from raw edge logs, shows a fall of about 75%.

The fall took two days, and the level held for the eighteen that followed
Crawler fetches from OpenAI per day, indexed to the level before August 20, 2026 set to 100.
0 25 50 75 100 Aug 18: 108 against 100 before August 20 Aug 18 108 Aug 19: 106 against 100 before August 20 Aug 19 106 Aug 20: 39 against 100 before August 20 Aug 20 39 Aug 21: 27 against 100 before August 20 Aug 21 27 Aug 22-Sep 8: 26 against 100 before August 20 Aug 22-Sep 8 26

The last point is the daily average across the rest of the period rather than a single day, because the figure barely moved once it landed.

The whole descent took two days, and the eighteen days after it sit flat at the new level. An outage would have recovered by then.

What didn’t change

The people arriving from ChatGPT kept arriving. Over the same days that took three quarters off the crawler number, arrivals moved by single digits.

What each number did after August 20, 2026
Each measure indexed to its own level before that date, set to 100. Same properties, same days.
0 25 50 75 100 Level before August 20 Crawler fetches from OpenAI Crawler fetches from OpenAI: fell to about a quarter of the level before August 20 26 People arriving from ChatGPT People arriving from ChatGPT: two independent measurements put it between 92 and 96 People arriving from ChatGPT: at least 92 against a level of 100 before August 20 92 to 96

The arrivals figure is measured twice by different routes and the two runs land a few points apart, so it prints as the range they span rather than a single number. The crawler figure is the sustained level over the eighteen days that followed.

This series is noisy. Across a comparable stretch earlier in the summer with no known event in it, the same measurement swung by 8.7% on average between ordinary periods, and reached 18.8%. A single-digit move is the kind of thing it does anyway. A 74% move is not.

The crawler number fell off a cliff while the arrivals number did something this series does most months. Those arrivals are people. Every one of them read an answer, tapped a link inside it, and landed on the site, and they’re the reason you care about getting cited at all.

Why we think it’s happening

Two things fell together: the number of times each page got fetched, down 47%, and the number of distinct pages fetched at all, down 53%.

That pairing rules out the first explanation most people reach for. If a system were caching your pages harder, rechecking copies it already holds, you would expect it to fetch the same pages less often while still touching roughly the same set of pages. The set shrank as much as the frequency did. Caching alone doesn’t produce that.

Reducing query fanout does. When you ask ChatGPT Search something, the question gets expanded into several sub-queries, each of which goes and pulls its own sources. Cut the number of sub-queries and you reach fewer sources, each of them less often, which is both of the numbers above moving together.

Reporting elsewhere in the trade press attributes the August change to exactly that, a reduction in ChatGPT Search’s query fanout. We have not seen a first-party announcement from OpenAI, so treat the named cause as someone else’s reporting rather than an established fact. What we can show you is the measured shape, and it fits.

The same reporting puts Reddit’s drop at 86%. Set against roughly 73% and 57% on the two properties we measured, it suggests this landed broadly rather than on any particular kind of site.

How we checked the traffic was OpenAI’s

A crawler tells your server which software it is. That name is the only thing separating OpenAI’s requests from anybody else’s, in your logs and in ours, and it’s the weak point under everything above.

A rename would have produced identical evidence. If OpenAI had quietly relabeled a crawler, or moved this work to an agent nobody maps back to OpenAI, you would see a global change, inside one 48-hour window, on every property, across different codebases, plainly visible in raw logs. Exactly what we saw.

The check therefore had to avoid names altogether. OpenAI publishes the IP ranges its crawlers operate from. Counting requests from those ranges instead of by user agent, the drop is still there, at 63%, and there was no unfamiliar user agent hiding inside them. Nothing was renamed. The traffic genuinely isn’t arriving.

That was the one alternative that would have forced a retraction, and it’s dead.

What else it isn’t

Two more explanations came up during the analysis and both got killed. Both are in here because either one would have made a tidier story than the truth did, and because you may be reaching for one of them about your own data.

  • A reporting glitch. The drop is there in raw sanitized logs with no classifier anywhere in the path. The properties involved run different codebases, and one of them had nothing deployed across the whole window. The count of raw log files per day never moved.
  • A code change on our side. One deployment landed at a very convenient hour on August 19. Somebody raised it as the likely cause and then took it back, because the rows it touches can’t carry the value this metric counts. Convenient timing isn’t evidence.
  • A scraped-mirror cleanup. Copies of these sites had been scraped and republished elsewhere, and that got cleaned up around the same time. It peaked on August 14 and was done by the 16th. Fetches touching mirrored paths were under 2% of the total on both sides of the change, and their share went up slightly afterward rather than down.

Two unrelated things landed within a week of each other. Any single explanation that covers both is wrong before you start.

Your AI tool counts one of three different things

Before you can read your own numbers you need to know which of them you’re holding. The AI measurement market sells three different products and describes all three in roughly the same language. They watch different things.

What your tool watches How it gets the number What it showed in August 2026
Prompt tracking, or share of voice Puts a set of questions to the assistants on a schedule and records whether your brand gets named Nothing, by construction. It never counts crawler requests
AI bot activity, from CDN bot analytics or server logs Counts requests from AI crawlers, identified by the name the software gives The whole drop of about three quarters
Arrivals from assistants Counts the people who tapped a link in an answer and landed on your site An ordinary line

If you’re watching this through Profound, Searchable, or another tool that reads your logs for AI bot activity, the fall in front of you isn’t a penalty. Nothing demoted your site, and nothing you did caused it. Those tools count how often OpenAI’s crawlers asked for your pages, and OpenAI changed how often it asks. The same drop landed on every property we measured, inside the same three days, and a far larger site was reported down 86% that week. A system-wide change in fetching behavior produces exactly this shape in a log-based report, and it says nothing about your standing.

A prompt tracker can’t show you this event. A change in how often your pages get fetched has no path into a number built out of answers. That’s a statement about how the instrument works rather than a finding: we didn’t measure answer-side visibility in August, and this piece doesn’t claim it held or fell.

The tools that did show it were reading crawler activity, and they reported the number correctly. The trouble starts with what people take it to mean. Tell a store owner their AI citations fell 74% in August and you’ve told them something true about a metric and false about their customers.

How to read your new numbers

  • Pull the same dates from a report that counts people instead of requests. A step of roughly three quarters in the week of August 20, 2026 sitting next to an ordinary arrivals line is this event, and it isn’t lost audience.
  • Ask any tool that says citations which quantity it means. Fetches and arrivals both get called that, and on the same site in the same month they differ by orders of magnitude.
  • Treat August 20 as a break in the series rather than a decline. Comparing September against July on a fetch count now measures two different behaviors, so your year-over-year and month-over-month numbers on that metric will stay wrong until a full year has passed.
  • Keep the bot dashboard for what it’s good at, which is telling you whether your pages are reachable and getting read at all. Judge the channel itself on arrivals, and on what those arrivals buy.

August 2026 works as a test of whatever AI measurement you rely on. Whichever tool showed you a collapse that month was reporting on crawler behavior, and now you know what the ones next to it should have been showing you instead.

FAQs

My AI bot dashboard dropped 70% in August. Did I lose visibility?

Not on the evidence we have. The people arriving at the properties we analyze from ChatGPT moved by single digits across the same change, in a series that swings 8.7% on average between ordinary periods anyway. A bot dashboard counts crawler requests rather than customers, so a fall in it tells you something about crawling. Check an arrivals report for the same dates before you treat it as lost audience.

Did OpenAI stop crawling the web in August 2026?

It is fetching far fewer pages, which is not the same as stopping. Requests carrying the names of OpenAI's recognized crawlers fell by roughly three quarters across August 19-21, 2026 and stayed down. Because a renamed crawler would look identical, the drop was rechecked by counting requests from OpenAI's published crawler IP ranges instead of by name: it is still there, at 63%, with no unfamiliar user agent among them. The traffic genuinely isn't arriving.

Does this affect my prompt tracking or share-of-voice tool?

Not by construction. Those tools put a set of questions to the assistants on a schedule and record whether your brand gets named in the answers, which never involves counting crawler requests, so a change in fetching can't show up in one. We didn't measure whether answer-side visibility moved in August and we're not claiming it did or didn't. The point is that the two products measure different things, and only one of them had a cliff.

What's the difference between a crawler fetch and a referral?

A crawler fetch is an AI company's software asking your server for a page, with nobody involved, and one page can get fetched many times a month. A referral is a person reading an answer, tapping a link in it, and landing on your site. The first tells you how often machines read your pages. Only the second counts customers.

How do I tell which quantity my tool counts?

Look at what the number does around August 20, 2026. A tool counting crawler fetches shows a step down of roughly three quarters that week. A tool counting human arrivals shows an ordinary line. If your tool calls its number citations, ask which of the two it means, because the word gets used for both and on the same site in the same month they differ by orders of magnitude.

Could this be a change in how the crawler identifies itself rather than less crawling?

That was the obvious alternative and it has been ruled out. A rename predicts everything we saw, so the check had to avoid names: counting requests from OpenAI's published crawler IP ranges rather than by user agent, the drop is still present at 63%, and no unfamiliar user agent appears inside those ranges. Nothing was relabeled.

Is my site being penalized or demoted by OpenAI?

Nothing in this event is a judgment about any individual site. The drop landed on every property we measured inside the same three days, on different codebases, and a far larger site was reported down 86% the same week. What changed is how often OpenAI's crawlers request pages across the board. A tool that reads your logs for bot activity, such as Profound or Searchable, is correctly reporting that change, and it carries no information about your standing or your rankings.

Why did OpenAI fetch fewer pages?

The measurements point at reaching fewer sources per question rather than caching pages harder. Fetches per page fell 47% and the number of distinct pages fetched fell 53%, and they fell together. Caching harder would cut how often the same pages get rechecked while leaving the set of pages roughly intact. Trade press attributes the change to OpenAI reducing ChatGPT Search's query fanout, which fits that shape, though there is no first-party announcement, so treat the named cause as reporting rather than settled fact.