← Blog · October 16, 2026
Cloudflare changed its AI bot defaults on September 15, 2026. Here is how to check if your store turns away ChatGPT, Claude and Perplexity, and how to fix it in five minutes.
You can do everything right on your store and still be invisible to AI assistants. The reason often sits one layer above your store: the CDN or firewall in front of it. If you use Cloudflare, check one setting this week.
On July 1, 2026, Cloudflare replaced its single "Block AI bots" switch with controls by behavior. Its official changelog splits AI crawlers into three groups:
For each group you can block on all pages, block only on pages that show ads, or allow.
The part that matters: starting September 15, 2026, new domains that join Cloudflare get new defaults. Training and Agent bots are blocked on pages that display ads. Search stays allowed. Engadget reported that free plan accounts also move to these defaults unless they opted out before the deadline.
Many stores show ads or ad tags without thinking of themselves as "ad supported". A retargeting pixel, an affiliate widget or a display banner can be enough for a page to count. The changelog does not explain how Cloudflare detects ad pages, so you cannot assume your store is safe.
The Agent group is the one that hurts. When someone asks ChatGPT "which store sells this in my size", the assistant may open your product page live to read price and stock. If Cloudflare answers with a challenge or a 403, the assistant moves on to a store it can read.
There is a second trap. Cloudflare says multi-purpose crawlers that mix Search and Training are affected by the block on Training. So a crawler you want for discovery can get caught by a rule meant for training.
And it is silent. Your analytics never see the blocked request. No error reaches your inbox. Your store looks fine in your browser, because you are a human.
Your robots.txt only shows what you ask bots to do. The firewall decides what actually happens. Test the firewall. From a terminal, request a product page with the user agent of each bot and look at the status code:
curl -s -o /dev/null -w "%{http_code}\n" -A "Mozilla/5.0 (compatible; GPTBot/1.1; +https://openai.com/gptbot)" https://yourstore.com/products/your-product
curl -s -o /dev/null -w "%{http_code}\n" -A "Mozilla/5.0 (compatible; ChatGPT-User/1.0; +https://openai.com/bot)" https://yourstore.com/products/your-product
curl -s -o /dev/null -w "%{http_code}\n" -A "Mozilla/5.0 (compatible; ClaudeBot/1.0; +claudebot@anthropic.com)" https://yourstore.com/products/your-product
curl -s -o /dev/null -w "%{http_code}\n" -A "Mozilla/5.0 (compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)" https://yourstore.com/products/your-productA 200 means the page loaded. A 403, 429 or 503 means something in front of your store turned the bot away. Repeat the test on your homepage and on one collection page.
One caveat. Cloudflare can also verify real bots by IP, so a spoofed user agent from your laptop is not a perfect copy of the real crawler. A 200 here is a good sign. A 403 is a clear red flag.
If you do not want to touch a terminal, use the free AI bot blocker checker. It reads your robots.txt rule by rule and sends real requests as GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot and PerplexityBot, so it also catches firewall and Cloudflare blocks. Some providers verify the bot's IP, so treat a firewall rejection there as a strong signal. Then run the full AISaysYes scan for a 0 to 100 score in about 30 seconds.
Open yourstore.com/robots.txt. Look for Disallow: / under GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot or PerplexityBot. Cloudflare can also manage robots.txt for you, so lines you never wrote can show up there.
The exact menu labels move from time to time, so here is the logic rather than a pixel-perfect path:
Shopify runs its own edge, and most Shopify stores do not sit behind a Cloudflare account their owner controls. Still check if you put your own proxy or a bot protection app in front of the store. On Shopify, the robots.txt file comes from a theme template (robots.txt.liquid), so check that no app or developer added a Disallow for AI bots.
WooCommerce stores are the most exposed. They often sit behind Cloudflare, a hosting firewall and a security plugin at the same time. Any of the three can block a bot. Test with curl after each change, not just once at the end.
Not necessarily. It is fair to block training crawlers if you do not want your product copy used to train models. But treat search and agent traffic as customers. AI referred traffic is still small: Similarweb measured 0.89% of ecommerce traffic in May 2026. It is growing fast, though: Adobe saw AI traffic to US retail sites up 393% year over year in Q1 2026, and those visitors converted better than other traffic. Blocking it by accident is a cheap mistake to avoid.
Once the door is open, the next step is making sure the bot understands what it reads. The step by step guide covers product data, prices and store identity. To see how other stores score, browse the public ranking. And if you want a short map of your key pages for AI agents, the free llms.txt generator takes two minutes.
Which product data AI assistants and Google really use: schema.org Product and Offer, price, currency, availability and product feeds. What is live in the US today and what is not.
How to get your store recommended by ChatGPT: a guide for Shopify and WooCommerceWhat ChatGPT, Perplexity, Claude and Gemini need before they recommend a store. Four fixes for Shopify and WooCommerce, with honest numbers on AI shopping traffic.
WooCommerce and AI shopping: get your store ready in one afternoonA practical WooCommerce checklist so ChatGPT, Perplexity, Claude and Gemini can read and recommend your store. Bots, firewall, product data, feeds and llms.txt.