Should You Block AI Crawlers on Your Shopify Store?

TL;DR: For merchants, the right move in 2026 is to allow verified AI crawlers and block their impersonators. Shoppers ask ChatGPT, Claude, and Perplexity what to buy, and those assistants can only recommend stores their crawlers have read: blocking GPTBot removes you from answers your competitors will appear in. The real threat is scrapers wearing AI crawler names as camouflage, and the fix is network-level verification, not blocking by name.

Publishers spent the last few years blocking AI crawlers, and plenty of Shopify merchants have copied the pattern without asking whether it fits commerce. It mostly does not. This post lays out why the publisher logic fails for stores, what the actual threat is, and how to keep the real crawlers in while keeping the fakes out.

AI assistants are the new product discovery

A growing share of shoppers now starts with a question instead of a search box: "what's a good waterproof hiking boot under 150 euros," typed into ChatGPT or Perplexity. The assistant answers with specific products from specific stores, often with links. Those recommendations do not come from nowhere. They come from what the assistant's crawlers have read.

GPTBot and OAI-SearchBot (OpenAI), ClaudeBot (Anthropic), and PerplexityBot are, for product discovery, what Googlebot was for the last twenty years of search. If they have never read your product pages, your store effectively does not exist in that answer. There is no ads budget that fixes absence from the index.

This is the same structural argument as classic SEO, which is why the reasoning in do bot blockers hurt SEO transfers almost unchanged: the crawlers that feed discovery channels are the last traffic you want to block, and a blocker that cannot tell them apart from bad bots is a liability.

The instinct to block, and where it comes from

The block-AI-crawlers reflex comes from publishing. News sites and content platforms blocked GPTBot in waves, arguing that AI companies were training on their work without licensing it. Whatever you think of that fight, the underlying logic is coherent for a publisher: their product is the content. If an AI reads the article and summarizes it to a user, that summary substitutes for the visit. The reader never arrives, the pageview never happens, the ad is never shown. Reading is consumption.

Merchants saw big names blocking AI crawlers and copied the robots.txt lines, or installed blockers that treat every non-human visitor as hostile. The pattern transferred. The logic did not.

Why the publisher logic does not transfer to commerce

A merchant's product is not the product page. It is the boots in the box. An AI reading your catalog cannot substitute for buying from you; it can only describe you, and describing you to a shopper with intent is called distribution.

When a shopper asks an assistant what to buy, some set of stores will be in the answer. Blocking AI crawlers does not make the question go away. It just guarantees the answer is built from your competitors' catalogs instead of yours.

There is a narrow exception: if your product pages contain genuinely proprietary content that is itself the value (paid research, licensed imagery under strict terms), you have a publisher-shaped problem for those pages. That describes very few stores.

The real threat: scrapers in AI crawler costumes

None of this means merchants have nothing to worry about. The legitimate worry is that a user agent is just a text string. Any scraper can send GPTBot in its headers, and many do, precisely because merchants have learned to wave that name through.

The price scraper harvesting your catalog for a competitor, the bot farming your inventory data, the content thief cloning your product pages: all of them benefit from looking like a crawler you want to allow. Blocking by name fails in both directions at once. You block the real GPTBot and lose the recommendation, or you allow everything named GPTBot and wave the scrapers through with it.

The answer: verify networks, not names

Real AI crawlers are verifiable. OpenAI, Anthropic, Perplexity, Google, and Microsoft all crawl from published network ranges, and reverse DNS confirms whether a visitor claiming to be one of them actually comes from the operator's infrastructure. A "GPTBot" arriving from a random hosting ASN is not GPTBot. It is a scraper in costume, and it should be blocked like one. (If ASNs are new to you, the ASN explainer covers how network-level identity works.)

This is how Cordon handles it:

• Verified good bots are always allowed. GPTBot, ClaudeBot, PerplexityBot, Googlebot, Bingbot, and legitimate SEO crawlers like Ahrefs pass verification and cannot be blocked, even by your own rules. You cannot accidentally remove yourself from AI answers or search with an overly broad country or ASN rule.
• Verification is network-level. Cordon checks reverse DNS, not the user agent string, so the name alone earns nothing.
• Impostors get caught. Spoofed-crawler detection flags visitors wearing a good bot's name from the wrong network and blocks them, alongside headless-browser markers and request-velocity detection for the scrapers that do not bother with a costume. The full detection breakdown lists every signal.

The result is the only defensible policy: the real GPTBot always gets in, the fake one never does, and you did not have to maintain a robots.txt guess about which is which. For stores where scraping is the primary concern, the content scraping use case goes deeper, including the anti-scraping decoy mode on the Plus plan that feeds impostors garbage data instead of a block screen.

What AI crawlers respect

Two conventions worth knowing:

robots.txt. The major AI crawlers generally respect robots.txt directives, and each operator documents its user agent tokens. This is also why robots.txt is not a security control: it is a request that polite crawlers honor and scrapers ignore. Use it to express policy (for example, disallowing crawl of cart and checkout paths), not to enforce it. Enforcement is what network-level blocking is for.

llms.txt. A newer convention: a plain-text file at /llms.txt giving language models a clean, structured summary of what your site is and where the important pages are. Adoption is early and no assistant guarantees it reads the file, but it costs an hour and it is the closest thing to telling AI systems directly what you want them to know about your store.

Make your store AI-readable

Allowing crawlers in is step zero. What they find when they arrive decides whether you get recommended. A short checklist:

1. Write product descriptions with facts, not fragments. Assistants quote complete sentences with concrete attributes (materials, dimensions, weight, compatibility). "Premium quality" gives a model nothing to recommend you for.
2. Ship structured data. Product schema with price, availability, and review markup is machine-readable by design. Most Shopify themes emit the basics; verify yours does and that the values match the page.
3. Keep pricing information consistent. One clear price per variant, currency stated, no prices that only render after JavaScript-heavy interactions a crawler may not execute.
4. Give your about page real facts. Founded when, based where, ships to which countries, returns policy in plain language. Assistants weigh whether a store looks legitimate before recommending it.
5. Add llms.txt. A short summary of the store, top collections, shipping regions, and links to key pages.
6. Confirm your blocker is not sabotaging you. Check your visitor log for verified crawlers being served block screens. In Cordon this cannot happen by construction, but if you run another blocker or hand-rolled rules, audit them. Shopify's own guidance on storefront access lives at help.shopify.com.

Frequently asked questions

Should I block GPTBot on my Shopify store?

For most merchants, no. GPTBot is how ChatGPT learns your catalog exists, and assistants can only recommend stores their crawlers have read. Blocking it removes you from AI shopping answers while your competitors stay in them.

Can a scraper pretend to be an AI crawler?

Yes, trivially. The user agent is just a text string anyone can send, so a price scraper can call itself GPTBot. Real AI crawlers come from published, verifiable networks, which is why verification has to happen at the network level, not by name.

Why do publishers block AI crawlers if merchants should not?

A publisher's product is the content itself, so an AI reading it can substitute for the visit. A merchant's product is a physical thing. An AI reading your catalog is distribution, not substitution: the recommendation is the goal.

What is llms.txt?

A convention for placing a plain-text summary of your site at /llms.txt so language models can read a clean, structured overview. It is young and not universally honored, but it is cheap to add and signals what you want AI systems to know.

Does Cordon ever block real AI crawlers by mistake?

No. Verified good bots like GPTBot, ClaudeBot, PerplexityBot, Googlebot, and Bingbot are always allowed and cannot be blocked, even by your own rules. Verification is network-level, so an impostor sending the same name from a hosting ASN still gets caught.

---

If you want the real AI crawlers reading your catalog and the fake ones on a block screen, that is Cordon's default behavior out of the box: verified good bots always allowed, spoofed crawlers detected at the network level, decisions in under 50 milliseconds. Try Cordon on the Shopify App Store with a 7-day free trial on paid plans.