Which AI crawlers should a Shopify store allow?

The major ones in 2026 are GPTBot and OAI-SearchBot (OpenAI), ChatGPT-User (live retrieval during a chat), ClaudeBot (Anthropic), PerplexityBot, Google-Extended (Google's AI training and AI answers), and Applebot-Extended (Apple). If you want to appear in AI shopping answers, you want these reading your storefront.

What does blocking AI crawlers cost me?

If your robots.txt blocks a crawler, that assistant cannot read your store at all, so you cannot appear in its answers no matter how good your structured data is. The most damaging version is a blanket 'Disallow: /' under 'User-agent: *', which blocks everyone. It is often inherited from a copied robots.txt or a security tool's default, and you get no error, you just never show up.

Does Shopify let me edit robots.txt?

Yes. Shopify generates robots.txt automatically but lets you customize it via the robots.txt.liquid template in your theme. You can add or remove rules for specific user agents there. Make changes carefully, since a wrong rule can quietly block crawlers across the whole store.

How do I tell if AI crawlers are blocked right now?

Run your store through the free AI-readiness checker. It reads your robots.txt and reports, crawler by crawler, whether GPTBot, ClaudeBot, PerplexityBot, Google-Extended and the others are allowed, so you can see at a glance if anything is shutting them out.

Which AI Crawlers to Allow in Your Shopify robots.txt

You can publish perfect Product schema and a tidy llms.txt, and still be invisible to AI shopping assistants for one boring reason: your robots.txt is turning the crawlers away at the door.

This is the cheapest mistake to make and the cheapest to fix. Here is what to check.

The crawlers that matter in 2026

AI assistants read the web through named crawlers. The ones worth knowing for a store:

GPTBot and OAI-SearchBot (OpenAI) feed ChatGPT and its search.
ChatGPT-User is the live fetch when someone asks ChatGPT about a specific page during a chat.
ClaudeBot (Anthropic) feeds Claude.
PerplexityBot feeds Perplexity.
Google-Extended governs whether Google can use your content for its AI features and answers.
Applebot-Extended (Apple) governs Apple's AI use.

If you want to be found, quoted, and recommended in AI shopping, these are the visitors you want to welcome.

How a block happens by accident

Almost nobody decides to block AI crawlers on purpose. It happens in two quiet ways:

A copied robots.txt. A Disallow: / under User-agent: * blocks every crawler from the whole site. It gets inherited from a template, a staging config, or another site, and nobody notices because there is no error message. You simply never appear.
A security tool or plugin default. Some bot-mitigation defaults lump AI crawlers in with scrapers and block them.

The damage is total and silent. A blocked crawler cannot read a single page, so none of your structured-data work can help you in that channel.

How to check and fix it on Shopify

Shopify generates your robots.txt automatically, and lets you customize it through the robots.txt.liquid template in your theme. That is where you can confirm the major AI crawlers are not disallowed, and remove any blanket block.

Edit it carefully. Because one rule can affect the whole store, a careless change can shut out crawlers everywhere. When in doubt, the safe default for a store that wants AI visibility is simple: do not disallow the AI crawlers listed above, and never ship a blanket Disallow: / for User-agent: *.

See your status crawler by crawler

The free AI-readiness checker reads your robots.txt and reports each major AI crawler as allowed or blocked, so you can see at a glance whether anything is shutting them out, alongside your structured data and discovery files. The full discoverability playbook puts crawler access in context with the other four pillars.

Where AgentReady fits

Crawler access is one thing AgentReady does not silently change for you, because your robots.txt is yours to control and a store may have legitimate reasons for specific rules. What AgentReady does is surface the problem clearly (so a blanket block cannot hide), then make the rest of your store worth crawling: complete structured data, a maintained llms.txt, and policy signals, all kept in sync automatically.

Start by checking your store. If the crawlers are blocked, fixing that one line is the highest-return change you can make today.