Does blocking GPTBot stop ChatGPT recommending my store?

No. GPTBot collects data for model training. ChatGPT reaches your store through OAI-SearchBot and ChatGPT-User, which are separate names you allow or block independently.

Is ClaudeBot the crawler that lets Claude read my site?

No, and this is the one most people get wrong. ClaudeBot is Anthropic’s training crawler. Claude reaches your store through Claude-SearchBot and Claude-User.

Does Google-Extended affect my Google rankings?

No. Google states explicitly that Google-Extended does not affect inclusion in Google Search and is not a ranking signal. It controls whether your content can be used for Gemini training and grounding.

Should I just allow everything?

Allow every agent that brings shoppers. Whether you allow the training crawlers is a business decision with no effect on sales today — plenty of sites block them deliberately, and that is fine.

Every AI crawler, and whether to allow it

Nine named agents, four assistants, and one distinction that decides whether blocking one costs you nothing or costs you every sale it would have sent. This page is generated from the same configuration the scanner uses, so it cannot drift out of sync with the tool.

The distinction that matters

Every one of these agents identifies itself by name in your robots.txt, and you can allow or refuse each one independently. But they do four different jobs, and the names give you no clue which is which.

Answer agents build the index the assistant recommends from. Refusing one removes you from that assistant’s answers.
Live-fetch agents open your page at the moment a shopper asks about you. Refusing one means the assistant cannot check your details on request.
Grounding controls decide whether your content may be used when the assistant answers, while leaving ordinary search alone.
Training crawlers collect pages to train future models. Refusing one changes nothing about whether shoppers find you today.

The full list

Agent	Assistant	What it does
`OAI-SearchBot`	ChatGPT	Builds the answer index. Block it and your store cannot be recommended at all.
`ChatGPT-User`	ChatGPT	Opens your page on demand. Block it and the assistant cannot check your page when a shopper asks.
`GPTBot`	ChatGPT	Collects training data. Block it and nothing changes for shoppers today.
`PerplexityBot`	Perplexity	Builds the answer index. Block it and your store cannot be recommended at all.
`ClaudeBot`	Claude	Collects training data. Block it and nothing changes for shoppers today.
`Google-Extended`	Gemini	Permits use when answering. Block it and you get vaguer mentions, but ordinary search is untouched.
`Claude-SearchBot`	Claude	Builds the answer index. Block it and your store cannot be recommended at all.
`Claude-User`	Claude	Opens your page on demand. Block it and the assistant cannot check your page when a shopper asks.
`Perplexity-User`	Perplexity	Opens your page on demand. Block it and the assistant cannot check your page when a shopper asks.

If you only allow four things, allow these

These are the agents that decide whether a shopper ever hears your name. Putting them above any catch-all rule in your robots.txt is the single highest-value edit on this page.

# Let AI assistants read this store.
# Keep these above any "User-agent: *" section.

User-agent: OAI-SearchBot
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Claude-SearchBot
Allow: /

User-agent: ChatGPT-User
Allow: /

User-agent: Claude-User
Allow: /

User-agent: Perplexity-User
Allow: /

The ones you can refuse without losing anything

Blocking these stops your pages being used for model training. It does not remove you from any assistant’s recommendations. It is a legitimate choice either way, and the scanner reports it as a minor note rather than a problem.

GPTBot — your pages are not used to train OpenAI models.
ClaudeBot — your pages are not used to train Anthropic models.

The awkward one

Perplexity documents that Perplexity-User may disregard robots.txt entirely, on the grounds that a person asked for the page rather than a crawler deciding to visit. Blocking it may therefore not have the effect you intend. Worth knowing before you rely on it.

Keeping this honest

These names and roles change without notice, and a reference page that quietly goes stale is worse than no reference page. Everything above was last checked against the vendors’ own documentation on 2026-09-20. The sources are:

To find out which of these your own store currently allows, run the check — it reports every agent individually, and generates the corrected robots.txt lines for any that are refused.

Questions

Does blocking GPTBot stop ChatGPT recommending my store?: No. GPTBot collects data for model training. ChatGPT reaches your store through OAI-SearchBot and ChatGPT-User, which are separate names you allow or block independently.
Is ClaudeBot the crawler that lets Claude read my site?: No, and this is the one most people get wrong. ClaudeBot is Anthropic’s training crawler. Claude reaches your store through Claude-SearchBot and Claude-User.
Does Google-Extended affect my Google rankings?: No. Google states explicitly that Google-Extended does not affect inclusion in Google Search and is not a ranking signal. It controls whether your content can be used for Gemini training and grounding.
Should I just allow everything?: Allow every agent that brings shoppers. Whether you allow the training crawlers is a business decision with no effect on sales today — plenty of sites block them deliberately, and that is fine.