Check if ChatGPT can see your website
Type in your domain and we'll test whether GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot, and Google-Extended can actually read it — the same way each bot does, not just an eyeball read of the file. If any of them are blocked, AI can't quote, cite, or recommend your site, no matter how good it is.
We read your public robots.txt live, the same way GPTBot and friends do. Nothing about your site is stored.
5 AI crawlers you actually need to allow
These are the bots this checker tests for. Block any one of them in robots.txt and that AI can't read your pages at all — it doesn't matter how good your content is if the crawler never gets past the front door.
GPTBot
Crawls pages to train and improve OpenAI's models. Blocking it keeps your content out of future model training — separate from what ChatGPT can cite right now.
OAI-SearchBot
Fetches pages so ChatGPT's search feature can cite and link to them in live answers. This is the bot most directly tied to "can ChatGPT see my site right now."
ClaudeBot
Crawls pages so Claude's answers and web-search features can reference your content.
PerplexityBot
Crawls and indexes pages for Perplexity's answer engine, which cites sources directly inside its answers.
Google-Extended
A separate opt-in signal from classic Googlebot — controls whether Google can use your content to train Gemini and generate AI Overviews.
Two more worth knowing: ChatGPT-User and anthropic-ai aren't crawlers that build an index — they fetch a single page live, the moment someone pastes your link into a chat. Blocking them breaks that one-off fetch even if your site is otherwise fully crawlable. Robots.txt only controls the front door, though — pair an open robots.txt with a clean llms.txt file so every one of these bots gets a structured summary instead of parsing your HTML from scratch.
How robots.txt blocks AI without anyone deciding to
Almost nobody sits down and decides to keep ChatGPT out. It happens by accident, in one of three ways:
- Plugin or theme defaults. Some SEO plugins and store themes ship a broad
Disallowrule for any user-agent they don't recognize — which silently catches every new AI bot as it's released. - Leftover staging rules. A
robots.txtwritten to keep search engines out of a staging or dev site gets copied to production and never updated. - Network-level blocking. Cloudflare and similar CDNs and firewalls now ship a one-click block-AI-bots toggle that blocks GPTBot and friends at the network level — returning a 403 even when the
robots.txtfile itself looks completely open.
That last one is why reading the file by eye isn't enough — the checker above tests live access, the way each bot actually experiences your site.
The fix: a robots.txt that welcomes AI
Add (or un-block) these in your robots.txt so the major AI crawlers can read your store:
# Allow AI crawlers (robots.txt — yourstore.com/robots.txt) User-agent: GPTBot Allow: / User-agent: OAI-SearchBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: ClaudeBot Allow: / User-agent: anthropic-ai Allow: / User-agent: PerplexityBot Allow: / User-agent: Google-Extended Allow: /
ChatGPT search leans on Bing — so Bing indexing matters too
ChatGPT's search feature is built on top of Microsoft Bing's index, not a separate web index OpenAI crawled from scratch. That means two things both have to be true before ChatGPT can surface and cite your page in a live search answer:
- OAI-SearchBot has to be allowed to crawl your site (the check above).
- Bing has to have actually indexed your pages.
A store that's fine for Google but was never submitted to Bing can still be invisible to ChatGPT search, even with a wide-open robots.txt. If you've only ever set up Google Search Console, add Bing Webmaster Tools to the list.
Questions, answered
Which AI bots should I check for?
The five that matter most are GPTBot and OAI-SearchBot (OpenAI/ChatGPT), ClaudeBot (Anthropic), PerplexityBot (Perplexity), and Google-Extended (Google's AI models). If your robots.txt blocks any of these, that crawler can't read your pages — so it can't cite, quote, or recommend you.
Why would robots.txt be blocking ChatGPT without me knowing?
Usually nobody blocked it on purpose. Common causes: an SEO plugin or theme shipped a broad Disallow rule for unrecognized bots, a staging-site robots.txt got pushed to production, or a CDN/firewall's block-AI-bots toggle got switched on at the network level — which can block GPTBot even when robots.txt itself looks wide open. That's why testing live access, not just reading the file, is what actually tells you.
Does ChatGPT search depend on more than my own robots.txt?
Yes. ChatGPT's search feature is built on Microsoft Bing's index, so OAI-SearchBot being allowed gets you real-time crawl access, but Bing still needs to have indexed your pages. A store that's never been submitted to Bing Webmaster Tools can be invisible to ChatGPT search even with a completely open robots.txt.
How do I fix robots.txt to allow AI crawlers?
Add explicit Allow rules for GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, anthropic-ai, PerplexityBot, and Google-Extended to your robots.txt — the exact snippet is on this page, ready to copy. Most platforms let you edit robots.txt directly; Shopify stores need a robots.txt.liquid template override.
Crawlable is step one. Are you actually recommendable?
Letting AI in is necessary but not sufficient. Run the full free check to see your AI-Visibility + Agent-Commerce score — schema, answer-readiness, competitor comparison, and the rest.