Enter your website and we read its robots.txt the way each AI crawler does, then show which assistants can reach your pages and which ones you have shut out, on purpose or not.
Free, no sign-up. We fetch public pages only and store nothing.
People now ask ChatGPT, Claude and Perplexity for recommendations. An assistant can only cite pages its crawler is allowed to read, and one line in robots.txt, often added by a plugin or a cautious developer, can quietly remove you from all of them.
Most AI companies run one crawler to train models and another to answer questions. You can block training (GPTBot, ClaudeBot, Google-Extended) and still appear in answers (OAI-SearchBot, Claude-SearchBot, PerplexityBot). This check shows the two groups apart so the choice is a deliberate one.
Block the training crawlers if you do not want your content used to train models; that has no effect on being cited in answers. Blocking the search and answer crawlers is rarely a good idea for a business, because it removes you from the places people increasingly ask for recommendations.
No. Google-Extended only controls whether Google may use your content for Gemini training and grounding. Google Search, including AI Overviews, uses Googlebot.
robots.txt is a request, not a lock. Firewalls and CDNs can block crawlers outright; Cloudflare, for example, has a setting that blocks AI crawlers on every site that turns it on. If this check says you are open but assistants never mention you, look there next.
Edit robots.txt at the root of your site: remove the Disallow: / line under that crawler's User-agent, or add a group for it with Allow: /. Each crawler picks up the change the next time it fetches the file, usually within a day.
OAI-SearchBot builds the index behind ChatGPT search results. ChatGPT-User fetches a page when a person asks ChatGPT to look at it. Blocking either one limits how ChatGPT can use your pages.