AI bot access checker
Test robots.txt and a live request for every major AI bot. Find the firewall rule that quietly hides you from AI answers.
Result
Can ChatGPT, Claude and Perplexity reach your site?
How it works, what it checks and why it matters.
- 1Enter a URL
A domain or page address. No install, nothing to add to your site.
- 2We fetch it live
Our servers request the page read-only, the way a crawler would.
- 3Read and fix
Get the results in seconds, with what each one means.
Why it matters
Your robots.txt can say “allow” while your CDN answers 403. Only a live request shows what the bot actually gets.
Since July 2025, new Cloudflare sites block AI crawlers unless you opt in. Many owners never notice.
Blocking GPTBot stops training. Blocking OAI-SearchBot or ChatGPT-User removes you from ChatGPT answers.
What it checks
- OpenAI
- OAI-SearchBot (ChatGPT search), ChatGPT-User (browsing and agent) and GPTBot (training).
- Anthropic
- Claude-SearchBot, Claude-User and ClaudeBot, each tested separately.
- Perplexity
- PerplexityBot for search results and Perplexity-User for live answers.
- Google-Extended, the robots.txt switch for Gemini training.
- Common Crawl
- CCBot, which feeds the datasets many models are trained on.
- Firewalls and paywalls
- HTTP 403, 429, 503, pay-per-crawl 402 and challenge pages like “Just a moment…”.
Questions
Should I block AI training bots?
That’s your call. Blocking GPTBot, ClaudeBot, Google-Extended or CCBot keeps your content out of training data but doesn’t remove you from AI search answers. Blocking the search and user bots does.
Why does robots.txt say allowed but the live test fails?
Almost always a CDN or firewall rule, like Cloudflare’s AI bot blocking or a WAF rule that challenges unknown user-agents.
Is the live test exact?
Close, not exact. We send each bot’s user-agent from our servers. Real bots come from their own IP ranges, which some firewalls treat differently. Treat a live block as “likely”.
Is it free?
Yes. Every tool is free with no signup.
Do you store what I enter?
Browser tools never leave your device. Live tools fetch the URL you enter and don’t save the result.
How is this different from the site audit?
This tool checks one thing in depth. The site audit checks everything at once and scores SEO, AI Search and Agent Readiness.
More in AI crawlers & agents
Tools that pair well with this one