AI Bot Access Checker

See at a glance which AI crawlers (GPTBot, ClaudeBot, Perplexity, Google-Extended, and more) your site is blocking — and which ones can actually read your content.

100% free No signup required Instant result

Why AI bot access matters

Every major LLM (ChatGPT, Claude, Perplexity, Gemini, Mistral) crawls the web with a named user agent so site owners can allow or deny them in robots.txt. If you block GPTBot, your content will never train ChatGPT. If you block OAI-SearchBot or PerplexityBot, you won't appear in AI-generated answers — even when a user asks a question your site perfectly answers. Many sites block these bots by accident (a CDN default, an outdated template, a copy-pasted snippet). This tool shows you exactly which ones are blocked in seconds.

Frequently asked questions about AI crawlers

What is the difference between training and search bots?

Training bots (GPTBot, ClaudeBot, Google-Extended, CCBot) crawl your site to build or improve future AI models. Search / answer bots (OAI-SearchBot, PerplexityBot, Claude-Web) fetch pages at query time to include in live AI answers. Fetch bots (ChatGPT-User, Perplexity-User) pull specific URLs when a user explicitly shares or requests them. Blocking training bots keeps your content out of future models. Blocking search bots keeps you out of AI-generated answers today.

Should I block AI bots?

It depends on your strategy. If your content is a competitive asset you don't want to see in AI answers, blocking is reasonable. If you want visibility in ChatGPT, Claude, Perplexity, and Gemini, you need to allow the relevant bots. Many marketing-heavy sites unintentionally block AI crawlers because Cloudflare, WP plugins, or old templates include default disallow rules. This tool helps you spot those.

Where does the checker look?

We fetch https://yoursite.com/robots.txt and parse it with the standard Robots Exclusion Protocol. We then check each of ~20 known AI crawlers against the rules: if a bot has its own User-agent section with Disallow: /, we flag it as explicitly blocked. If there's no matching section but User-agent: * has Disallow: /, we flag it as wildcard-blocked. Everything else counts as allowed.

What is the difference between GPTBot and ChatGPT-User?

GPTBot is OpenAI's training crawler — it collects data to improve future GPT models. ChatGPT-User is a user-triggered fetcher that pulls a URL when someone shares it in a ChatGPT conversation. OAI-SearchBot is the newest of the three: it powers ChatGPT Search and visits pages to include them in real-time answers. Allowing all three maximizes your visibility; blocking GPTBot while allowing the others keeps you out of training but present in live answers.

Does this tool check every AI bot?

We check about 20 of the most widely declared AI user agents across OpenAI, Anthropic, Google, Perplexity, Meta, Apple, Amazon, ByteDance, Mistral, Cohere, You.com, DuckDuckGo, and Common Crawl. New bots appear regularly, so we refresh the list periodically. If you're using a private or enterprise agent that isn't covered, add it to the same snippet we provide as a fix.

Other free AI visibility tools