Scan how well your website performs for ChatGPT Search, Claude, Perplexity, and AI agents. Inspect robots.txt rules, Content-Signals, Markdown negotiation, and MCP cards with 1-click prompts.
Checks if GPTBot, ClaudeBot, and Perplexity are permitted to crawl your content in robots.txt.
Tests Content-Signals RFC training rules and Accept: text/markdown negotiation for token-efficient reading.
Provides copy-paste prompts for ChatGPT, Claude, and Cursor to auto-fix missing rules and files.
Ensure search bots and AI agents never encounter 5xx outages, expired certificates, or slow response times. Uzvera monitors uptime and response health continuously.
The developer guide to LLM crawlers, token-efficient Markdown delivery, and agentic protocols.
Traditional search engines evaluate backlinks and keyword frequency to produce a list of links. In contrast, AI answer engines (ChatGPT Search, Perplexity, Gemini) read full page text into a neural context window to synthesize direct answers. Websites optimized for AI SEO earn citations and brand references directly inside synthesized responses.
Rather than a blunt wildcard rule, modern best practice is to declare permissions explicitly for top AI crawlers:
# Allow AI search citation while protecting private directories User-agent: GPTBot Allow: / Disallow: /admin Disallow: /api/private User-agent: ClaudeBot Allow: / Disallow: /admin User-agent: PerplexityBot Allow: / # Prevent training on proprietary content while keeping search visible Content-Signal: ai-train=no, search=yes, ai-input=yes Sitemap: https://yourdomain.com/sitemap.xml
When an AI agent visits your page with Accept: text/markdown, delivering clean markdown instead of full HTML eliminates navigation menus, CSS, and script tags. This reduces token consumption by up to 70%, making your site faster and significantly cheaper for AI systems to read and reference.
Everything you need to know about preparing your website for autonomous agents and LLMs.
AI SEO (Artificial Intelligence Search Engine Optimization) optimizes websites for AI answer engines (such as ChatGPT Search, Perplexity AI, Claude, and Google AI Overviews) rather than traditional blue-link search. It ensures AI crawlers can discover, comprehend, and cite your content accurately in real-time answers.
When AI agents fetch a webpage, standard HTML includes heavy scripts, styles, and navigation boilerplate that wastes LLM token context. Modern AI-ready servers inspect the HTTP 'Accept: text/markdown' header and return clean, token-efficient Markdown directly.
Content-Signals is an emerging web standard declared in robots.txt (e.g. Content-Signal: ai-train=no, search=yes, ai-input=yes). It gives publishers granular control: preventing AI companies from using their content to train foundational models while granting permission to cite them in live AI search answers.
Major AI bots include GPTBot (OpenAI crawler), ChatGPT-User (real-time browsing for ChatGPT users), ClaudeBot (Anthropic crawler), PerplexityBot (Perplexity AI real-time indexer), and Google-Extended (controls Google Gemini training without affecting Google Search ranking).
Model Context Protocol (MCP) allows AI models and agent harnesses (like Claude Desktop, Cursor, and AGY) to connect to your website's public APIs and tools directly via /.well-known/mcp.json, enabling users to interact with your services autonomously.
Yes! Click 'Download Branded PDF Report' to generate a full PDF audit, or click 'Share Audit Link' to share a link with your score badge.
