Три быстрых проверки перед тем, как ожидать, что AI-поиск будет цитировать вашу страницу

Оригинал

Three quick checks before you expect AI search to quote your page

I've been building small Claude/Cursor skills that review pages for AI-search readiness. Running them on public pages kept turning up the same three gaps. You can check all three yourself in about ten minutes with curl and view-source. You don't need a tool. 1. Does robots.txt say anything about AI crawlers? curl -s https://yoursite.com/robots.txt | grep -iE 'gptbot|claudebot|perplexitybot|google-extended|ccbot' No output means you have no AI-specific rules, so these bots fo