Generoi optimoituja robots.txt-tiedostoja tuella 40+ tekoälybotille mukaan lukien ChatGPT, Claude, Perplexity ja Gemini. Hallitse, mitkä botit voivat käyttää sisältöäsi.
OpenAI ChatGPT web crawler
ChatGPT Browse with Bing
Google Gemini AI crawler
Perplexity AI search engine
Anthropic Claude web crawler
Claude web search feature
Aggressive content scraper
Spam indexer bot
Aggressive crawler
Deprecated link crawler
Aggressive Huawei crawler
/admin//api//dashboard//auth//login/Usually at https://yoursite.com/sitemap.xml — check by visiting that URL. This tells bots where to find all your pages.
1# ==========================================================================2# robots.txt3# Generated by GEO-Score robots.txt Generator4# https://geo-score.online/tools/robots-generator5# Generated: 2026-08-236# ==========================================================================78# Blocked bots9User-agent: BLEXBot10User-agent: MegaIndex11User-agent: SeznamBot12User-agent: linkdexbot13User-agent: PetalBot14Disallow: /1516# Default rules for all other bots17User-agent: *18Disallow: /admin/19Disallow: /api/20Disallow: /dashboard/21Disallow: /auth/22Disallow: /login/23
No sitemap URL declared
Add your sitemap URL (e.g. https://yoursite.com/sitemap.xml) above. This is the bridge between robots.txt and your sitemap — it helps all bots discover your pages.