# Orbator.io — robots.txt # Current product: the AI Recommendation Index ("what AI actually recommends") # + the AI Visibility report. Crawlers are allowed everywhere except private, # authenticated, and funnel pages. Public SEO surfaces (/ai-index/*, /vs/*, # /launched, /blog, /developers) are covered by the blanket Allow below — they # do not need to be listed individually. User-agent: * Allow: / # Private / authenticated app Disallow: /dashboard Disallow: /settings Disallow: /onboarding Disallow: /onboarding-rotation Disallow: /admin Disallow: /sign-in Disallow: /sign-up Disallow: /notifications Disallow: /products/ Disallow: /competitors Disallow: /directories Disallow: /seo/ Disallow: /automation/ Disallow: /keyword-gaps # Funnel / per-token pages (also carry ) Disallow: /claim/ Disallow: /proof/ Disallow: /r/ Disallow: /checkout Disallow: /extension-auth # LLM/AI guidance # See https://www.orbator.io/llms.txt Sitemap: https://www.orbator.io/sitemap.xml