User-agent: Googlebot Allow: / Crawl-delay: 1 User-agent: Bingbot Allow: / Crawl-delay: 2 User-agent: Twitterbot Allow: / User-agent: facebookexternalhit Allow: / User-agent: LinkedInBot Allow: / User-agent: Slurp Allow: / Crawl-delay: 2 User-agent: DuckDuckBot Allow: / User-agent: Applebot Allow: / # AI search & answer engines — allow citation of public content. # These crawlers cite our pages in ChatGPT, Perplexity, Claude, Google AI # Overviews, Bing Copilot, etc. Allowing them maximises AI-search visibility. User-agent: GPTBot Allow: / Disallow: /admin Disallow: /dashboard Disallow: /profile Disallow: /preferences User-agent: OAI-SearchBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: PerplexityBot Allow: / User-agent: Perplexity-User Allow: / User-agent: anthropic-ai Allow: / User-agent: ClaudeBot Allow: / User-agent: Claude-Web Allow: / User-agent: Claude-SearchBot Allow: / User-agent: Google-Extended Allow: / User-agent: GoogleOther Allow: / User-agent: Bingbot-Discover Allow: / User-agent: Amazonbot Allow: / User-agent: Meta-ExternalAgent Allow: / User-agent: Meta-ExternalFetcher Allow: / User-agent: cohere-ai Allow: / User-agent: YouBot Allow: / User-agent: Diffbot Allow: / User-agent: Bytespider Allow: / # Block bulk-scraping/training-only crawlers that don't drive citations. User-agent: CCBot Disallow: / User-agent: Omgilibot Disallow: / User-agent: ImagesiftBot Disallow: / User-agent: * Allow: / Disallow: /admin Disallow: /admin/ Disallow: /onboarding Disallow: /recommendation Disallow: /recommendation-result Disallow: /quick-pick Disallow: /preferences Disallow: /dashboard Disallow: /profile Disallow: /analytics Disallow: /wrapped Disallow: /subscription Disallow: /reset-password Disallow: /forgot-password Disallow: /email-confirmation Disallow: /shared/ Disallow: /social Disallow: /taste-dna Disallow: /couple Disallow: /watchlist Disallow: /confirm-email Disallow: /auth Host: https://kairivo.com Sitemap: https://kairivo.com/sitemap.xml # RSS Feed # https://kairivo.com/rss.xml