User-agent: * Disallow: /api/ Disallow: /telemetry/ Disallow: /legal/ # AI crawlers — same rules, explicit visibility User-agent: ChatGPT-User Allow: / Allow: /llms.txt Allow: /agents.txt Allow: /robots.txt Allow: /llms-full.txt Allow: /manual.html Disallow: /api/ Disallow: /telemetry/ Disallow: /legal/ User-agent: ClaudeBot Allow: / Allow: /llms.txt Allow: /agents.txt Allow: /robots.txt Allow: /llms-full.txt Allow: /manual.html Disallow: /api/ Disallow: /telemetry/ Disallow: /legal/ User-agent: Google-Extended Allow: / Allow: /llms.txt Allow: /agents.txt Allow: /robots.txt Allow: /llms-full.txt Allow: /manual.html Disallow: /api/ Disallow: /telemetry/ Disallow: /legal/ User-agent: PerplexityBot Allow: / Allow: /llms.txt Allow: /agents.txt Allow: /robots.txt Allow: /llms-full.txt Allow: /manual.html Disallow: /api/ Disallow: /telemetry/ Disallow: /legal/ User-agent: Applebot-Extended Allow: / Allow: /llms.txt Allow: /agents.txt Allow: /robots.txt Allow: /llms-full.txt Allow: /manual.html Disallow: /api/ Disallow: /telemetry/ Disallow: /legal/ User-agent: OAI-SearchBot Allow: / Allow: /llms.txt Allow: /agents.txt Allow: /robots.txt Allow: /llms-full.txt Allow: /manual.html Disallow: /api/ Disallow: /telemetry/ Disallow: /legal/ User-agent: BytesSpider Allow: / Allow: /llms.txt Allow: /agents.txt Allow: /robots.txt Allow: /llms-full.txt Allow: /manual.html Disallow: /api/ Disallow: /telemetry/ Disallow: /legal/ User-agent: GPTBot Allow: / Allow: /llms.txt Allow: /agents.txt Allow: /robots.txt Allow: /llms-full.txt Allow: /manual.html Disallow: /api/ Disallow: /telemetry/ Disallow: /legal/ User-agent: Applebot Allow: / Allow: /llms.txt Allow: /agents.txt Allow: /robots.txt Allow: /llms-full.txt Allow: /manual.html Disallow: /api/ Disallow: /telemetry/ Disallow: /legal/ User-agent: Diffbot Allow: / Allow: /llms.txt Allow: /agents.txt Allow: /robots.txt Allow: /llms-full.txt Allow: /manual.html Disallow: /api/ Disallow: /telemetry/ Disallow: /legal/ # Block training-data crawlers User-agent: CCbot Disallow: / Sitemap: https://thebottomline.tech/sitemap.xml