# OpenAI crawler used to surface websites in ChatGPT search results. User-agent: OAI-SearchBot Allow: / # Perplexity crawler used for search result discovery. User-agent: PerplexityBot Allow: / # Anthropic crawler used to improve search result quality. User-agent: Claude-SearchBot Allow: / User-agent: * Allow: / Sitemap: https://trendsagi.com/sitemap.xml