# All Robots # Disallow unwanted URL patterns to be crawled and indexed User-agent: * Disallow: /search/ Disallow: /qwerty/ Disallow: /*.ece$ Disallow: /utils/ Disallow: /account/ Disallow: /LoadTest/ Disallow: /api/ Disallow: /qa/ Disallow: /ad-test Disallow: /service-archive Disallow: /subscribe-archive Disallow: /messagent/ Disallow: /extra/messagent/ # Disallow Sponsored Articles for Google News User-agent: Googlebot-News Disallow: /storyplus/* Disallow: /sponsored-features/* # Sitemap Files Sitemap: https://www.independent.ie/sitemap/sitemap_googlenews.xml       Sitemap: https://www.independent.ie/sitemap/sitemap_channels.xml Sitemap: https://www.independent.ie/sitemap/sitemap.xml Sitemap: https://www.independent.ie/sitemap/sitemap_video.xml # Allow Adsense User-agent: Mediapartners-Google Disallow: User-agent: GPTBot Disallow: / User-agent: CCBot Disallow: / User-agent: Google-Extended Disallow: /