User-agent: * Allow: / Disallow: */search Disallow: */tag-search Disallow: /read Disallow: /article/ Disallow: /a/ User-agent: Googlebot Allow: / Disallow: */search Disallow: */tag-search Disallow: /read Disallow: /article/ Disallow: /a/ User-agent: facebookexternalhit Allow: / Disallow: */search Disallow: */tag-search Disallow: /read User-agent: Slurp Allow: / Disallow: */search Disallow: */tag-search Disallow: /read Disallow: /article/ Disallow: /a/ User-agent: Twitterbot Allow: / Disallow: */search Disallow: */tag-search Disallow: /read Disallow: /article/ Disallow: /a/ User-agent: Bingbot Allow: / Disallow: */search Disallow: */tag-search Disallow: /read Disallow: /article/ Disallow: /a/ User-agent: DuckDuckBot Allow: / Disallow: */search Disallow: */tag-search Disallow: /read Disallow: /article/ Disallow: /a/ User-agent: Baiduspider Allow: / Disallow: */search Disallow: */tag-search Disallow: /read Disallow: /article/ Disallow: /a/ User-agent: Yandexbot Allow: / Disallow: */search Disallow: */tag-search Disallow: /read Disallow: /article/ Disallow: /a/ # Block specific bots User-agent: msnbot Disallow: / User-agent: archive.org_bot Disallow: / User-agent: ia_archiver Disallow: / User-agent: Barkrowler Disallow: / Sitemap: https://a5r5br.net/sitemap/sitemap-index-countries.xml Sitemap: https://a5r5br.net/sitemap/sitemap-index-categories.xml Sitemap: https://a5r5br.net/sitemap/sitemap-sections.xml Sitemap: https://a5r5br.net/sitemap/sitemap-index-posts.xml Sitemap: https://a5r5br.net/sitemap/sitemap-index-tags.xml