User-agent: * Allow: /staff/$ Disallow: /staff/* Disallow: /staff_category/ Disallow: /staff_category/* # Jesse - 2021-04-21 # High traffic crawlers - they might not honour robots.txt though? User-Agent: trendkite-akashic-crawler Disallow: /