From adade90a2c5fab07abddbe9c567d45e5a3574923 Mon Sep 17 00:00:00 2001 From: Michael Simacek Date: Fri, 24 Feb 2017 16:02:34 +0100 Subject: [PATCH] Disallow robots from expensive queries on koschei --- .../website/files/robots/robots.txt.apps.fedoraproject.org | 2 ++ 1 file changed, 2 insertions(+) diff --git a/roles/httpd/website/files/robots/robots.txt.apps.fedoraproject.org b/roles/httpd/website/files/robots/robots.txt.apps.fedoraproject.org index 49afb40b31..33f12bef99 100644 --- a/roles/httpd/website/files/robots/robots.txt.apps.fedoraproject.org +++ b/roles/httpd/website/files/robots/robots.txt.apps.fedoraproject.org @@ -1,4 +1,6 @@ User-agent: * Disallow: /packages/ Disallow: /datagrepper/ +Disallow: /koschei/user/ +Disallow: /koschei/affected-by/ Crawl-delay: 1