Infrastructure/ansible
1
0
Fork 0
Code Issues Pull requests 6 Projects Releases Packages Wiki Activity Actions

Also tell riddler we don't like them

Browse source
This commit is contained in:
Patrick Uiterwijk 2015-04-23 14:50:45 +00:00
parent f77abb46e8
commit 88cc733244
1 changed files with 4 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

4
roles/httpd/website/files/robots/robots.txt.admin.fedoraproject.org
View file

@ -6,3 +6,7 @@ Disallow: /pkgdb/packages/name
# Temp block http://www.80legs.com/webcrawler.html # Temp block http://www.80legs.com/webcrawler.html
User-agent: 008 User-agent: 008
Disallow: / Disallow: /
# Block riddler.io
User-agent: Riddler
Disallow: /