Infrastructure/ansible
3
0
Fork 0
Code Issues Pull requests 6 Projects Releases Packages Wiki Activity Actions

Add pkgdb to robots.txt.

Browse source
This commit is contained in:
Ralph Bean 2017-12-06 20:37:39 +00:00
parent 9b3c452be1
commit a654c55201
1 changed files with 1 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
roles/httpd/website/files/robots/robots.txt.admin.fedoraproject.org
View file

@ -1,7 +1,7 @@
User-agent: * User-agent: *
Disallow: /voting Disallow: /voting
Disallow: /mirrormanager Disallow: /mirrormanager
Disallow: /pkgdb/packages/name Disallow: /pkgdb
Disallow: /updates Disallow: /updates
# Temp block http://www.80legs.com/webcrawler.html # Temp block http://www.80legs.com/webcrawler.html