Infrastructure/ansible
1
0
Fork 0
Code Issues Pull requests 6 Projects Releases Packages Wiki Activity Actions

websites: Disallow robots crawling fas.

Browse source 
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
This commit is contained in:
Kevin Fenzi 2019-05-14 16:54:18 +00:00
parent 7a4781f380
commit d056090c7f
1 changed files with 1 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

1
roles/httpd/website/templates/robots/admin.fedoraproject.org-robots.txt
View file

@ -2,6 +2,7 @@ User-agent: *
Disallow: /voting
Disallow: /mirrormanager
Disallow: /updates
Disallow: /accounts
# Temp block http://www.80legs.com/webcrawler.html
User-agent: 008