Infrastructure/ansible
1
0
Fork 0
Code Issues Pull requests 6 Projects Releases Packages Wiki Activity Actions

Drop crawl-delay down to 1

Browse source
This commit is contained in:
Kevin Fenzi 2015-01-29 00:00:21 +00:00
parent 4fe887e988
commit 09fb09d20d
2 changed files with 3 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

2
roles/httpd/website/files/robots/robots.txt
View file

@ -1,2 +1,2 @@
User-agent: * User-agent: *
Crawl-delay: 10 Crawl-delay: 1

3
roles/httpd/website/files/robots/robots.txt.lockbox01
View file

@ -1,4 +1,5 @@
User-agent: * User-agent: *
Crawl-delay: 10 Crawl-delay: 1
Allow: /infra/docs/ Allow: /infra/docs/
Disallow: /infra Disallow: /infra
Allow: /cgit/