Attempt to make crawlers stop scanning stg.fp.o (infra #7514)
Signed-off-by: Rick Elrod <relrod@redhat.com>
This commit is contained in:
Rick Elrod
parent
4c8cf933fc
commit
77644a5183
10 changed files with 6 additions and 3 deletions
|
|
@ -1,2 +0,0 @@
|
|||
User-agent: *
|
||||
Disallow: /wikiold
|
||||
|
|
@ -57,7 +57,7 @@
|
|||
- httpd/website
|
||||
|
||||
- name: And lastly, the robots.txt file
|
||||
copy: >
|
||||
template: >
|
||||
src={{item}}
|
||||
dest=/srv/web/robots.txt.{{site_name}}
|
||||
mode=0644
|
||||
|
|
|
|||
|
|
@ -0,0 +1,5 @@
|
|||
User-agent: *
|
||||
{% if env == "staging" %}
|
||||
Disallow: /
|
||||
{% endif %}
|
||||
Disallow: /wikiold
|
||||
Loading…
Add table
Add a link
Reference in a new issue