Patrick Uiterwijk
|
25f71933ab
|
Robots have no use in fedoracommunity as its just an aggregator
Signed-off-by: Patrick Uiterwijk <puiterwijk@redhat.com>
|
2015-09-16 11:50:59 +00:00 |
|
Kevin Fenzi
|
4f2d7f0362
|
Just disallow /updates. Not much point in bodhi updates being in search engines.
|
2015-04-23 15:01:16 +00:00 |
|
Patrick Uiterwijk
|
88cc733244
|
Also tell riddler we don't like them
|
2015-04-23 14:50:53 +00:00 |
|
Kevin Fenzi
|
09fb09d20d
|
Drop crawl-delay down to 1
|
2015-01-29 00:00:21 +00:00 |
|
Ralph Bean
|
00269dc0a8
|
Working on the proxies-websites.yml playbook.
|
2014-12-07 23:39:51 +00:00 |
|