infrastructure/ansible
1
0
Fork 0
Code Pull requests 7 Activity Actions

pagure: update ansible for latest blocks from ai

Browse source 
Update the latest project blocks that were added.

Also, extend the bot block to docs.pagure.io, which was
being hammered by scrapers.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
This commit is contained in:
Kevin Fenzi 2025-05-30 10:02:24 -07:00
parent 77384338e5
commit 836d79193c
1 changed files with 22 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

22
roles/pagure/templates/0_pagure.conf
View file

@ -176,6 +176,21 @@ MaxConnectionsPerChild 1000
RewriteCond %{REQUEST_URI} ^/fork/mohanboddu/fedora-infra/ansible.*$
RewriteRule .* - [F]
RewriteCond %{REQUEST_URI} ^/fedora-docs/fedora-docs-web/blob.*$
RewriteRule .* - [F]
RewriteCond %{REQUEST_URI} ^fedora-qa/os-autoinst-distri-fedora/blob.*$
RewriteRule .* - [F]
RewriteCond %{REQUEST_URI} ^releng/fedora-scm-requests/blob.*$
RewriteRule .* - [F]
RewriteCond %{REQUEST_URI} ^pagure/usage/index.html.*$
RewriteRule .* - [F]
RewriteCond %{REQUEST_URI} ^/fork/.*/koji/.*$
RewriteRule .* - [F]
# Spiders-gone-wild
# These spiders may not follow robots.txt and will
# hit admin sections which consume large amounts of CPU
@ -256,6 +271,13 @@ MaxConnectionsPerChild 1000
RedirectMatch ^/SSSD.sssd/index.html https://sssd.io/index.html
RedirectMatch ^/SSSD.sssd/(.*) https://sssd.io/docs/$1
# Spiders-gone-wild
# These spiders may not follow robots.txt and will
# hit admin sections which consume large amounts of CPU
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} "(Bytespider|ClaudeBot|Amazonbot|YandexBot|ChatGLM-Spider|GPTBot|Barkrowler|YisouSpider|MJ12bot)" [NC]
RewriteRule .* - [F,L]
<Location />
WSGIProcessGroup paguredocs
<IfModule mod_authz_core.c>