The API didn't work and failed with 500, because the REST framework didn't know
how to render the response. This fixes the issue.
https://pagure.io/fedora-infrastructure/issue/12134
Signed-off-by: Michal Konecny <mkonecny@redhat.com>
The index generated by the hyperkitty-hourly script generated unusually big
index (around 1TB).
This script should help with generating the search index from scratch, after
that we can leave it to hyperkitty-hourly job again.
Signed-off-by: Michal Konecny <mkonecny@redhat.com>
The database log is really big even on info level, let' set it manually to
warning to save space and don't generate GB of logs.
Signed-off-by: Michal Konecny <mkonecny@redhat.com>
There was a typo in logging section of mailman configuration, which caused that
only default log files were created.
Also there is separate logging for gunicorn now, let's use that.
Signed-off-by: Michal Konecny <mkonecny@redhat.com>
Mailman has huge memory spikes after migration, those are caused by visiting
`/archives/search` URL. This is probably because of the search index still being
regenerated.
This patch will disable the search from both web UI and as a HTTP URL endpoint.
Fixes https://pagure.io/fedora-infrastructure/issue/12043
Signed-off-by: Michal Konecny <mkonecny@redhat.com>
To be able to enable timer it needs to have WantedBy in [Install] section. Let's
add a hyperkitty.target to that section.
Signed-off-by: Michal Konecny <mkonecny@redhat.com>
This commit is setting both mailman and hyperkitty to devmode and doesn't allow
them to sent mails anywhere.
I tested that from web UI and using `mail` command on the machine.
Signed-off-by: Michal Konecny <mkonecny@redhat.com>
Set django to have 4 workers and have a retry longer than the timeout.
Disable local memcached and use the memcached01 instance.
Enabled debug logging on gunicorn to see what it's doing.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
Looked at logs of servers being hit by the 'non-responsive' bots and
the following were hit heavily every day multiple times a day:
100006 nagios.fedoraproject.org-access.log
102150 koschei.fedoraproject.org-access.log
162296 lists.fedoraproject.org-access.log
495776 fedoraproject.org-access.log
850471 dl.fedoraproject.org-access.log
Added bloks to dl.fedoraproject to try and lower its hit rate. Others
need review from people who know their internals more.
Signed-off-by: Stephen Smoogen <ssmoogen@redhat.com>
I went through the last couple of logs afer the first round of 'turn
off the spiders' went out. I looked at the areas which the /robots.txt
disregard and then looked for the bots which ignored it and still
looked up stuff in 'accounts'. This may cut down CPU spikes as these
are looking at dynamic data which can 'blow' things up.
It might be good to add similar tooling to pagure and src since they
seem to be hit a lot in the logs also.
Signed-off-by: Stephen Smoogen <ssmoogen@redhat.com>
Use the recommended CPU*2+1 amount of workers and limit max requests done. After
that the gunicorn reloads itself and free up all the memory.
Signed-off-by: Michal Konecny <mkonecny@redhat.com>
It seems that I forgot to actually create the certificates for new mailman
fedora archiver. Let's fix that.
Signed-off-by: Michal Konecny <mkonecny@redhat.com>
Mailman hourly job is making the web unresponsive, let's try if increasing the
amount of gunicorn workers improves the situation.
Signed-off-by: Michal Konecny <mkonecny@redhat.com>
It seems that setting `gatenews_every` to 0 is now throwing `ValueError`. Let's
keep this to default (5m) and just ignore the error.
Signed-off-by: Michal Konecny <mkonecny@redhat.com>
