In prod db-fas01 is rhel7 and uses postgresql 9.6.
In staging db-fas01.stg is rhel8, and should also use postgresql 9.6,
but we were blanket making rhel8 hosts use postgresql 12.
We could drop this by reinstalling db-fas01.stg with rhel7, or waiting
until we finally kill fas2 and just setting them both to use postgresql
12.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
Turns out copy module doesn't actually have a 'absent' state.
So, just remove this (we no longer need it as there's a timer on koji
hub that does this from there).
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
db-koji01 has been running with this since before the mass rebuild, and
it seems to make it have a higher load, but process faster and without
stalling when doing backups or when long/bad koji-gc queries for old
versions of texlive hit it.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
Turns out we were not setting effective_cache_size even tho it was set
for some servers (pagure). Adjust a few parameters on db-koji to try and
get some more performance out of it.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
I took the default postgresql.conf from postgresql 12 and then added in
various changes we already manually made and variable substitions we
already had setup back in the postgresq 9.2 days.
This will apply to db-koji01, db-qa01, db-datanommer01 at least.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
In phx2 we had a warm standby database host (db-koji02), but we no
longer have that host in iad2, so we shouldn't try and make db-koji01
handle that. Also, this was just changed mistakenly as it's the warm
standby host that should get the recovery.conf file.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
Basically, if the variables are defined in the host, use them, otherwise
use the current values.
Signed-off-by: Pierre-Yves Chibon <pingou@pingoured.fr>
Our postgresql.conf is from postgresql 9.2 while RHEL8 ships 10.x which
leads to postgresql no longer wanting to start (as seen on pagure-stg01).
Signed-off-by: Pierre-Yves Chibon <pingou@pingoured.fr>
By default apache uses prefork and a limit of 250. It's possible that this limit was
the thing causing us issues over the last week. This moves to the event mpm and ups limits
a lot. It also needs to up limits on db connections or the increased workers will just
cause the db server to overload.
With this setup, builders are no longer dropping out, but it's not clear if it's solved
all the issues we have been seeing.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
The recent changes to postgresql_server increased the required memory past
what one of my VMs had. I've added a conditional in postgresql.conf to put
some memory settings back where they used to be (controlled by
small_postgres_instance, default is false) and created a default to not use
the small_postgres_instance settings unless specified
Turns out I had set this on the master (db-koji01) which is ignored.
We need to set it on the standby. With a value of -1, the standby will wait
for conflicting transactions/locks to complete however long it takes.
If this doesn't work to get us a good backup on db-koji02, no harm and we
can try something else.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
