Without shutting httpd down, sometimes - when there's high enough load -
post scriptlet of copr-frontend takes too long to finish (restarging
httpd, gently waiting for existing connections). But that makes the
outage window larger as httpd doesn't accept the new connections when
shutting down.
So automatize (forced) shutdown of httpd first (when there's existing
frontend update) -> upgrade the package -> run the migrations -> and
make httpd started again.
There is a problem with our current playbooks, that they can be
executed automatically without us knowing about it. That is an issue
particularly during release process because we can prepare new
packages into infra-tags repo or bodhi and a nightly reprovision
can upgrade to them outside of an outage window or any of us being
prepared for it.
Therefore `groups/copr-*.yml` playbooks *should not* upgrade any
packages, but only ensure, that those packages are installed. For
upgrade, there should be separate `manual/copr/copr-*-upgrade.yml`
playbooks. Because they are located under `manual` directory, it
is secured, that they can't be run automatically.