This script will generate a list of builder keytabs and place them on
the riscv secondary hub where they can be distributed to builders.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
The patch provided originally was made against the latest pagure code,
but we are not on the latest dev version. So I created the patch
directly for the source file deployed on src.stg.fedoraproject.org.
There is a local fix on src.fedoraproject.org that is missing on
src.stg.fedoraproject.org. I provided the fix in
https://pagure.io/pagure/pull-request/5525 and this commit is adding it
as patch to ansible, so the change is permanent.
This commit is fixing ansible lint errors for distgit role.
It also introduces two new addition to skip list as the structure of our
ansible repository doesn't adhere to ansible standards.
The errors that will be now skipped are:
- role-name[path] - we have plenty of roles that have sub-roles inside
them and we need to access them
- var-naming[no-role-prefix] - variables for roles are not usually
prefixed correctly in our repository and forcing people to change that
will introduce more issues than what it solves
Before the custom rules was actually intended to _allow_ more things
on a particular host. Putting those blocks in there was useless because
custom rules were applied _after_ all the allowed ports, so it wasn't
really blocking anything.
This moves them to a block_rules applied before the ports are allowed
Also move pagure's to that new rule list.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
In the beginning, this just handled Azure images. Now it does Azure,
AWS, GCP, and containers. Currently, it processes images serially, which
is mostly okay. However, it does mean that whatever service is handled
last has to wait for all the others to succeed before it starts, and it
also means if any of the handlers for their respective platform fail, it
retries *all* the images again. For most things this is a no-op (or a
few inexpensive calls), but it does have to re-download the image from
Koji to checksum it.
This adds an AMQP message queue for each content type we handle, and
produces a fedora-messaging config for each content type. The deployment
is now made up of 4 containers: azure-image-uploader,
aws-image-uploader, container-image-uploader, and
google-cloud-image-uploader. They only differ in the secrets injected
into them and the fedora-messaging config file they use. The end result
is that images should be available faster and its more resilient to
remote services being down.
Finally, it's worth noting that this bumps the warning threshold for
queue sizes. It can take some services (Azure and AWS) upwards of 30
minutes to replicate the images around the world, and since we subscribe
to _any_ compose status changes, it's not unreasonable for 5-10 messages
to stack up when we hit a compose change that is "FINISHED" with images.
Signed-off-by: Jeremy Cline <jeremycline@linux.microsoft.com>
Right now we have to add external to everything in iad2, but most of it
isn't external at all. This way we can just assume it's not external if
it's not defined and just define it on the ones where it's true.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
Enables the `image-builder` plugin from `koji-image-builder` in the
production environment for both the koji hub, and the koji builder
(kojid).
This is based on the earlier enablement in staging where I've
succesfully tested some builds and it didn't seem to bring down all of
the staging instance.
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
