I tweaked the playbook to not patch the templates for non-infra
deployments, but then forgot to make test loading work using
non-patched templates for non-infra...
necessary updates for openqa roles have gone stable for now, so
disable updates-testing usage (keep the plays around commented,
though, since we'll likely need them again in future). Also, a
bit more attempted support for non-infra use: make the monkey
patching of the repo URLs in the test templates only happen if
deployment_type is defined, actually respect the openqa_consumer
var (don't enable the job scheduling consumer unless it's truey)
and only enable any wiki reporting consumer if deployment_type
is defined.
rwmjones says the guestfs / rpm bug has been fixed (a new base
fedora-23 image has been uploaded which should avoid it, anyway)
so let's try turning disk image generation back on and see how
it flies.
I'm still kinda trying to make the openqa roles usable outside
of infra, so now I have a minute, let's do this: it makes the
static UID/GID for geekotest optional and configurable, instead
of hard coding it. For infra we set the value to 601, as we
are already using, in the openqa and openqa-stg group config.
this is a database value and there's no openQA API function to
set it, so we have to do it directly in the database...this
*should* work. I think. I should add equivalent functionality
for sqlite use as well, really...
there seems to be a bug in python2-guestfs which causes the
disk image with an updates.img file to be broken (the updates
image is only 4096 bytes long and incomplete). When built with
Python 3 the image seems to be fine. So, run the script with
python3 (as its hashbang says) and ensure necessary packages
are installed.
with openQA jobs being scheduled and wiki results reported
(well, when we get that working again) by fedmsg-hub consumers,
we have to let the fedmsg user read these files.
we really shouldn't always_run these, as they actually *do*
stuff. That does mean that in check mode we won't check if
database init and admin user creation are necessary, but these
are only ever necessary exactly one time at initial deployment
which is likely to be kicked off manually anyway, so it should
not be a problem. We adjust the 'when' condition for the 'wait
for services to start up' action, so it will work in check mode
with the 'services' variable undefined (because the step that
defines it now won't be run).
This puts in place the bits needed to use the new createhdds.py
for image generation, which works much nicer with ansible than
the old createhdds.sh. For now, the disk creation step is
disabled, because creating some images (including the ones that
get refreshed regularly) requires running VMs, and the openqa
server hosts are VMs themselves and do not currently have nested
virt enabled, so the creation would run extremely slow. Once we
get nested virt enabled, I'll change the condition for the image
creation step to what it should be. Until then I'll carry on
handling image creation manually (both hosts have all the images
at present anyhow).
basically, dump the config from the server before loading the
templates file, dump it after, and use json_diff to compare.
I wanted to make the actual template loading script from openQA
do some kind of check in '--clean' mode, but given how it's
designed that's not really terribly practical without doing
some major re-architecting, and this seems like a relatively
simple alternative. With this change, the actual "Load tests"
step will never show 'changed', but the subsequent 'Check if
the tests changed in previous step' step will show as changed
when appropriate.
I didn't do this originally because I was trying to make it
only show changed when something had really changed, but it was
too conservative - it wouldn't update when a test definition
changed, only when one was added. Doing it manually is a bit
annoying (you have to do the sed step and the ownership change)
so it's better to let you just re-run the play to update the
tests, even if it does mean getting the occasional spurious
changed state.
Turns out this was the reason why desktop HDD image generation
wasn't working: we need the XFS support bits for libguestfs or
else it can't resize the partitions.
So apparently hosts within infra can't reach dl.fp.o. This is a
problem as we have that host written in as a reliable repository
URL for a couple of openQA tests. infra hosts can reach dl.fp.o
by the name 'dl.phx2.fedoraproject.org', but nothing outside
infra can reach that hostname. So we don't just want to change
it in the openQA test git repo because we want that to work for
external deployments. So, let's just sed it on the fly for
the infra deployment...
this is needed for openID auth to work properly - otherwise
ipsilon thinks the request is coming from 'https://localhost'
and redirect after auth fails. openQA uses this config setting
for the openID base URL if it's set.
This adds openQA server, worker and dispatcher roles, and
applies them to the appropriate hosts. A few secret vars are
required. See trac #4958 for discussion.
