1f6e9b8cbd
This follows https://pagure.io/fedora-infrastructure/issue/11003 Signed-off-by: Aurélien Bompard <aurelien@bompard.org>
185 lines
6.8 KiB
Text
185 lines
6.8 KiB
Text
= PDC SOP
|
|
|
|
Store metadata about composes we produce and "component groups".
|
|
|
|
App: https://pdc.fedoraproject.org/
|
|
|
|
Source for frontend: https://github.com/product-definition-center/product-definition-center
|
|
|
|
Source for backend: https://github.com/fedora-infra/pdc-updater
|
|
|
|
== Contact Information
|
|
|
|
Owner::
|
|
Release Engineering, Fedora Infrastructure Team
|
|
Contact::
|
|
#fedora-apps, #fedora-releng, #fedora-admin, #fedora-noc
|
|
Servers::
|
|
pdc-web0\{1,2}, pdc-backend01
|
|
Purpose::
|
|
Store metadata about composes and "component groups"
|
|
|
|
== Description
|
|
|
|
The Product Definition Center (PDC) is a webapp and API designed for
|
|
storing and querying product metadata. We automatically populate our
|
|
instance with data from our existing releng tools/processes. It doesn't
|
|
do much on its own, but the goal is to enable us to develop more sane
|
|
tooling down the road for future releases.
|
|
|
|
The webapp is a django app running on pdc-web0\{1,2}. Unlike most of our
|
|
other apps, it does not use OpenID for authentication, but it instead
|
|
uses SAML2. It uses _mod_auth_mellon_ to achieve this (in
|
|
cooperation with ipsilon). The webapp allows new data to be POST'd to it
|
|
by admin users.
|
|
|
|
The backend is a _fedmsg-hub_ process running on
|
|
_pdc-backend01_. It listens for new composes over fedmsg and then POSTs
|
|
data about those composes to PDC. It also listens for changes to the
|
|
fedora atomic host git repo in pagure and updates "component groups" in
|
|
PDC to reflect what rpm components constitute fedora atomic host.
|
|
|
|
For long-winded history and explanation, see the original Change
|
|
document: https://fedoraproject.org/wiki/Changes/ProductDefinitionCenter
|
|
|
|
[NOTE]
|
|
====
|
|
PDC is being replaced by fpdc (Fedora Product Definition Center)
|
|
====
|
|
|
|
== Upgrading the Software
|
|
|
|
There is an upgrade playbook in `playbooks/manual/upgrade/pdc.yml` which
|
|
will upgrade both the frontend and the backend if new packages are
|
|
available. Database schema upgrades should be handled automatically with
|
|
a run of that playbook.
|
|
|
|
== Logs
|
|
|
|
Logs for the frontend are in `/var/log/httpd/error_log` on
|
|
pdc-web0\{1,2}.
|
|
|
|
Logs for the backend can be accessed with
|
|
`journalctl -u fedmsg-hub -f` on _pdc-backend01_.
|
|
|
|
== Restarting Services
|
|
|
|
The frontend runs under apache. So either `apachectl graceful`
|
|
or `systemctl restart httpd` should do it.
|
|
|
|
The backend runs as a _fedmsg-hub_, so
|
|
`systemctl restart fedmsg-hub` should restart it.
|
|
|
|
== Scripts
|
|
|
|
The _pdc-updater_ package (installed on _pdc-backend01_) provides three
|
|
scripts:
|
|
|
|
* `pdc-updater-audit`
|
|
* `pdc-updater-retry`
|
|
* `pdc-updater-initialize`
|
|
|
|
A possible failure scenario is that we will lose a fedmsg message and
|
|
the backend will not update the frontend with info about that compose.
|
|
To detect this, we provide the `pdc-updater-audit` command
|
|
(which gets run once daily by cron with emails sent to the releng-cron
|
|
list). It compare all of the entries in PDC with all of the entries in
|
|
kojipkgs and then raises an alert if there is a discrepancy.
|
|
|
|
Another possible failure scenario is that the fedmsg message is
|
|
published and received correctly, but there is some processing error
|
|
while handling it. The event occurred, but the import to the PDC db
|
|
failed. The `pdc-updater-audit` script should detect this
|
|
discrepancy, and then an admin will need to manually repair the problem
|
|
and retry the event with the `pdc-updater-retry` command.
|
|
|
|
If doomsday occurs and the whole thing is totally hosed, you can delete
|
|
the db and re-ingest all information available from releng with the
|
|
`pdc-updater-initialize` tool. (Creating the initial schema needs to
|
|
happen on pdc-web01 with the standard django settings.py commands.)
|
|
|
|
== Manually Updating Information
|
|
|
|
In general, you shouldn't have to do these things. `pdc-updater` will
|
|
automatically create new releases and update information, but if you
|
|
ever need to manipulate PDC data, you can do it with the _pdc-client_
|
|
tool. A copy is installed on _pdc-backend01_ and there are some
|
|
credentials there you'll need, so ssh there first.
|
|
|
|
Make sure that you are root so that you can read
|
|
`/etc/pdc.d/fedora.json`.
|
|
|
|
Try listing all of the releases:
|
|
|
|
....
|
|
$ pdc -s fedora release list
|
|
....
|
|
|
|
Deactivating an EOL release:
|
|
|
|
....
|
|
$ pdc -s fedora release update fedora-21-updates --deactivate
|
|
....
|
|
|
|
[NOTE]
|
|
====
|
|
There are lots more attribute you can manipulate on a release (you can
|
|
change the type, and rename them, etc..) See `pdc --help`
|
|
and `pdc release --help` for more information.
|
|
====
|
|
|
|
Listing all composes:
|
|
|
|
....
|
|
$ pdc -s fedora compose list
|
|
....
|
|
|
|
We're not sure yet how to flag a compose as the Gold compose, but when
|
|
we do, the answer should appear here:
|
|
https://github.com/product-definition-center/product-definition-center/issues/428
|
|
|
|
== Adding superusers
|
|
|
|
Some small group of release engineers need to be superuser to set eol
|
|
dates and add/remove components. You can grant them permissions to do
|
|
this via some direct database calls. First find out their email address
|
|
listed in fas, then login to _db01.iad2.fedoraproject.org_:
|
|
|
|
....
|
|
sudo -u postgresql psql pdc pdc-
|
|
# update kerb_auth_user set is_superuser = 'true' where email = 'usersemailfromfas';
|
|
....
|
|
|
|
The user will now have privs with their normal tokens.
|
|
|
|
== Updating SAML2 certificates
|
|
|
|
As stated previously, the authentication uses SAML2 with _mod_auth_mellon_ (as the Service Provider on PDC's side) and Ipsilon (as the Identity Provider). This form of authentication relies on SSL certificates and XML metadata.
|
|
|
|
PDC's certificates live in the _ansible-private_ repository, in `files/saml2/pdc{,.stg}.fedoraproject.org/certificate.{pem,key}`. They are generated from the PKI in `files/saml2/ {staging,production}/`. The certificates can be self-signed as long as they are properly embedded in the metadata XML file and this file is distributed identically to PDC and to Ipsilon.
|
|
|
|
To renew the certificate, generate a new one with the provided script in the _ansible-private_ repo:
|
|
|
|
....
|
|
$ files/saml2/staging/build-key-server pdc.stg.fedoraproject.org
|
|
$ mv files/saml2/staging/keys/pdc.stg.fedoraproject.org.crt files/saml2/pdc.stg.fedoraproject.org/certificate.pem
|
|
$ mv files/saml2/staging/keys/pdc.stg.fedoraproject.org.key files/saml2/pdc.stg.fedoraproject.org/certificate.key
|
|
....
|
|
|
|
And for production:
|
|
|
|
....
|
|
$ files/saml2/production/build-key-server pdc.fedoraproject.org
|
|
$ mv files/saml2/production/keys/pdc.fedoraproject.org.crt files/saml2/pdc.fedoraproject.org/certificate.pem
|
|
$ mv files/saml2/production/keys/pdc.fedoraproject.org.key files/saml2/pdc.fedoraproject.org/certificate.key
|
|
....
|
|
|
|
And commit the changes:
|
|
|
|
....
|
|
$ git commit -a -s -m "PDC: new certificate"
|
|
$ git pull --rebase
|
|
$ git push
|
|
....
|
|
|
|
Then run the PDC and the Ipsilon playbooks. The PDC playbook will push the new certificates and re-generate the `metadata.xml` file in `/etc/httpd/saml2/`. The Ipsilon playbook will retrieve this `metadata.xml` file from the PDC server and insert it into the `/etc/ipsilon/root/configuration.conf` file.
|