Review koji SOP

Signed-off-by: Michal Konečný <mkonecny@redhat.com>
This commit is contained in:
Michal Konečný 2021-09-03 11:08:22 +02:00
parent 886ac43079
commit 76093bd388
2 changed files with 16 additions and 34 deletions

View file

@ -58,7 +58,7 @@
** xref:jenkins-fedmsg.adoc[Jenkins Fedmsg - SOP] ** xref:jenkins-fedmsg.adoc[Jenkins Fedmsg - SOP]
** xref:kerneltest-harness.adoc[Kerneltest-harness - SOP] ** xref:kerneltest-harness.adoc[Kerneltest-harness - SOP]
** xref:kickstarts.adoc[Kickstart Infrastructure - SOP] ** xref:kickstarts.adoc[Kickstart Infrastructure - SOP]
** xref:koji.adoc[koji - SOP in review ] ** xref:koji.adoc[Koji Infrastructure - SOP]
** xref:koji-archive.adoc[koji-archive - SOP in review ] ** xref:koji-archive.adoc[koji-archive - SOP in review ]
** xref:koji-builder-setup.adoc[koji-builder-setup - SOP in review ] ** xref:koji-builder-setup.adoc[koji-builder-setup - SOP in review ]
** xref:koschei.adoc[koschei - SOP in review ] ** xref:koschei.adoc[koschei - SOP in review ]

View file

@ -1,7 +1,6 @@
= Koji Infrastructure SOP = Koji Infrastructure SOP
[NOTE] [NOTE]
.Note
==== ====
We are transitioning from two buildsystems, koji for Fedora and plague We are transitioning from two buildsystems, koji for Fedora and plague
for EPEL, to just using koji. This page documents both. for EPEL, to just using koji. This page documents both.
@ -12,27 +11,16 @@ machines to do their work.
== Contents == Contents
[arabic] * <<_contact_information>>
. Contact Information * <<_description>>
. Description * <<_add_packages_into_buildroot>>
. Add packages into Buildroot * <<_troubleshooting_and_resolution>>
. Troubleshooting and Resolution ** <<_restarting_koji>>
** <<_kojid_wont_start_or_some_builders_wont_connect>>
[arabic] ** <<_oom_out_of_memory_issues>>
. Restarting Koji *** <<_increase_memory>>
. kojid won't start or some builders won't connect *** <<_decrease_weight>>
. OOM (Out of Memory) Issues ** <<_disk_space_issues>>
[arabic]
. Increase Memory
. Decrease weight
[arabic, start=4]
. Disk Space Issues
{empty}5. Should there be mention of being sure filesystems in chroots
are unmounted before you delete the chroots?
== Contact Information == Contact Information
@ -42,8 +30,6 @@ Contact::
#fedora-admin, sysadmin-build group #fedora-admin, sysadmin-build group
Persons:: Persons::
mbonnet, dgilmore, f13, notting, mmcgrath, SmootherFrOgZ mbonnet, dgilmore, f13, notting, mmcgrath, SmootherFrOgZ
Location::
Phoenix
Servers:: Servers::
* koji.fedoraproject.org * koji.fedoraproject.org
* buildsys.fedoraproject.org * buildsys.fedoraproject.org
@ -54,12 +40,11 @@ Purpose::
== Description == Description
Users submit builds to koji.fedoraproject.org or Users submit builds to _koji.fedoraproject.org_ or
buildsys.fedoraproject.org. From there it gets passed on to the _buildsys.fedoraproject.org_. From there it gets passed on to the
builders. builders.
[IMPORTANT] [IMPORTANT]
.Important
==== ====
At present plague and koji are unaware of each other. A result of this At present plague and koji are unaware of each other. A result of this
may be an overloaded builder. A easy fix for this is not clear at this may be an overloaded builder. A easy fix for this is not clear at this
@ -92,7 +77,6 @@ service kojid restart
.... ....
[IMPORTANT] [IMPORTANT]
.Important
==== ====
If postgres becomes interrupted in some way, koji will need to be If postgres becomes interrupted in some way, koji will need to be
restarted. As long as the koji master daemon gets restarted the builders restarted. As long as the koji master daemon gets restarted the builders
@ -108,7 +92,7 @@ This is common if koji crashes and the db connections aren't properly
cleared. Upon restart many of the connections are full so koji cannot cleared. Upon restart many of the connections are full so koji cannot
reconnect. Clearing old connections is easy, guess about how long it the reconnect. Clearing old connections is easy, guess about how long it the
new koji has been up and pick a number of minutes larger then that and new koji has been up and pick a number of minutes larger then that and
kill those queries. From db3 as postgres run: kill those queries. From _db3_ as _postgres_ run:
.... ....
echo "select procpid from pg_stat_activity where usename='koji' and now() - query_start \ echo "select procpid from pg_stat_activity where usename='koji' and now() - query_start \
@ -141,7 +125,7 @@ Edit `/etc/xen/xenbuilder[1-4]` and add more memory.
Each builder has a weight as to how much work can be given to it. Each builder has a weight as to how much work can be given to it.
Presently the only way to alter weight is actually changing the database Presently the only way to alter weight is actually changing the database
on db3: on _db3_:
.... ....
$ sudo su - postgres $ sudo su - postgres
@ -176,10 +160,9 @@ df -h
.... ....
[IMPORTANT] [IMPORTANT]
.Important
==== ====
If any one of the following directories is full, send an outage If any one of the following directories is full, send an outage
notification as outlined in: [62]Infrastructure/OutageTemplate to the notification as outlined in: Infrastructure/OutageTemplate to the
fedora-infrastructure-list and fedora-devel-list, then contact Mike fedora-infrastructure-list and fedora-devel-list, then contact Mike
McGrath McGrath
@ -215,7 +198,6 @@ up:
=== Unmounting === Unmounting
[WARNING] [WARNING]
.Warning
==== ====
Should there be mention of being sure filesystems in chroots are Should there be mention of being sure filesystems in chroots are
unmounted before you delete the chroots? unmounted before you delete the chroots?