Review koji SOP

Signed-off-by: Michal Konečný <mkonecny@redhat.com>
This commit is contained in:
Michal Konečný 2021-09-03 11:08:22 +02:00
parent 886ac43079
commit 76093bd388
2 changed files with 16 additions and 34 deletions

View file

@ -58,7 +58,7 @@
** xref:jenkins-fedmsg.adoc[Jenkins Fedmsg - SOP]
** xref:kerneltest-harness.adoc[Kerneltest-harness - SOP]
** xref:kickstarts.adoc[Kickstart Infrastructure - SOP]
** xref:koji.adoc[koji - SOP in review ]
** xref:koji.adoc[Koji Infrastructure - SOP]
** xref:koji-archive.adoc[koji-archive - SOP in review ]
** xref:koji-builder-setup.adoc[koji-builder-setup - SOP in review ]
** xref:koschei.adoc[koschei - SOP in review ]

View file

@ -1,7 +1,6 @@
= Koji Infrastructure SOP
[NOTE]
.Note
====
We are transitioning from two buildsystems, koji for Fedora and plague
for EPEL, to just using koji. This page documents both.
@ -12,27 +11,16 @@ machines to do their work.
== Contents
[arabic]
. Contact Information
. Description
. Add packages into Buildroot
. Troubleshooting and Resolution
[arabic]
. Restarting Koji
. kojid won't start or some builders won't connect
. OOM (Out of Memory) Issues
[arabic]
. Increase Memory
. Decrease weight
[arabic, start=4]
. Disk Space Issues
{empty}5. Should there be mention of being sure filesystems in chroots
are unmounted before you delete the chroots?
* <<_contact_information>>
* <<_description>>
* <<_add_packages_into_buildroot>>
* <<_troubleshooting_and_resolution>>
** <<_restarting_koji>>
** <<_kojid_wont_start_or_some_builders_wont_connect>>
** <<_oom_out_of_memory_issues>>
*** <<_increase_memory>>
*** <<_decrease_weight>>
** <<_disk_space_issues>>
== Contact Information
@ -42,8 +30,6 @@ Contact::
#fedora-admin, sysadmin-build group
Persons::
mbonnet, dgilmore, f13, notting, mmcgrath, SmootherFrOgZ
Location::
Phoenix
Servers::
* koji.fedoraproject.org
* buildsys.fedoraproject.org
@ -54,12 +40,11 @@ Purpose::
== Description
Users submit builds to koji.fedoraproject.org or
buildsys.fedoraproject.org. From there it gets passed on to the
Users submit builds to _koji.fedoraproject.org_ or
_buildsys.fedoraproject.org_. From there it gets passed on to the
builders.
[IMPORTANT]
.Important
====
At present plague and koji are unaware of each other. A result of this
may be an overloaded builder. A easy fix for this is not clear at this
@ -92,7 +77,6 @@ service kojid restart
....
[IMPORTANT]
.Important
====
If postgres becomes interrupted in some way, koji will need to be
restarted. As long as the koji master daemon gets restarted the builders
@ -108,7 +92,7 @@ This is common if koji crashes and the db connections aren't properly
cleared. Upon restart many of the connections are full so koji cannot
reconnect. Clearing old connections is easy, guess about how long it the
new koji has been up and pick a number of minutes larger then that and
kill those queries. From db3 as postgres run:
kill those queries. From _db3_ as _postgres_ run:
....
echo "select procpid from pg_stat_activity where usename='koji' and now() - query_start \
@ -141,7 +125,7 @@ Edit `/etc/xen/xenbuilder[1-4]` and add more memory.
Each builder has a weight as to how much work can be given to it.
Presently the only way to alter weight is actually changing the database
on db3:
on _db3_:
....
$ sudo su - postgres
@ -176,10 +160,9 @@ df -h
....
[IMPORTANT]
.Important
====
If any one of the following directories is full, send an outage
notification as outlined in: [62]Infrastructure/OutageTemplate to the
notification as outlined in: Infrastructure/OutageTemplate to the
fedora-infrastructure-list and fedora-devel-list, then contact Mike
McGrath
@ -215,7 +198,6 @@ up:
=== Unmounting
[WARNING]
.Warning
====
Should there be mention of being sure filesystems in chroots are
unmounted before you delete the chroots?