added code as example
Signed-off-by: Pedro Moura <pmoura@redhat.com>
This commit is contained in:
Pedro Moura
parent
b8a08cdfb5
commit
24b185baab
1 changed files with 11 additions and 132 deletions
|
|
@ -21,143 +21,22 @@ Follow the process to upgrade builders in staging environment:
|
|||
|
||||
=== Production Environment
|
||||
|
||||
For production a similar process is followed, but to avoid outages, builders are disabled, reinstalled and re-added small groups at a time. From _buildvm-s390x-01_ to _buildvm-s390x-14_ are **zvm** instances, they have to be updated with `dnf --releasever N distro-sync`.
|
||||
For production a similar process is followed, but to avoid outages, builders are disabled, reinstalled and re-added small groups at a time. From _buildvm-s390x-01_ to _buildvm-s390x-14_ are **zvm** instances, they have to be updated with `dnf --releasever NAME distro-sync`.
|
||||
In production builders, are usually done per bvmhost at a time:
|
||||
|
||||
. To do this process the user should be koji admin
|
||||
. Find all the builders on the host looking in `/var/log/virthost-lists.out` on batcave01 or look for `vmhost` variable in `inventory/host_vars/` on https://pagure.io/fedora-infra/ansible[ansible repository]
|
||||
. In a loop, disable all those builders using `koji disable-host <vmname>`
|
||||
. In a loop, disable all those builders using `koji disable-host <vmname>`. Like the example below:
|
||||
+
|
||||
....
|
||||
for i in seq -w 01 35
|
||||
do
|
||||
echo $i
|
||||
koji disable-host buildvm-s390x-$i.s390.fedoraproject.org
|
||||
done
|
||||
....
|
||||
. In a loop check for running builds on them with `koji list-tasks --host <vmname>`
|
||||
. Either wait for all of them to finish or optionally run `koji free-task NNNNN` to free a task and another builder will pick it up, but this second option will restart the build and if it is a long running build it could disturb maintainers
|
||||
. Either wait for all of them to finish or optionally run `koji free-task NAME` to free a task and another builder will pick it up, but this second option will restart the build and if it is a long running build it could disturb maintainers
|
||||
. Once they are all empty, destroy them as above and reinstall them with the `buildvm` playbook. Optionally, this is a good time to update the bvmhost and reboot it after destroying, but before installing builders
|
||||
. After installed, check that each builder is checking in with `koji list-hosts | grep <vmname>`
|
||||
. In a loop, reenable them
|
||||
|
||||
|
||||
|
||||
////
|
||||
From this comment below, everything is going to be deleted
|
||||
Keeping all commented for reference while writing a new SOP
|
||||
////
|
||||
|
||||
////
|
||||
= Setup Koji Builder SOP
|
||||
|
||||
== Contents
|
||||
|
||||
* <<_builder_setup>>
|
||||
** <<_network_overview>>
|
||||
** <<_setup_overview>>
|
||||
** <<_resettinginstalling_an_old_koji_builder>>
|
||||
|
||||
== Builder Setup
|
||||
|
||||
Setting up a new koji builder involves a goodly number of steps:
|
||||
|
||||
=== Network Overview
|
||||
|
||||
[arabic]
|
||||
. First get an instance spun up following the kickstart sop.
|
||||
. Define a hostname for it on the 125 network and a $hostname-nfs name
|
||||
for it on the .127 network.
|
||||
. make sure the instance has 2 network connections:
|
||||
* eth0 should be on the .125 network
|
||||
* eth1 should be on the .127 network
|
||||
+
|
||||
[NOTE]
|
||||
=====
|
||||
For VM _eth0_ should be on _br0_, _eth1_ on _br1_ on the vmhost.
|
||||
=====
|
||||
|
||||
=== Setup Overview
|
||||
|
||||
* install the system as normal:
|
||||
+
|
||||
....
|
||||
virt-install -n $builder_fqdn -r $memsize \
|
||||
-f $path_to_lvm --vcpus=$numprocs \
|
||||
-l http://10.5.126.23/repo/rhel/RHEL6-x86_64/ \
|
||||
-x "ksdevice=eth0 ks=http://10.5.126.23/repo/rhel/ks/kvm-rhel-6 \
|
||||
ip=$ip netmask=$netmask gateway=$gw dns=$dns \
|
||||
console=tty0 console=ttyS0" \
|
||||
--network=bridge=br0 --network=bridge=br1 \
|
||||
--vnc --noautoconsole
|
||||
....
|
||||
* run python `/root/tmp/setup-nfs-network.py` this should print out the
|
||||
-nfs hostname that you made above
|
||||
* change root pw
|
||||
* disable selinux on the machine in /etc/sysconfig/selinux
|
||||
* reboot
|
||||
* setup ssl cert into private/builders - use fqdn of host as DN
|
||||
** login to _fas01_ as root
|
||||
** `cd /var/lib/fedora-ca`
|
||||
** `./kojicerthelper.py normal --outdir=/tmp/ \ --name=$fqdn_of_the_new_builder --cadir=. --caname=Fedora`
|
||||
** info for the cert should be like this:
|
||||
+
|
||||
....
|
||||
Country Name (2 letter code) [US]:
|
||||
State or Province Name (full name) [North Carolina]:
|
||||
Locality Name (eg, city) [Raleigh]:
|
||||
Organization Name (eg, company) [Fedora Project]:
|
||||
Organizational Unit Name (eg, section) []:Fedora Builders
|
||||
Common Name (eg, your name or your servers hostname) []:$fqdn_of_new_builder
|
||||
Email Address []:buildsys@fedoraproject.org
|
||||
....
|
||||
** scp the file in `/tmp/$\{fqdn}_key_and_cert.pem` over to _batcave01_
|
||||
** put file in the private repo under `private/builders/$dn}.pem`
|
||||
** `git add` + `git commit`
|
||||
** `git push`
|
||||
* run `./sync-hosts` in infra-hosts repo; `git commit; git push`
|
||||
* as a koji admin run:
|
||||
+
|
||||
....
|
||||
koji add-host $fqdnr i386 x86_64
|
||||
|
||||
(note: those are yum basearchs on the end - season to taste)
|
||||
....
|
||||
|
||||
=== Resetting/installing an old koji builder
|
||||
|
||||
* disable the builder in koji (ask a koji admin)
|
||||
* halt the old system (`halt -p`)
|
||||
* undefine the vm instance on the buildvmhost:
|
||||
+
|
||||
....
|
||||
virsh undefine $builder_fqdn
|
||||
....
|
||||
* reinstall it - from the buildvmhost run:
|
||||
+
|
||||
....
|
||||
virt-install -n $builder_fqdn -r $memsize \
|
||||
-f $path_to_lvm --vcpus=$numprocs \
|
||||
-l http://10.5.126.23/repo/rhel/RHEL6-x86_64/ \
|
||||
-x "ksdevice=eth0 ks=http://10.5.126.23/repo/rhel/ks/kvm-rhel-6 \
|
||||
ip=$ip netmask=$netmask gateway=$gw dns=$dns \
|
||||
console=tty0 console=ttyS0" \
|
||||
--network=bridge=br0 --network=bridge=br1 \
|
||||
--vnc --noautoconsole
|
||||
....
|
||||
* watch install via vnc:
|
||||
+
|
||||
....
|
||||
vncviewer -via bastion.fedoraproject.org $builder_fqdn:1
|
||||
....
|
||||
* when the install finishes:
|
||||
** start the instance on the buildvmhost:
|
||||
+
|
||||
....
|
||||
virsh start $builder_fqdn
|
||||
....
|
||||
** set it to autostart on the buildvmhost:
|
||||
+
|
||||
....
|
||||
virsh autostart $builder_fqdn
|
||||
....
|
||||
* when the guest comes up
|
||||
** login via ssh using the temp root password
|
||||
** python `/root/tmp/setup-nfs-network.py`
|
||||
** change root password
|
||||
** disable selinux in `/etc/sysconfig/selinux`
|
||||
** `reboot`
|
||||
** ask a koji admin to re-enable the host
|
||||
////
|
||||
Loading…
Add table
Add a link
Reference in a new issue