Infrastructure/infra-docs-fpo
3
0
Fork 0
Code Issues 100 Pull requests 6 Projects Releases Packages Wiki Activity Actions
infra-docs-fpo/modules/release_guide/pages/release_eol.adoc
Adam Williamson 070b9a24fc EOL SOP: various updates 
* Link to docs.fp.o not the wiki
* Send announcements to test@ as well as test-announce@
* Only update fedora-appstream-metadata package if necessary
* Don't use bullets in sections with only one instruction
* Drop the FNNM release from Bodhi commands (no longer exists)
* Clean up the flow of ansible section a bit
* Correct syntax for the rbac-playbook command

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-11-26 09:23:33 -08:00

248 lines
7.3 KiB
Text
Raw Blame History

include::_partials/attributes.adoc[]
= End Of Life
== Description
Each release of Fedora is maintained as laid out in the
https://docs.fedoraproject.org/en-US/releases/lifecycle/#_maintenance_schedule[maintenance
schedule]. At the conclusion of the maintenance period, a Fedora release
enters `end of life` status. This procedure describes the tasks
necessary to move a release to that status. To get the correct EOL date visit
https://fedorapeople.org/groups/schedule/f-{branched}/f-{branched}-key-tasks.html[release schedule] for the last release.
== Actions
=== Reminder announcement
Around a week before the EOL date, end an email to `devel@`, `devel-announce@`, `test@`, `test-announce@`, `announce@` lists as remainder about the release EOL.
Use the https://pagure.io/releng/blob/main/f/mail-templates/08-end-of-life.txt[template] from release engineering repo.
[NOTE]
====
Please ensure that the reminder email is scheduled to be sent one week before the end of life.
====
=== `fedora-appstream-metadata` package update
If necessary, update the
https://src.fedoraproject.org/rpms/fedora-appstream-metadata[`fedora-appstream-metadata`]
package with the new EOL date. Follow the instructions from the repo README.
Build the package, create an update and make sure it lands in the stable repo
before proceeding.
This is not necessary if the package already has the correct EOL date.
=== Koji tasks
* Disable builds by removing targets
[source,subs="attributes+"]
....
$ koji remove-target f{old_release}
$ koji remove-target f{old_release}-candidate
$ koji remove-target f{old_release}-container-candidate
$ koji remove-target f{old_release}-flatpak-candidate
$ koji remove-target f{old_release}-infra
$ koji remove-target f{old_release}-coreos-continuous
$ koji remove-target f{old_release}-rebuild
....
* Remove sidetags
Remove any existing sidetags from the eol release.
[source,subs="attributes+"]
....
for i in `koji list-sidetags --basetag f{old_release}`
do
koji remove-sidetag $i
done
....
You need to do this differently because just removing the target of
a sidetag will leave it in a weird state where it cannot be removed.
* Purge from disk the signed copies of rpms that are signed with the
EOL'd release key. To acheive this, add the release key to
*koji_cleanup_signed.py* script in https://pagure.io/releng[releng] repo
and the script on compose-branched01.iad2.fedoraproject.org
....
./scripts/koji_cleanup_signed.py
....
=== Final stable push
Do a final stable push for the release in Bodhi, following the
xref:releng_misc_guide:sop_pushing_updates.adoc[Pushing Updates SOP].
=== Bodhi tasks
Run the following bodhi commands to set the releases state to
*archived*
[source,subs="attributes+"]
....
$ bodhi releases edit --name "F{old_release}" --state archived
$ bodhi releases edit --name "F{old_release}C" --state archived
$ bodhi releases edit --name "F{old_release}F" --state archived
....
=== Fedora Infra Ansible Changes
We need to make some changes and then run several roles in the ansible repo.
==== Update FedoraPreviousPrevious.yaml in ansible repository
Edit /vars/all/FedoraPreviousPrevious.yaml and set the value to `False`.
==== Adjust the EOL for gnome-software
Edit /roles/pkgdb-proxy/files/pkgdb-gnome-software-collections.json in ansible
and set the release to status "EOL".
==== Run the playbooks
First, push the above edits. Then run the associated playbooks on _batcave01_:
....
$ sudo ansible-playbook /srv/web/infra/ansible/playbooks/groups/bodhi-backend.yml
$ sudo ansible-playbook /srv/web/infra/ansible/playbooks/groups/koji-hub.yml
$ sudo ansible-playbook /srv/web/infra/ansible/playbooks/groups/releng-compose.yml
$ sudo ansible-playbook /srv/web/infra/ansible/playbooks/groups/proxies.yml -t pkgdb2
$ sudo ansible-playbook /srv/web/infra/ansible/playbooks/manual/autosign.yml
$ sudo ansible-playbook /srv/web/infra/ansible/playbooks/openshift-apps/bodhi.yml
....
[NOTE]
====
Another way to run the playbook is using rbac-playbook, in case you don't have sysadmin-main rights or can't become root.
Syntax: sudo rbac-playbook groups/bodhi-backend.yml
====
=== Final announcement
Send the final announcement to `devel@`, `devel-announce@`, `test@`, `test-announce@`, `announce@` lists.
Use the https://pagure.io/releng/blob/main/f/mail-templates/08-end-of-life.txt[template] from release engineering repo.
==== stage the EOL release to archive
. Log into to bodhi-backend01 and become root
+
____
....
$ ssh bodhi-backend01.iad2.fedoraproject.org
$ sudo su
$ su - ftpsync
....
____
. Then change into the releases directory.
+
____
....
$ cd /pub/fedora/linux/releases
....
____
. Check to see that the target directory doesnt already exist.
+
____
....
$ ls /pub/archive/fedora/linux/releases/
....
____
. Do a recursive rsync to update any changes in the trees since the previous
copy.
+
____
[source,subs="attributes+"]
....
$ rsync -avAXSHP ./{old_release}/ /pub/archive/fedora/linux/releases/{old_release}/
....
____
. We will now do the updates and updates/testing in similar ways.
+
____
[source,subs="attributes+"]
....
$ cd ../updates/
$ rsync -avAXSHP {old_release}/ /pub/archive/fedora/linux/updates/{old_release}/
$ cd testing
$ rsync -avAXSHP {old_release}/ /pub/archive/fedora/linux/updates/testing/{old_release}/
....
____
. Do the same with fedora-secondary.
+
____
[source,subs="attributes+"]
....
$ cd /pub/fedora-secondary/releases/
$ rsync -avAXSHP ./{old_release}/ /pub/archive/fedora-secondary/releases/{old_release}/
$ cd ../updates/
$ rsync -avAXSHP {old_release}/ /pub/archive/fedora-secondary/updates/{old_release}/
$ cd testing
$ rsync -avAXSHP {old_release}/ /pub/archive/fedora-secondary/updates/testing/{old_release}/
....
____
. Announce to the mirror list this has been done and that in 2 weeks you will
move the old trees to archives.
==== Move the EOL release to archive
. In two weeks, run the following playbook from batcave:
____
[source,subs="attributes+"]
....
$ rbac-playbook -v /srv/web/infra/ansible/playbooks/manual/mirrormanager/move-to-archive.yml --extra-vars="product='Fedora' version='{old_release}'"
....
____
. If there are problems, the postgres DB may have issues and so you need
to get a DBA to update the backend to fix items.
. Wait an hour or so then you can remove the files from the main tree.
[NOTE]
====
We need to do a cleanup here for both fedora and fedora-secondary for releases, updates, and updates/testing for the {old_release} release.
====
____
[source,subs="attributes+"]
....
$ ssh bodhi-backend01
$ cd /pub/fedora/linux/releases/{old_release}
$ ls # make sure you have stuff here
$ rm -rf *
$ ln ../20/README .
$ cd /pub/fedora/linux/updates/{old_release}
$ ls # make sure you have stuff here
$ rm -rf *
$ ln ../20/README .
$ cd /pub/fedora/linux/updates/testing/{old_release}
$ ls # make sure you have stuff here
$ rm -rf *
$ ln ../20/README .
#for fedora-secondary
$ cd /pub/fedora-secondary/releases/{old_release}
$ ls # make sure you have stuff here
$ rm -rf *
$ ln ../20/README .
$ cd /pub/fedora-secondary/updates/{old_release}
$ ls # make sure you have stuff here
$ rm -rf *
$ ln ../20/README .
$ cd /pub/fedora-secondary/updates/testing/{old_release}
$ ls # make sure you have stuff here
$ rm -rf *
$ ln ../20/README .
....
____
== Consider Before Running
* Resource contention in infrastructure, such as outages
* Extenuating circumstances for specific planned updates, if any
* Send the reminder announcement, if it isn't sent already
Reference in a new issue View git blame Copy permalink