Infrastructure/infra-docs-fpo
5
0
Fork 0
Code Issues 100 Pull requests 5 Projects Releases Packages Wiki Activity Actions
infra-docs-fpo/modules/release_guide/pages/sop_mass_branching.adoc
Timothée Ravier 65b13f371e sop_mass_branching: Improve formatting
2023-08-09 19:15:43 +02:00

582 lines
18 KiB
Text
Raw Blame History

include::_partials/attributes.adoc[]
= Mass Branching
At each alpha freeze we branch the pending release away from `devel/`
which allows rawhide (currently F{rawhide}) to move on while the pending release goes into
bugfix and polish mode.
You will find below the list of steps to follow to branch a new Fedora release.
== Send announcement
One day before the mass branching, we send out announcemt because during mass branching,
new koji builds for rawhide are disabled.
== Disable rawhide builds in koji
We need to configure outage in koji. Look at https://pagure.io/fedora-infra/ansible/pull-request/1528[this PR]
=== Cancel all running build for rawhide
List all running tasks and select onle those relevant for current branching.
[source,bash,subs="attributes"]
----
$ koji list-builds --state=0 --type=rpm | grep fc{branched} | awk '{print $1}'
----
Cancel each of those tasks
```
$ koji cancel <build>
```
== Repos to branch
All the following listed repos needs updating, including adding a new
branch for branched release and updating rawhide branch with new release
values.
* https://pagure.io/pungi-fedora
* https://pagure.io/fedora-kickstarts
* https://pagure.io/fedora-comps
* https://pagure.io/fedora-lorax-templates/
* https://pagure.io/workstation-ostree-config/
* https://src.fedoraproject.org/rpms/fedora-release
* https://src.fedoraproject.org/rpms/fedora-repos
== PDC
The "product-release" needs to be created in PDC.
In the `scripts/pdc/` directory, run:
```
$ python create-product-release.py fedora $TOKEN Fedora $NEW_VERSION
```
On `pdc-backend01.stg` (for testing) or `pdc-backend01` (for production)
clone (or update an existing one) the releng repo:
```
$ git clone https://pagure.io/releng.git
```
In the `scripts/pdc/` directory, run (see the `--help` of the script for
information on how to run it):
```
$ python create-new-release-branches.py ... --createfile
```
NOTE: The `--createfile` argument is necessary, that file is needed for the next step.
[NOTE]
====
Due to memory leak issue in pdc, we need to set the config in
/etc/pdc.d/fedora.json \{ "fedora": \{ "host":
"http://pdc-web02.iad2.fedoraproject.org/rest_api/v1/", "develop":
false, "ssl-verify": false, } }
====
== dist-git
Now that pdc has the new release and each package has been branched in
pdc we need to update dist-git in two steps:
* Create the new branch in git
* Update the gitolite.conf to allow user to push to this new branch
For both of these actions you will need the file generated by pdc above.
=== Create the git branches
On `pkgs01.stg` (for testing) or `pkgs02` (for production), run:
```
$ sudo -u pagure python /usr/local/bin/mass-branching-git.py <new branch name> <input file>
```
Where `<new branch name>` will be like `f{rawhide}` and the
`<input file>` the path to the file generated by pdc above.
== Ansible
Apps in https://pagure.io/fedora-infra/ansible[ansible] need to be updated to be aware of a new branch.
=== Bodhi
Bodhi needs to be updated to add new release. This needs to be done in
https://pagure.io/fedora-infra/ansible/blob/main/f/roles/bodhi2[bodhi2
role] in infra ansible repo. This change includes, updating
koji-sync-listener.py, new-updates-sync, pungi configs for rpm updates, bodhi templates.
....
- roles/bodhi2/backend/files/new-updates-sync
- roles/bodhi2/backend/tasks/main.yml
- roles/bodhi2/backend/templates/pungi.rpm.conf.j2
- roles/bodhi2/backend/templates/koji_sync_listener.toml
....
Please check these files from the https://pagure.io/fedora-infra/ansible/c/549e5d3ace41c04fdbef9d81f359f16c2fe0c2fa?branch=main[commit] for your reference.
=== Enable Branched Compose
We need to enable the branched compose. This is done in
https://pagure.io/fedora-infra/ansible/blob/main/f/roles/releng[releng
role] of infra ansbile repo.
Please check the file `roles/releng/files/branched` from the https://pagure.io/fedora-infra/ansible/c/549e5d3ace41c04fdbef9d81f359f16c2fe0c2fa?branch=main[commit] for your reference.
=== Fedora Branched
. Set FedoraBranched variable to True in infra ansible repo.
. Set FedoraBranchedBodhi variable to preenable in infra ansible repo.
Please check the file `FedoraBranched.yaml` and `FedoraBranchedBodhi.yaml` from the https://pagure.io/fedora-infra/ansible/c/549e5d3ace41c04fdbef9d81f359f16c2fe0c2fa?branch=main[commit] for your reference.
=== Koji hub
Update the koji hub config to allow side tags for new koji rawhide (currently f{rawhide}) tag.
Please check the file `roles/koji_hub/templates/hub.conf.j2` from the https://pagure.io/fedora-infra/ansible/c/549e5d3ace41c04fdbef9d81f359f16c2fe0c2fa?branch=main[commit] for your reference.
=== Robosignatory
Robosignatory has two parts, which can be found in
https://pagure.io/fedora-infra/ansible/blob/main/f/roles/robosignatory[robosignatory
role] in infra ansible repo.:
[arabic]
. Disable branched signing, so that we can freeze branched until we get
a compose.
. Adding new release.
Please check the file `roles/robosignatory/templates/robosignatory.toml.j2` from the https://pagure.io/fedora-infra/ansible/c/a7dcdbd794e728f7b80b1afc6c142c0196b8c1a3?branch=main[commit] for your reference.
=== Push the changes
When done editing the files, commit, push and apply them via the
corresponding ansible playbook:
```
$ sudo rbac-playbook groups/koji-hub.yml
$ sudo rbac-playbook groups/releng-compose.yml
$ sudo rbac-playbook groups/bodhi-backend.yml
$ sudo rbac-playbook openshift-apps/greenwave.yml
$ sudo -i ansible-playbook /srv/web/infra/ansible/playbooks/$ groups/proxies.yml -t pkgdb2
$ sudo rbac-playbook groups/mbs.yml -t mbs
```
Ask someone in fedora infra to run the robosignatory playbook.
== Koji
The koji build system needs to have some tag/target work done to handle
builds from the new branch and to update where builds from rawhide go.
Run
https://pagure.io/releng/blob/main/f/scripts/branching/make-koji-release-tags[make-koji-release-tags]
script in https://pagure.io/releng[pagure releng] repo
== Fedora Release
The `fedora-release` package needs to be updated in Rawhide and
Branched.
Changes to `fedora-release.spec` in the *rawhide* branch:
(can also check this https://src.fedoraproject.org/rpms/fedora-release/c/841f1fb8e5f7ff2b11a5e0905f15e765da405edd?branch=rawhide[commit] for reference)
[arabic]
. Increment `%define dist_version` to {rawhide}:
+
[source,subs="attributes"]
----
%define dist_version {rawhide}
----
. Increment `Version:` and reset `Release:`:
+
[source,subs="attributes"]
----
Version: {rawhide}
Release: 0.1%{?eln:.eln%{eln}}
----
. Add a `%changelog` entry:
+
[source,subs="attributes"]
----
%changelog
* Day Mon DD YYYY Name <Email> - {rawhide}-0.1
- Setup for rawhide being F{rawhide}
----
Changes to `fedora-release.spec` in the *branched* (currently {branched}) branch:
(can also check this https://src.fedoraproject.org/rpms/fedora-release/c/a2c289b44335eec89efaddee02d807b048edd51f?branch=f37[commit] for reference)
[arabic]
. Adjust `release_name` and unset `is_rawhide`:
+
[source,subs="attributes"]
----
%define release_name {branched_name}
%define is_rawhide 0
----
. Verify the correct number for `dist_version` and `Version:`:
+
[source,subs="attributes"]
----
%define dist_version {branched}
Version: {branched}
----
. Bump `Release:`:
+
[source,subs="attributes"]
----
Release: 0.4%{?eln:.eln%{eln}}
----
. Add a `%changelog` entry:
+
[source,subs="attributes"]
----
%changelog
* Day Mon DD YYYY Name <Email> - {branched}-0.4
- Branching F{branched} from rawhide
----
== Fedora Repos
The `fedora-repos` package needs to be updated in Rawhide, Branched, and
also in all stable release branches (in order to receive new GPG keys
and updated symlinks).
Changes to the *rawhide* branch (mostly in `fedora-repos.spec`):
(can also check this https://src.fedoraproject.org/rpms/fedora-repos/c/043010400f007c9f7167c9411d4463f4c2f08b9e?branch=rawhide[commit] for reference)
[arabic]
. Generate and add a _Rawhide+1_ which is {rawhide_next} GPG key file, then add it to the spec
file:
+
[source,subs="attributes"]
----
Source57: RPM-GPG-KEY-fedora-{rawhide_next}-primary
----
. Update the `archmap` file and define architectures for _Rawhide+1_:
+
[source,subs="attributes"]
----
fedora-{rawhide+1}-primary: x86_64 armhfp aarch64 ppc64le s390x
----
. Increment `%global rawhide_release`:
+
[source,subs="attributes"]
----
%global rawhide_release {rawhide}
----
. Bump `Version:` and reset `Release:`:
+
[source,subs="attributes"]
----
Version: {rawhide}
Release: 0.1%{?eln:.eln%{eln}}
----
. Add a `%changelog` entry:
+
[source,subs="attributes"]
----
%changelog
* Day Mon DD YYYY Name <Email> - {rawhide}-0.1
- Setup for rawhide being F{rawhide}
----
Changes to the *branched* branch (mostly in `fedora-repos.spec`):
(can also check this https://src.fedoraproject.org/rpms/fedora-repos/c/cd678e7b4aa59e234f32c9697198f58d9c89d862?branch=f38[commit] for reference)
[arabic]
. Copy the _Rawhide+1_ which is {rawhide_next} GPG key file from the _rawhide_ branch, then add
it to the spec file:
+
[source,subs="attributes"]
----
Source57: RPM-GPG-KEY-fedora-{rawhide_next}-primary
----
. Copy the `archmap` file from the _rawhide_ branch.
. Update `%global rawhide_release`:
+
[source,subs="attributes"]
----
%global rawhide_release {rawhide}
----
. Enable `updates_testing_enabled`:
+
[source,subs="attributes"]
----
%global updates_testing_enabled 1
----
. Bump `Release`:
+
[source,subs="attributes"]
----
Release: 0.3%{?eln:.eln%{eln}} +
----
. Add a `%changelog` entry:
+
[source,subs="attributes"]
----
%changelog
*Day Mon DD YYYY Name <Email> - {branched}-0.3 +
- Update Rawhide definition, enable updates-testing for Branched +
----
[NOTE]
====
Build `fedora-release` and `fedora-repos` packages for Branched release
*before enabling the Rawhide gating*.
====
Changes to the *stable* branches (mostly in `fedora-repos.spec`):
[arabic]
. Copy the _Rawhide+1_ GPG key which is {rawhide_next} file from the _rawhide_ branch, then add
it to the spec file:
+
[source,subs="attributes"]
----
Source57: RPM-GPG-KEY-fedora-{rawhide_next}-primary
----
. Copy the `archmap` file from the _rawhide_ branch.
. Update `%global rawhide_release`:
+
[source,subs="attributes"]
----
%global rawhide_release {rawhide}
----
. Bump `Release:`:
+
[source,subs="attributes"]
----
Release: 0.3%{?eln:.eln%{eln}}
----
. Add a `%changelog` entry:
+
[source,subs="attributes"]
----
%changelog
*Day Mon DD YYYY Name <Email> - {current}-0.3
- Update Rawhide definition
----
== Bodhi
=== Linking Empty Repos
We need to link empty repos so that new-updates-sync wont complain about
missing repos. The following commands should be run on
*bodhi-backend01.phx2.fedoraproject.org*
[source,bash,subs="attributes"]
----
$ sudo ln -s /mnt/koji/compose/updates/empty-repo/ /mnt/koji/compose/updates/f{branched}-updates
$ sudo ln -s /mnt/koji/compose/updates/empty-repo/ /mnt/koji/compose/updates/f{branched}-updates-testing
----
=== Creating Empty Repos
To create empty repos on the master mirror, run
https://pagure.io/releng/blob/main/f/scripts/branching/create_empty_repos.sh[create_emtpy_repos.sh]
from https://pagure.io/releng[pagure releng] repo. This should be run on
*bodhi-backend01.phx2.fedoraproject.org*
[source,bash,subs="attributes"]
----
$ sudo -u ftpsync sh scripts/branching/create_empty_repos.sh {branched}
----
[NOTE]
====
Please verify the repo permissions that are created under
/pub/fedora/linux/development/<fedora_release_number> and
/pub/fedora-secondary/development/<fedora_release_number>. They should
be owned by _ftpsync:ftpsync_
====
=== Creating rawhide release
To create a rawhide release in bodhi, you need to run
[attributes]
``
$ bodhi releases create --name "F{rawhide}" --long-name "Fedora {rawhide}" --id-prefix FEDORA --version {rawhide} --branch f{rawhide} --dist-tag f{rawhide} --stable-tag f{rawhide} --testing-tag f{rawhide}-updates-testing --candidate-tag f{rawhide}-updates-candidate --pending-stable-tag f{rawhide}-updates-pending --pending-testing-tag f{rawhide}-updates-testing-pending --pending-signing-tag f{rawhide}-signing-pending --state pending --override-tag f{rawhide}-override --create-automatic-updates --not-composed-by-bodhi
``
To create a container release for rawhide in bodhi, you need to run
[attributes]
``
$ bodhi releases create --name "F{rawhide}C" --long-name "Fedora {rawhide} Containers" --id-prefix FEDORA-CONTAINER --version {rawhide} --branch f{rawhide} --dist-tag f{rawhide}-container --stable-tag f{rawhide}-container-updates --testing-tag f{rawhide}-container-updates-testing --candidate-tag f{rawhide}-container-updates-candidate --pending-stable-tag f{rawhide}-container-updates-pending --pending-testing-tag f{rawhide}-container-updates-testing-pending --state pending --override-tag f{rawhide}-container-override
``
To create a flatpak release for branched in bodhi, you need to run
[attributes]
``
$ bodhi releases create --name "F{branched}F" --long-name "Fedora {branched} Flatpaks" --id-prefix FEDORA-FLATPAK --version {branched} --branch f{branched} --dist-tag f{branched}-flatpak --stable-tag f{branched}-flatpak-updates --testing-tag f{branched}-flatpak-updates-testing --candidate-tag f{branched}-flatpak-updates-candidate --pending-stable-tag f{branched}-flatpak-updates-pending --pending-testing-tag f{branched}-flatpak-updates-testing-pending --state pending --override-tag f{branched}-flatpak-override
``
You need to run the `bodhi openshift` playbook, so that UI will know
about the new release. Then, you need to restart
*fm-consumer@config.service* and *bodhi-celery.service* services on
*bodhi-backend01.phx2.fedoraproject.org*
``
$ sudo rbac-playbook openshift-apps/bodhi.yml
$ sudo systemctl restart fm-consumer@config.service bodhi-celery.service
``
[NOTE]
====
Build fedora-release, fedora-repos package for *rawhide after enabling
the rawhide gating*
====
=== Update rawhide koji repo
We need to point the _rawhide_ buildroot repo to the newly created
rawhide buildroot. This way kojira doesn't make a newrepo for _rawhide_
target as often as fxx-build (new rawhide buildroot).
Run the following commands from any of the compose boxes:
[source,bash,subs="attributes"]
----
$ cd /mnt/koji/repos/rawhide
$ rm -f latest
$ ln -s ../f{rawhide}-build/latest ./latest
----
=== Update block_retired.py script
https://pagure.io/releng/blob/main/f/scripts/block_retired.py[block_retired.py]
script in releng repo should be updated with rawhide release and also
branched release should be added to the script.
Please look at this
https://pagure.io/releng/c/9eb97f491f7a767ab8b90498adfa3b34ee235247?branch=main[block_retired.py
commit] as an example.
=== Updating MirrorManager
We need to update the mirrormanager so that it will point rawhide to the
new rawhide release.
Please follow the instructions in the
https://pagure.io/fedora-infrastructure/issue/9239#comment-671446[fedora
infra ticket] to update the database of mirrormanager.
=== Enable autosigning on branched release
Once the branched compose is composed, we need to re-enable
robosignatory on branched release
=== ELN related work
Add the new rawhide key to eln pungi config. For example, look at this
https://pagure.io/pungi-fedora/c/e993441164ee83374df7f463777f2bf1d456fd6d?branch=eln[pungi
eln config commit]
Change the trigger notification for DistroBuildSync to the new Rawhide
version. For example, look at this
https://gitlab.com/redhat/centos-stream/ci-cd/distrosync/distrobuildsync-config/-/commit/1497d9aea42cf00af646b4a0f9f9ed1a7f0a477f[commit].
=== Branch new rawhide in Koschei
Branch new fedora rawhide in
https://docs.fedoraproject.org/en-US/infra/sysadmin_guide/koschei/#_branching_a_new_fedora_release[koschei].
== Fedora Container Base Image
In order to enable builds for Container Base Images via the
https://docs.pagure.org/releng/layered_image_build_service.html[Fedora
Layered Image Build System] we will need to import a new image for
Rawhide as well as for the new `fedora:rawhide` and `fedora:${RAWHIDE}`
tags.
Check for the latest successful Rawhide Base Image composed image
https://koji.fedoraproject.org/koji/packageinfo?packageID=21546[here].
On `compose-x86-01.phx2` run:
[source,bash,subs="attributes"]
----
# Update this to be the correct URL for your image
$ BASEIMAGE_URL="https://kojipkgs.fedoraproject.org//packages/Fedora-Docker-Base/Rawhide/20170310.n.0/images/Fedora-Docker-Base-Rawhide-20170310.n.0.x86_64.tar.xz"
# Update this to whatever version number Rawhide now points to
$ RAWHIDE="27"
# Load the latest, find it's image name
$ sudo docker load < <(curl -s "${BASEIMAGE_URL}")
$ sudo docker images | grep base-rawhide
fedora-docker-base-rawhide-20170310.n.0.x86_64 latest ffd832a990ca 5 hours ago 201.8 MB
# Tag everything
$ sudo docker tag fedora-docker-base-rawhide-20170310.n.0.x86_64 candidate-registry.fedoraproject.org/fedora:rawhide
$ sudo docker tag fedora-docker-base-rawhide-20170310.n.0.x86_64 candidate-registry.fedoraproject.org/fedora:${RAWHIDE}
$ sudo docker tag fedora-docker-base-rawhide-20170310.n.0.x86_64 registry.fedoraproject.org/fedora:rawhide
$ sudo docker tag fedora-docker-base-rawhide-20170310.n.0.x86_64 registry.fedoraproject.org/fedora:${RAWHIDE}
# Push the images
$ sudo docker push candidate-registry.fedoraproject.org/fedora:rawhide
$ sudo docker push candidate-registry.fedoraproject.org/fedora:${RAWHIDE}
$ sudo docker push registry.fedoraproject.org/fedora:rawhide
$ sudo docker push registry.fedoraproject.org/fedora:${RAWHIDE}
# Clean up after ourselves
$ sudo docker rmi fedora-docker-base-rawhide-20170310.n.0.x86_64
Untagged: fedora-docker-base-rawhide-20170310.n.0.x86_64:latest
$ for i in $(sudo docker images -q -f 'dangling=true'); do sudo docker rmi $i; done
----
=== Update sync script
In releng repository update
https://pagure.io/releng/blob/main/f/scripts/sync-latest-container-base-image.sh#_38[script].
And set current_rawhide variable.
== Toddlers
=== Add new SLA to the toddlers App
Use https://pagure.io/fedora-infra/ansible/pull-request/1527[this PR] for reference and add new version to the config.
= Consider Before Running
[NOTE]
====
FIXME: Need some love here
====
Reference in a new issue View git blame Copy permalink