Review fedora-releases SOP

Signed-off-by: Michal Konečný <mkonecny@redhat.com>
This commit is contained in:
Michal Konečný 2021-08-19 15:06:42 +02:00
parent d83da1965b
commit 7ea4b6ed0f
2 changed files with 61 additions and 57 deletions

View file

@ -33,7 +33,7 @@
** xref:fedmsg-relay.adoc[fedmsg-relay - SOP] ** xref:fedmsg-relay.adoc[fedmsg-relay - SOP]
** xref:fedmsg-websocket.adoc[WebSocket - SOP] ** xref:fedmsg-websocket.adoc[WebSocket - SOP]
** xref:fedocal.adoc[Fedocal - SOP] ** xref:fedocal.adoc[Fedocal - SOP]
** xref:fedora-releases.adoc[fedora-releases - SOP in review ] ** xref:fedora-releases.adoc[Fedora Release Infrastructure - SOP]
** xref:fedorawebsites.adoc[fedorawebsites - SOP in review ] ** xref:fedorawebsites.adoc[fedorawebsites - SOP in review ]
** xref:fmn.adoc[fmn - SOP in review ] ** xref:fmn.adoc[fmn - SOP in review ]
** xref:gather-easyfix.adoc[gather-easyfix - SOP in review ] ** xref:gather-easyfix.adoc[gather-easyfix - SOP in review ]

View file

@ -25,35 +25,41 @@ Before a release ships, the following items need to be completed.
[arabic] [arabic]
. New website from the websites team (typically hosted at . New website from the websites team (typically hosted at
http://getfedora.org/_/) https://getfedora.org/_/)
. Verify mirror space (for all test releases as well) . Verify mirror space (for all test releases as well)
. Verify with rel-eng permissions on content are right on the mirrors. . Verify with rel-eng permissions on content are right on the mirrors.
Don't leak. Don't leak.
. {blank}
+ . Communication with Red Hat IS (Give at least 2 months notice, then
Communication with Red Hat IS (Give at least 2 months notice, then:: reminders as the time comes near) (final release only)
reminders as the time comes near) (final release only)
. Infrastructure change freeze . Infrastructure change freeze
. Modify Template:FedoraVersion to reference new version. (Final release
. Modify _Template:FedoraVersion_ to reference new version. (Final release
only) only)
. Move old releases to archive (post final release only) . Move old releases to archive (post final release only)
. {blank}
+ . Switch release from development/N to normal releases/N/ tree in mirror
Switch release from development/N to normal releases/N/ tree in mirror:: manager (post final release only)
manager (post final release only)
== Change Freeze == Change Freeze
The rules are simple: The rules are simple:
* Hosts with the ansible variable "freezes" "True" are frozen. * Hosts with the ansible variable "freezes" "True" are frozen.
* You may make changes as normal on hosts that are not frozen. (For * You may make changes as normal on hosts that are not frozen. (For
example, staging is never frozen) example, staging is never frozen)
* Changes to frozen hosts requires a freeze break request sent to the * Changes to frozen hosts requires a freeze break request sent to the
fedora infrastructure list, containing a description of the problem or fedora infrastructure list, containing a description of the problem or
issue, actions to be taken and (if possible) patches to ansible that issue, actions to be taken and (if possible) patches to ansible that
will be applied. These freeze breaks must then get two approvals from will be applied. These freeze breaks must then get two approvals from
sysadmin-main or sysadmin-releng group members before being applied. sysadmin-main or sysadmin-releng group members before being applied.
* Changes to recover from outages are acceptable to frozen hosts if * Changes to recover from outages are acceptable to frozen hosts if
needed. needed.
@ -79,7 +85,6 @@ mirrors it is our job to make sure everything else (except the bit flip)
gets done as painlessly and easily as possible. gets done as painlessly and easily as possible.
[NOTE] [NOTE]
.Note
==== ====
All communication is typically done in #fedora-admin. Typically these All communication is typically done in #fedora-admin. Typically these
channels are laid back and staying on topic isn't strictly enforced. On channels are laid back and staying on topic isn't strictly enforced. On
@ -96,7 +101,7 @@ were offline for hours. Some (like the docs) were off for days. A large
part of this outage was due to the wiki not being able to handle the part of this outage was due to the wiki not being able to handle the
load, part was a lack of planning by the Infrastructure team, and part load, part was a lack of planning by the Infrastructure team, and part
is still a mystery. There are questions as to whether or not all of the is still a mystery. There are questions as to whether or not all of the
traffic was legit or a ddos. traffic was legit or a DDoS.
The Fedora 7 release went much better. Some services were offline for The Fedora 7 release went much better. Some services were offline for
minutes at a time but very little of it was out longer then that. The minutes at a time but very little of it was out longer then that. The
@ -160,8 +165,7 @@ or not we release and they may pull the plug at any moment.
Once given the ok to release, the Infrastructure team should publish the Once given the ok to release, the Infrastructure team should publish the
torrent and encourage people to seed. Complete the steps on the torrent and encourage people to seed. Complete the steps on the
https://fedora-infra-docs.readthedocs.io/en/latest/sysadmin-guide/sops/torrentrelease.html <<torrentrelease.adoc#>> after step 4.
after step 4.
=== Step 3 (Bit flip) === Step 3 (Bit flip)
@ -176,7 +180,7 @@ spot check. Once that is complete move on.
Once all of the distribution pieces are verified (mirrors and torrent), Once all of the distribution pieces are verified (mirrors and torrent),
all that is left is to publish the website. At present this is done by all that is left is to publish the website. At present this is done by
making sure the master branch of fedora-web is pulled by the making sure the master branch of fedora-web is pulled by the
syncStatic.sh script in ansible. It will sync in an hour normally but on `syncStatic.sh` script in ansible. It will sync in an hour normally but on
release day people don't like to wait that long so do the following on release day people don't like to wait that long so do the following on
sundries01 sundries01
@ -185,7 +189,7 @@ sudo -u apache /usr/local/bin/lock-wrapper syncStatic 'sh -x
/usr/local/bin/syncStatic' /usr/local/bin/syncStatic'
____ ____
Once that completes, on batcave01: Once that completes, on _batcave01_:
.... ....
sudo -i ansible proxy\* "/usr/bin/rsync --delete -a --no-owner --no-group bapp02::getfedora.org/ /srv/web/getfedora.org/" sudo -i ansible proxy\* "/usr/bin/rsync --delete -a --no-owner --no-group bapp02::getfedora.org/ /srv/web/getfedora.org/"
@ -251,43 +255,43 @@ mirrors.
Priorities of during release day (In order): Priorities of during release day (In order):
[arabic] [arabic]
. {blank} . Website
+ +
Website:: Anything related to a user landing at fedoraproject.org, and clicking
Anything related to a user landing at fedoraproject.org, and clicking through to a mirror or torrent to download something must be kept up.
through to a mirror or torrent to download something must be kept up. This is distribution, and without it we can potentially lose many
This is distribution, and without it we can potentially lose many users.
users.
. {blank}
+ +
Linked addresses:: . Linked addresses
We do not have direct control over what Hacker News, Phoronix or
anyone else links to. If they link to something on the wiki and it is
going down or link to any other site we control a rewrite should be
put in place to direct them to http://fedoraproject.org/get-fedora.
. {blank}
+ +
Torrent:: We do not have direct control over what Hacker News, Phoronix or
The torrent server has never had problems during a release. Make sure anyone else links to. If they link to something on the wiki and it is
it is up. going down or link to any other site we control a rewrite should be
. {blank} put in place to direct them to http://fedoraproject.org/get-fedora.
+ +
Release Notes:: . Torrent
Typically grouped with the docs site, the release notes are often
linked to (this is fine, no need to redirect) but keep an eye on the
logs and ensure that where we've said the release notes are, that they
can be found there. In previous releases we sometimes had to make this
available in more than one spot.
. {blank}
+ +
docs.fedoraproject.org:: The torrent server has never had problems during a release. Make sure
People will want to see whats new in Fedora and get further it is up.
documentation about it. Much of this is in the release notes. +
. {blank} . Release Notes
+
Typically grouped with the docs site, the release notes are often
linked to (this is fine, no need to redirect) but keep an eye on the
logs and ensure that where we've said the release notes are, that they
can be found there. In previous releases we sometimes had to make this
available in more than one spot.
+
. docs.fedoraproject.org
+
People will want to see whats new in Fedora and get further
documentation about it. Much of this is in the release notes.
+
. wiki
+
Because it is so resource heavy, and because it is so developer
oriented we have no choice but to give the wiki a lower priority.
+ +
wiki::
Because it is so resource heavy, and because it is so developer
oriented we have no choice but to give the wiki a lower priority.
. Everything else. . Everything else.
== Juggling Resources == Juggling Resources
@ -321,10 +325,10 @@ After release is a "go":
* Make sure torrents are setup and ready to go. * Make sure torrents are setup and ready to go.
* fedora-web needs a branch for fN-beta. In it: * fedora-web needs a branch for fN-beta. In it:
* Beta used on get-prerelease ** Beta used on get-prerelease
* get-prerelease doesn't direct to release ** get-prerelease doesn't direct to release
* verify is updated with Beta info ** verify is updated with Beta info
* releases.txt gets a branched entry for preupgrade ** releases.txt gets a branched entry for pre-upgrade
* bfo gets updated to have a Beta entry. * bfo gets updated to have a Beta entry.
After release: After release:
@ -345,8 +349,8 @@ After release is a "go":
* Make sure torrents are setup and ready to go. * Make sure torrents are setup and ready to go.
* fedora-web needs a branch for fN-alpha. In it: * fedora-web needs a branch for fN-alpha. In it:
* get-prerelease does direct to release ** get-prerelease does direct to release
* verify is updated with Final info ** verify is updated with Final info
* bfo gets updated to have a Final entry. * bfo gets updated to have a Final entry.
* update wiki version numbers and names. * update wiki version numbers and names.
@ -354,7 +358,7 @@ After release:
* Update /topic in #fedora-admin * Update /topic in #fedora-admin
* post to infrastructure list that freeze is over. * post to infrastructure list that freeze is over.
* Move MirrorManager repository tags from the development/$version/ * Move MirrorManager repository tags from the `development/$version/`
Directory objects, to the releases/$version/ Directory objects. This is Directory objects, to the `releases/$version/` Directory objects. This is
done using the `move-devel-to-release --version=$version` command on done using the `move-devel-to-release --version=$version` command on
bapp02. This is usually done now a week or two after release. _bapp02_. This is usually done now a week or two after release.