From 7ea4b6ed0f0ae73a33cfba20a33f8780209f2acc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Michal=20Kone=C4=8Dn=C3=BD?= Date: Thu, 19 Aug 2021 15:06:42 +0200 Subject: [PATCH] Review fedora-releases SOP MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Michal Konečný --- modules/sysadmin_guide/nav.adoc | 2 +- .../sysadmin_guide/pages/fedora-releases.adoc | 116 +++++++++--------- 2 files changed, 61 insertions(+), 57 deletions(-) diff --git a/modules/sysadmin_guide/nav.adoc b/modules/sysadmin_guide/nav.adoc index 8c358ae..4607e9b 100644 --- a/modules/sysadmin_guide/nav.adoc +++ b/modules/sysadmin_guide/nav.adoc @@ -33,7 +33,7 @@ ** xref:fedmsg-relay.adoc[fedmsg-relay - SOP] ** xref:fedmsg-websocket.adoc[WebSocket - SOP] ** xref:fedocal.adoc[Fedocal - SOP] -** xref:fedora-releases.adoc[fedora-releases - SOP in review ] +** xref:fedora-releases.adoc[Fedora Release Infrastructure - SOP] ** xref:fedorawebsites.adoc[fedorawebsites - SOP in review ] ** xref:fmn.adoc[fmn - SOP in review ] ** xref:gather-easyfix.adoc[gather-easyfix - SOP in review ] diff --git a/modules/sysadmin_guide/pages/fedora-releases.adoc b/modules/sysadmin_guide/pages/fedora-releases.adoc index d97dfac..1206bed 100644 --- a/modules/sysadmin_guide/pages/fedora-releases.adoc +++ b/modules/sysadmin_guide/pages/fedora-releases.adoc @@ -25,35 +25,41 @@ Before a release ships, the following items need to be completed. [arabic] . New website from the websites team (typically hosted at -http://getfedora.org/_/) +https://getfedora.org/_/) + . Verify mirror space (for all test releases as well) + . Verify with rel-eng permissions on content are right on the mirrors. Don't leak. -. {blank} -+ -Communication with Red Hat IS (Give at least 2 months notice, then:: - reminders as the time comes near) (final release only) + +. Communication with Red Hat IS (Give at least 2 months notice, then +reminders as the time comes near) (final release only) + . Infrastructure change freeze -. Modify Template:FedoraVersion to reference new version. (Final release + +. Modify _Template:FedoraVersion_ to reference new version. (Final release only) + . Move old releases to archive (post final release only) -. {blank} -+ -Switch release from development/N to normal releases/N/ tree in mirror:: - manager (post final release only) + +. Switch release from development/N to normal releases/N/ tree in mirror +manager (post final release only) == Change Freeze The rules are simple: * Hosts with the ansible variable "freezes" "True" are frozen. + * You may make changes as normal on hosts that are not frozen. (For example, staging is never frozen) + * Changes to frozen hosts requires a freeze break request sent to the fedora infrastructure list, containing a description of the problem or issue, actions to be taken and (if possible) patches to ansible that will be applied. These freeze breaks must then get two approvals from sysadmin-main or sysadmin-releng group members before being applied. + * Changes to recover from outages are acceptable to frozen hosts if needed. @@ -79,7 +85,6 @@ mirrors it is our job to make sure everything else (except the bit flip) gets done as painlessly and easily as possible. [NOTE] -.Note ==== All communication is typically done in #fedora-admin. Typically these channels are laid back and staying on topic isn't strictly enforced. On @@ -96,7 +101,7 @@ were offline for hours. Some (like the docs) were off for days. A large part of this outage was due to the wiki not being able to handle the load, part was a lack of planning by the Infrastructure team, and part is still a mystery. There are questions as to whether or not all of the -traffic was legit or a ddos. +traffic was legit or a DDoS. The Fedora 7 release went much better. Some services were offline for minutes at a time but very little of it was out longer then that. The @@ -160,8 +165,7 @@ or not we release and they may pull the plug at any moment. Once given the ok to release, the Infrastructure team should publish the torrent and encourage people to seed. Complete the steps on the -https://fedora-infra-docs.readthedocs.io/en/latest/sysadmin-guide/sops/torrentrelease.html -after step 4. +<> after step 4. === Step 3 (Bit flip) @@ -176,7 +180,7 @@ spot check. Once that is complete move on. Once all of the distribution pieces are verified (mirrors and torrent), all that is left is to publish the website. At present this is done by making sure the master branch of fedora-web is pulled by the -syncStatic.sh script in ansible. It will sync in an hour normally but on +`syncStatic.sh` script in ansible. It will sync in an hour normally but on release day people don't like to wait that long so do the following on sundries01 @@ -185,7 +189,7 @@ sudo -u apache /usr/local/bin/lock-wrapper syncStatic 'sh -x /usr/local/bin/syncStatic' ____ -Once that completes, on batcave01: +Once that completes, on _batcave01_: .... sudo -i ansible proxy\* "/usr/bin/rsync --delete -a --no-owner --no-group bapp02::getfedora.org/ /srv/web/getfedora.org/" @@ -251,43 +255,43 @@ mirrors. Priorities of during release day (In order): [arabic] -. {blank} +. Website + -Website:: - Anything related to a user landing at fedoraproject.org, and clicking - through to a mirror or torrent to download something must be kept up. - This is distribution, and without it we can potentially lose many - users. -. {blank} +Anything related to a user landing at fedoraproject.org, and clicking +through to a mirror or torrent to download something must be kept up. +This is distribution, and without it we can potentially lose many +users. + -Linked addresses:: - We do not have direct control over what Hacker News, Phoronix or - anyone else links to. If they link to something on the wiki and it is - going down or link to any other site we control a rewrite should be - put in place to direct them to http://fedoraproject.org/get-fedora. -. {blank} +. Linked addresses + -Torrent:: - The torrent server has never had problems during a release. Make sure - it is up. -. {blank} +We do not have direct control over what Hacker News, Phoronix or +anyone else links to. If they link to something on the wiki and it is +going down or link to any other site we control a rewrite should be +put in place to direct them to http://fedoraproject.org/get-fedora. + -Release Notes:: - Typically grouped with the docs site, the release notes are often - linked to (this is fine, no need to redirect) but keep an eye on the - logs and ensure that where we've said the release notes are, that they - can be found there. In previous releases we sometimes had to make this - available in more than one spot. -. {blank} +. Torrent + -docs.fedoraproject.org:: - People will want to see whats new in Fedora and get further - documentation about it. Much of this is in the release notes. -. {blank} +The torrent server has never had problems during a release. Make sure +it is up. ++ +. Release Notes ++ +Typically grouped with the docs site, the release notes are often +linked to (this is fine, no need to redirect) but keep an eye on the +logs and ensure that where we've said the release notes are, that they +can be found there. In previous releases we sometimes had to make this +available in more than one spot. ++ +. docs.fedoraproject.org ++ +People will want to see whats new in Fedora and get further +documentation about it. Much of this is in the release notes. ++ +. wiki ++ +Because it is so resource heavy, and because it is so developer +oriented we have no choice but to give the wiki a lower priority. + -wiki:: - Because it is so resource heavy, and because it is so developer - oriented we have no choice but to give the wiki a lower priority. . Everything else. == Juggling Resources @@ -321,10 +325,10 @@ After release is a "go": * Make sure torrents are setup and ready to go. * fedora-web needs a branch for fN-beta. In it: -* Beta used on get-prerelease -* get-prerelease doesn't direct to release -* verify is updated with Beta info -* releases.txt gets a branched entry for preupgrade +** Beta used on get-prerelease +** get-prerelease doesn't direct to release +** verify is updated with Beta info +** releases.txt gets a branched entry for pre-upgrade * bfo gets updated to have a Beta entry. After release: @@ -345,8 +349,8 @@ After release is a "go": * Make sure torrents are setup and ready to go. * fedora-web needs a branch for fN-alpha. In it: -* get-prerelease does direct to release -* verify is updated with Final info +** get-prerelease does direct to release +** verify is updated with Final info * bfo gets updated to have a Final entry. * update wiki version numbers and names. @@ -354,7 +358,7 @@ After release: * Update /topic in #fedora-admin * post to infrastructure list that freeze is over. -* Move MirrorManager repository tags from the development/$version/ -Directory objects, to the releases/$version/ Directory objects. This is +* Move MirrorManager repository tags from the `development/$version/` +Directory objects, to the `releases/$version/` Directory objects. This is done using the `move-devel-to-release --version=$version` command on -bapp02. This is usually done now a week or two after release. +_bapp02_. This is usually done now a week or two after release.