From 0b20370aa3ef9eb89c9b5631f96bcf2bb0d630d5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Michal=20Kone=C4=8Dn=C3=BD?= Date: Thu, 19 Aug 2021 17:10:16 +0200 Subject: [PATCH] Review fedorawebsites SOP MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Michal Konečný --- modules/sysadmin_guide/nav.adoc | 2 +- .../sysadmin_guide/pages/fedorawebsites.adoc | 331 +++++++++--------- 2 files changed, 167 insertions(+), 166 deletions(-) diff --git a/modules/sysadmin_guide/nav.adoc b/modules/sysadmin_guide/nav.adoc index 4607e9b..c586a26 100644 --- a/modules/sysadmin_guide/nav.adoc +++ b/modules/sysadmin_guide/nav.adoc @@ -34,7 +34,7 @@ ** xref:fedmsg-websocket.adoc[WebSocket - SOP] ** xref:fedocal.adoc[Fedocal - SOP] ** xref:fedora-releases.adoc[Fedora Release Infrastructure - SOP] -** xref:fedorawebsites.adoc[fedorawebsites - SOP in review ] +** xref:fedorawebsites.adoc[Websites Release - SOP] ** xref:fmn.adoc[fmn - SOP in review ] ** xref:gather-easyfix.adoc[gather-easyfix - SOP in review ] ** xref:gdpr_delete.adoc[gdpr_delete - SOP in review ] diff --git a/modules/sysadmin_guide/pages/fedorawebsites.adoc b/modules/sysadmin_guide/pages/fedorawebsites.adoc index 6c5727a..614c407 100644 --- a/modules/sysadmin_guide/pages/fedorawebsites.adoc +++ b/modules/sysadmin_guide/pages/fedorawebsites.adoc @@ -1,303 +1,304 @@ = Websites Release SOP -____ -* {blank} -[arabic] -. Preparing the website for a release -** 1.1 Obsolete GPG key of the EOL Fedora release -** 1.2 Update GPG key -*** 1.2.1 Steps -* {blank} -[arabic, start=2] -. Update website -** 2.1 For Alpha -** 2.2 For Beta -** 2.3 For GA -* {blank} -[arabic, start=3] -. Fire in the hole -* {blank} -[arabic, start=4] -. Tips -** 4.1 Merging branches +== Contents -[arabic] -. Preparing the website for a new release cycle -____ +* <<_preparing_the_website_for_a_new_release_cycle>> +** <<_obsolete_gpg_key>> +** <<_update_gpg_key>> +* <<_update_website>> +** <<_for_alpha>> +** <<_for_beta>> +** <<_for_ga>> +* <<_fire_in_the_hole>> +* <<_tips>> +** <<_merging_branches>> -1.1 Obsolete GPG key + +== Preparing the website for a new release cycle + +=== Obsolete GPG key One month after a Fedora release the release number 'FXX-2' (i.e. 1 month after F21 release, F19 will be EOL) will be EOL (End of Life). At this point we should drop the GPG key from the list in verify/ and move -the keys to the obsolete keys page in keys/obsolete.html. +the keys to the obsolete keys page in `keys/obsolete.html`. -1.2 Update GPG key +=== Update GPG key After another couple of weeks and as the next release approaches, watch -the fedora-release package for a new key to be added. Use the -update-gpg-keys script in the fedora-web git repository to add it to -static/. Manually add it to /keys and /verify in all websites where we +the _fedora-release_ package for a new key to be added. Use the +`update-gpg-keys` script in the https://pagure.io/fedora-web/websites/[fedora-web git repository] +to add it to `static/`. Manually add it to `/keys` and `/verify` in all websites where we use these keys: -____ -* arm.fpo -* getfedora.org -* labs.fpo -* spins.fpo -____ +* https://arm.fedoraproject.org/ +* https://getfedora.org/ +* https://labs.fedoraproject.org/ +* https://spins.fedoraproject.org/ -1.2.1 Steps +===== Steps [loweralpha] . Get a copy of the new key(s) from the fedora-release repo, you will -find FXX-primary and FXX-secondary keys. Save them in ./tools to make +find FXX-primary and FXX-secondary keys. Save them in `./tools` to make the update easier. + https://pagure.io/fedora-repos -. Start by editing ./tools/update-gpg-keys and adding the key-ids of any +. Start by editing `./tools/update-gpg-keys` and adding the key-ids of any obsolete keys to the obsolete_keys list. -. Then run that script to add the new key(s) to the fedora.gpg block: +. Then run that script to add the new key(s) to the `fedora.gpg` block: + -fedora-web git:(master) cd tools/ tools git:(master) ./update-gpg-keys -RPM-GPG-KEY-fedora-23-primary tools git:(master) ./update-gpg-keys -RPM-GPG-KEY-fedora-23-secondary +.... +fedora-web git:(master) cd tools/ +tools git:(master) ./update-gpg-keys RPM-GPG-KEY-fedora-23-primary +tools git:(master) ./update-gpg-keys RPM-GPG-KEY-fedora-23-secondary +.... + This will add the key(s) to the keyblock in static/fedora.gpg and create -a text file for the key in static/$KEYID.txt as well. Verify that these +a text file for the key in `static/$KEYID.txt` as well. Verify that these files have been created properly and contain all the keys that they should. -* Handy checks: gpg static/fedora.gpg or gpg static/$KEYID.txt -* Adding "--with-fingerprint" option will add the fingerprint to the ++ +* Handy checks: `gpg static/fedora.gpg` or `gpg static/$KEYID.txt` ++ +* Adding _--with-fingerprint_ option will add the fingerprint to the output + -The output of fedora.gpg should contain only the actual keys, not the +The output of `fedora.gpg` should contain only the actual keys, not the obsolete keys. The single text files should contain the correct information for the uploaded key. -. Next, add new key(s) to the list in data/verify.html and move the new -key informations in the keys page in data/content/keys/index.html. A ++ +. Next, add new key(s) to the list in `data/verify.html` and move the new +key informations in the keys page in `data/content/keys/index.html`. A script to aid in generating the HTML code for new keys is in -./tools/make-gpg-key-html. It will print HTML to stdout for each +`./tools/make-gpg-key-html`. It will print HTML to stdout for each RPM-GPG-KEY-* file given as arguments. This is suitable for copy/paste (or directly importing if your editor supports this). Check the copied HTML code and select if the key info is for a primary or secondary key (output says 'Primary or Secondary'). + -tools git:(master) ./make-gpg-key-html RPM-GPG-KEY-fedora-23-primary +`tools git:(master) ./make-gpg-key-html RPM-GPG-KEY-fedora-23-primary` + Build the website with 'make en test' and carefully verify that the data is correct. Please double check all keys in http://localhost:5000/en/keys and http://localhost:5000/en/verify. + -NOTE: the tool will give you an outdated output, adapt it to the new +[NOTE] +==== +The tool will give you an outdated output, adapt it to the new websites and bootstrap layout! -____ +==== -____ -[arabic, start=2] -. Update website -____ +== Update website -2.1 For Alpha +=== For Alpha ____ [loweralpha] -. Create the fXX-alpha branch from master fedora-web git:(master) git -push origin master:refs/heads/f22-alpha +. Create the fXX-alpha branch from master fedora-web + -and checkout to the new branch: fedora-web git:(master) git checkout -t --b f13-alpha origin/f13-alpha +`git:(master) git push origin master:refs/heads/f22-alpha` ++ +and checkout to the new branch: ++ +`fedora-web git:(master) git checkout -t -b f13-alpha origin/f13-alpha` . Update the global variables Change curr_state to Alpha for all arches . Add Alpha banner Upload the FXX-Alpha banner to -static/images/banners/f22alpha.png which should appear in every -$\{PRODUCT}/download/index.html page. Make sure the banner is shown in +`static/images/banners/f22alpha.png` which should appear in every +`$\{PRODUCT}/download/index.html` page. Make sure the banner is shown in all sidebars, also in labs, spins, and arm. . Check all Download links and paths in -$\{PRODUCT}/prerelease/index.html You can find all paths in bapp01 (sudo -su - mirrormanager first) or you can look at the downlaod page +`$\{PRODUCT}/prerelease/index.html`. You can find all paths in _bapp01_ +(`sudo su - mirrormanager first`) or you can look at the downlaod page http://dl.fedoraproject.org/pub/alt/stage -. Add CHECKSUM files to static/checksums and verify that the paths are -correct. The files should be in sundries01 and you can query them with: -$ find /pub/fedora/linux/releases/test/17-Alpha/ -type f -name -_CHECKSUM_ -exec cp '\{}' . ; Remember to add the right checksums to the -right websites (same path). -. Add EC2 AMI IDs for Alpha. All IDs now are in the globalvar.py file. +. Add CHECKSUM files to `static/checksums` and verify that the paths are +correct. The files should be in _sundries01_ and you can query them with: ++ +`$ find /pub/fedora/linux/releases/test/17-Alpha/ -type f -name CHECKSUM -exec cp '\{}' . ;` ++ +Remember to add the right checksums to the right websites (same path). +. Add EC2 AMI IDs for Alpha. All IDs now are in the `globalvar.py` file. We get all data from there, even the redirect path to trac the AMI IDs. We now also have a script which is useful to get all the AMI IDs uploaded with fedimg. Execute it to get the latest uploads, but don't -run the script too early, as new builds are added constantly. fedora-web -git:(fXX-alpha) python ~/fedora-web/tools/get_ami.py -. Add CHECKSUM files also to http://spins.fedoraproject.org in -static/checksums. Verify the paths are correct in -data/content/verify.html. (see point e) to query them on sundries01). -Same for labs.fpo and arm.fpo. -. Verify all paths and links on http://spins.fpo, labs.fpo and arm.fpo. +run the script too early, as new builds are added constantly. ++ +`fedora-web git:(fXX-alpha) python ~/fedora-web/tools/get_ami.py` +. Add CHECKSUM files also to https://spins.fedoraproject.org in +`static/checksums`. Verify the paths are correct in +`data/content/verify.html`. (see point e) to query them on _sundries01_). +Same for https://labs.fedoraproject.org/ and https://arm.fedoraproject.org. +. Verify all paths and links on https://spins.fedoraproject.org, +https://labs.fedoraproject.org/ and https://arm.fedoraproject.org. . Update Alpha Image sizes and pre_cloud_composedate in -./build.d/globalvar.py. Verify they are right in Cloud images and Docker +`./build.d/globalvar.py`. Verify they are right in Cloud images and Docker image. . Update the new POT files and push them to Zanata (ask a maintainer to do so) every time you change text strings. -. Add this build to stg.fedoraproject.org (ansible syncStatic.sh.stg) to +. Add this build to https://stg.fedoraproject.org (ansible `syncStatic.sh.stg`) to test the pages online. -____ - . Release Date: - -____ * Merge the fXX-alpha branch to master and correct conflicts manually * Remove the redirect of prerelease pages in ansible, edit: -* ansible/playbooks/include/proxies-redirects.yml +** `ansible/playbooks/include/proxies-redirects.yml` * ask a sysadmin-main to run playbook * When ready and about 90 minutes before Release Time push to master -* Tag the commit as new release and push it too: $ git tag -a FXX-Alpha --m 'Releasing Fedora XX Alpha' $ git push --tags +** Tag the commit as new release and push it too: ++ +`$ git tag -a FXX-Alpha -m 'Releasing Fedora XX Alpha' $ git push --tags` * If needed follow "Fire in the hole" below. ____ -2.2 For Beta + +=== For Beta ____ [loweralpha] -. Create the fXX-beta branch from master fedora-web git:(master) git -push origin master:refs/heads/f22-beta +. Create the fXX-beta branch from master + -and checkout to the new branch: fedora-web git:(master) git checkout -t --b f22-beta origin/f22-beta +`fedora-web git:(master) git push origin master:refs/heads/f22-beta` ++ +and checkout to the new branch: ++ +`fedora-web git:(master) git checkout -t -b f22-beta origin/f22-beta` . Update the global variables Change curr_state to Beta for all arches . Add Alpha banner Upload the FXX-Beta banner to -static/images/banners/f22beta.png which should appear in every -$\{PRODUCT}/download/index.html page. Make sure the banner is shown in -all sidebars, also in labs, spins, and arm. +`static/images/banners/f22beta.png` which should appear in every +`$\{PRODUCT}/download/index.html` page. Make sure the banner is shown in +all sidebars, also in https://spins.fedoraproject.org, +https://labs.fedoraproject.org/ and https://arm.fedoraproject.org. . Check all Download links and paths in -$\{PRODUCT}/prerelease/index.html You can find all paths in bapp01 (sudo -su - mirrormanager first) or you can look at the downlaod page -http://dl.fedoraproject.org/pub/alt/stage +`$\{PRODUCT}/prerelease/index.html` You can find all paths in _bapp01_ +(`sudo su - mirrormanager first`) or you can look at the downlaod page +https://dl.fedoraproject.org/pub/alt/stage . Add CHECKSUM files to static/checksums and verify that the paths are -correct. The files should be in sundries and you can query them with: $ -find /pub/fedora/linux/releases/test/17-Beta/ -type f -name _CHECKSUM_ --exec cp '\{}' . ; Remember to add the right checksums to the right -websites (same path). -. Add EC2 AMI IDs for Beta. All IDs now are in the globalvar.py file. We +correct. The files should be in sundries and you can query them with: ++ +`$ find /pub/fedora/linux/releases/test/17-Beta/ -type f -name CHECKSUM -exec cp '\{}' . ;` ++ +Remember to add the right checksums to the right websites (same path). +. Add EC2 AMI IDs for Beta. All IDs now are in the `globalvar.py` file. We get all data from there, even the redirect path to trac the AMI IDs. We now also have a script which is useful to get all the AMI IDs uploaded with fedimg. Execute it to get the latest uploads, but don't run the -script too early, as new builds are added constantly. fedora-web -git:(fXX-beta) python ~/fedora-web/tools/get_ami.py +script too early, as new builds are added constantly. ++ +`fedora-web git:(fXX-beta) python ~/fedora-web/tools/get_ami.py` . Add CHECKSUM files also to http://spins.fedoraproject.org in -static/checksums. Verify the paths are correct in -data/content/verify.html. (see point e) to query them on sundries01). -Same for labs.fpo and arm.fpo. -. Remove static/checksums/Fedora-XX-Alpha* in all websites. +`static/checksums`. Verify the paths are correct in +`data/content/verify.html`. (see point e) to query them on _sundries01_). +Same for https://labs.fedoraproject.org/ and https://arm.fedoraproject.org. +. Remove `static/checksums/Fedora-XX-Alpha*` in all websites. . Verify all paths and links on http://spins.fpo, labs.fpo and arm.fpo. . Update Beta Image sizes and pre_cloud_composedate in ./build.d/globalvar.py. Verify they are right in Cloud images and Docker image. . Update the new POT files and push them to Zanata (ask a maintainer to do so) every time you change text strings. -. Add this build to stg.fedoraproject.org (ansible syncStatic.sh.stg) to +. Add this build to https://stg.fedoraproject.org (ansible `syncStatic.sh.stg`) to test the pages online. -____ - . Release Date: - -____ * Merge the fXX-beta branch to master and correct conflicts manually * When ready and about 90 minutes before Release Time push to master -* Tag the commit as new release and push it too: $ git tag -a FXX-Beta --m 'Releasing Fedora XX Beta' $ git push --tags +* Tag the commit as new release and push it too: ++ +`$ git tag -a FXX-Beta -m 'Releasing Fedora XX Beta' $ git push --tags` * If needed follow "Fire in the hole" below. ____ -2.3 For GA +=== For GA ____ [loweralpha] -. Create the fXX branch from master fedora-web git:(master) git push -origin master:refs/heads/f22 +. Create the fXX branch from master + -and checkout to the new branch: fedora-web git:(master) git checkout -t --b f22 origin/f22 +`fedora-web git:(master) git push origin master:refs/heads/f22` ++ +and checkout to the new branch: ++ +`fedora-web git:(master) git checkout -t -b f22 origin/f22` . Update the global variables Change curr_state for all arches -. Check all Download links and paths in $\{PRODUCT}/download/index.html -You can find all paths in bapp01 (sudo su - mirrormanager first) or you +. Check all Download links and paths in `$\{PRODUCT}/download/index.html` +You can find all paths in _bapp01_ (`sudo su - mirrormanager first`) or you can look at the downlaod page http://dl.fedoraproject.org/pub/alt/stage -. Add CHECKSUM files to static/checksums and verify that the paths are -correct. The files should be in sundries01 and you can query them with: -$ find /pub/fedora/linux/releases/17/ -type f -name _CHECKSUM_ -exec cp -'\{}' . ; Remember to add the right checksums to the right websites -(same path). +. Add CHECKSUM files to `static/checksums` and verify that the paths are +correct. The files should be in _sundries01_ and you can query them with: ++ +`$ find /pub/fedora/linux/releases/17/ -type f -name CHECKSUM -exec cp '\{}' . ;` ++ +Remember to add the right checksums to the right websites (same path). . At some point freeze translations. Add an empty PO_FREEZE file to every website's directory you want to freeze. -. Add EC2 AMI IDs for GA. All IDs now are in the globalvar.py file. We +. Add EC2 AMI IDs for GA. All IDs now are in the `globalvar.py` file. We get all data from there, even the redirect path to trac the AMI IDs. We now also have a script which is useful to get all the AMI IDs uploaded with fedimg. Execute it to get the latest uploads, but don't run the -script too early, as new builds are added constantly. fedora-web -git:(fXX) python ~/fedora-web/tools/get_ami.py +script too early, as new builds are added constantly. ++ +`fedora-web git:(fXX) python ~/fedora-web/tools/get_ami.py` . Add CHECKSUM files also to http://spins.fedoraproject.org in -static/checksums. Verify the paths are correct in -data/content/verify.html. (see point e) to query them on sundries01). -Same for labs.fpo and arm.fpo. -. Remove static/checksums/Fedora-XX-Beta* in all websites. -. Verify all paths and links on http://spins.fpo, labs.fpo and arm.fpo. -. Update GA Image sizes and cloud_composedate in ./build.d/globalvar.py. +`static/checksums`. Verify the paths are correct in +`data/content/verify.html`. (see point e) to query them on _sundries01_). +Same for https://labs.fedoraproject.org/ and https://arm.fedoraproject.org. +. Remove `static/checksums/Fedora-XX-Beta*` in all websites. +. Verify all paths and links on https://spins.fedoraproject.org, +https://labs.fedoraproject.org/ and https://arm.fedoraproject.org. +. Update GA Image sizes and cloud_composedate in `./build.d/globalvar.py`. Verify they are right in Cloud images and Docker image. -. Update static/js/checksum.js and check if the paths and checksum still +. Update `static/js/checksum.js` and check if the paths and checksum still match. . Update the new POT files and push them to Zanata (ask a maintainer to do so) every time you change text strings. -. Add this build to stg.fedoraproject.org (ansible syncStatic.sh.stg) to +. Add this build to https://stg.fedoraproject.org (ansible `syncStatic.sh.stg`) to test the pages online. -____ - . Release Date: - -____ * Merge the fXX-beta branch to master and correct conflicts manually * Add the redirect of prerelease pages in ansible, edit: -* ansible/playbooks/include/proxies-redirects.yml -* ask a sysadmin-main to run playbook +** `ansible/playbooks/include/proxies-redirects.yml` +** ask a sysadmin-main to run playbook * Unfreeze translations by deleting the PO_FREEZE files * When ready and about 90 minutes before Release Time push to master * Update the short links for the Cloud Images for 'Fedora XX', 'Fedora XX-1' and 'Latest' -* Tag the commit as new release and push it too: $ git tag -a FXX -m -'Releasing Fedora XX' $ git push --tags +* Tag the commit as new release and push it too: ++ +.... +`$ git tag -a FXX -m 'Releasing Fedora XX'` +`$ git push --tags` +.... ++ * If needed follow "Fire in the hole" below. ____ -____ -[arabic, start=3] -. Fire in the hole -____ +== Fire in the hole We now use ansible for everything, and normally use a regular build to make the websites live. If something is not happening as expected, you should get in contact with a sysadmin-main to run the ansible playbook again. -All our stuff, such as SyncStatic.sh and SyncTranslation.sh scripts are +All our stuff, such as `SyncStatic.sh` and `SyncTranslation.sh` scripts are now also in ansible! -Staging server app02 and production server bapp01 do not exist anymore, -now our staging websites are on sundries01.stg and the production on -sundries01. Change your scripts accordingly and as sysadmin-web you +Staging server _app02_ and production server _bapp01_ do not exist anymore, +now our staging websites are on _sundries01.stg_ and the production on +_sundries01_. Change your scripts accordingly and as sysadmin-web you should have access to those servers as before. -____ -____ -[arabic, start=4] -. Tips -____ +== Tips -4.1 Merging branches +=== Merging branches -Suggested by Ricky This can be useful if you're _sure_ all new changes +Suggested by Ricky + +This can be useful if you're _sure_ all new changes on devel branch should go into the master branch. Conflicts will be solved directly accepting only the changes in the devel branch. If you're not 100% sure do a normal merge and fix conflicts manually! -$ git merge f22-beta $ git checkout --theirs f22-beta [list of -conflicting po files] $ git commit - +.... +$ git merge f22-beta +$ git checkout --theirs f22-beta [list of conflicting po files] +$ git commit +....