Add documentation about release engineering involvements
Signed-off-by: Akashdeep Dhar <akashdeep.dhar@gmail.com>
This commit is contained in:
Akashdeep Dhar
parent
3831828764
commit
8fb9b2fdf0
2 changed files with 301 additions and 0 deletions
|
|
@ -53,6 +53,7 @@ Index
|
||||||
toddlers
|
toddlers
|
||||||
messaging
|
messaging
|
||||||
monitorgating
|
monitorgating
|
||||||
|
releng
|
||||||
|
|
||||||
Conclusions
|
Conclusions
|
||||||
----
|
----
|
||||||
|
|
|
||||||
300
docs/dist-git-move/releng.rst
Normal file
300
docs/dist-git-move/releng.rst
Normal file
|
|
@ -0,0 +1,300 @@
|
||||||
|
.. _releng:
|
||||||
|
|
||||||
|
Pagure Dist Git Interactions With Release Engineering Scripts
|
||||||
|
====
|
||||||
|
|
||||||
|
Release Engineering Scripts (as the name suggests) contains scripts necessary
|
||||||
|
for the Fedora Linux release processes.
|
||||||
|
|
||||||
|
Possible Interactions
|
||||||
|
----
|
||||||
|
|
||||||
|
Documentation
|
||||||
|
^^^^
|
||||||
|
|
||||||
|
The repository houses the following set of documentation that details on the
|
||||||
|
interaction that the Release Engineering team has with the **Pagure Dist Git**
|
||||||
|
platform.
|
||||||
|
|
||||||
|
Process fedora-scm-requests tickets
|
||||||
|
""""
|
||||||
|
|
||||||
|
The documentation details on the process that is required to create new source
|
||||||
|
control management repositories or a branch within it whenever those are
|
||||||
|
requested by a packager. The said documentation can be found
|
||||||
|
`here <https://pagure.io/releng/blob/main/f/docs/source/sop_process_dist_git_requests.rst>`_.
|
||||||
|
|
||||||
|
Remove dist-git branches
|
||||||
|
""""
|
||||||
|
|
||||||
|
The documentation details on the process that is required to remove branches
|
||||||
|
from the source control management repositories when they are requested for by
|
||||||
|
the maintainers. The said documentation can be found
|
||||||
|
`here <https://pagure.io/releng/blob/main/f/docs/source/sop_remote_dist_git_branches.rst>`_.
|
||||||
|
|
||||||
|
Mass branching
|
||||||
|
""""
|
||||||
|
|
||||||
|
The documentation details on the process of branching the pending release away
|
||||||
|
from ``devel/`` at the alpha freeze and how many elements (including
|
||||||
|
**Pagure Dist Git**) are involved in making it happen. The said documentation
|
||||||
|
can be found
|
||||||
|
`here <https://pagure.io/releng/blob/main/f/docs/source/sop_mass_branching.rst>`_.
|
||||||
|
|
||||||
|
Fedora Layered Image Build System
|
||||||
|
""""
|
||||||
|
|
||||||
|
The documentation describes how the Fedora Layered Image Build System works and
|
||||||
|
how many elements (including **Pagure Dist Git**) are involved in making it
|
||||||
|
happen. The said documentation can be found
|
||||||
|
`here <https://pagure.io/releng/blob/main/f/docs/source/layered_image_build_service.rst>`_.
|
||||||
|
|
||||||
|
Adding a New Release Engineer
|
||||||
|
""""
|
||||||
|
|
||||||
|
The documentation elucidates about the onboarding process for a new release
|
||||||
|
engineer and how they must be provided access to various elements (including
|
||||||
|
**Pagure Dist Git**) to be able to fulfill their purposes. The said
|
||||||
|
documentation can be found
|
||||||
|
`here <https://pagure.io/releng/blob/main/f/docs/source/sop_adding_new_release_engineer.rst>`_.
|
||||||
|
|
||||||
|
Mass Rebuild of Modules
|
||||||
|
""""
|
||||||
|
|
||||||
|
The documentation explains about how periodical mass rebuilding of modules are
|
||||||
|
performed in rawhide and how many elements (including **Pagure Dist Git**) are
|
||||||
|
involved in the said process. The said documentation can be found
|
||||||
|
`here <https://pagure.io/releng/blob/main/f/docs/source/sop_mass_rebuild_modules.rst>`_.
|
||||||
|
|
||||||
|
Unretiring a Package Branch
|
||||||
|
""""
|
||||||
|
|
||||||
|
The documentation elaborates about how certain *retired packages* can be
|
||||||
|
brought to an *unretired state* and how many elements (including
|
||||||
|
**Pagure Dist Git**) are involved in the said process. The said documentation
|
||||||
|
can be found
|
||||||
|
`here <https://pagure.io/releng/blob/main/f/docs/source/sop_unretire.rst>`_.
|
||||||
|
|
||||||
|
Update Critical Path
|
||||||
|
""""
|
||||||
|
|
||||||
|
The documentation explicates how certain applications can be added, updated or
|
||||||
|
removed from the Fedora Linux release's critical path and how many elements
|
||||||
|
(including **Pagure Dist Git**) are involved in making it happen. The said
|
||||||
|
documentation can be found
|
||||||
|
`here <https://pagure.io/releng/blob/main/f/docs/source/sop_update_critpath.rst>`_.
|
||||||
|
|
||||||
|
Troubleshooting
|
||||||
|
""""
|
||||||
|
|
||||||
|
The documentation demonstrates the behaviour of many elements (including
|
||||||
|
**Pagure Dist Git**) can be diagnosed and corrected in case those do not work
|
||||||
|
how they are supposed to. The said documentation can be found
|
||||||
|
`here <https://pagure.io/releng/blob/main/f/docs/source/troubleshooting.rst>`_.
|
||||||
|
|
||||||
|
Scripts
|
||||||
|
^^^^
|
||||||
|
|
||||||
|
``retire_packagers.py``
|
||||||
|
""""
|
||||||
|
|
||||||
|
This script helps retiring a packager by orphaning all the packages that the
|
||||||
|
said packager is the main admin of, removing the access to the source control
|
||||||
|
management repository for the said packager and resetting their watch status.
|
||||||
|
|
||||||
|
The Pagure Dist Git API is used here and the base URL of the production
|
||||||
|
deployment is mentioned of in the variable ``dist_git_base``. The said script
|
||||||
|
can be found
|
||||||
|
`here <https://pagure.io/releng/blob/main/f/scripts/distgit/retire_packagers.py>`_.
|
||||||
|
|
||||||
|
|
||||||
|
``modulepkg.py``
|
||||||
|
""""
|
||||||
|
|
||||||
|
This script helps track modules that contain a specific package optionally
|
||||||
|
limited to its branch involving various systems like
|
||||||
|
`Product Definition Center <https://pdc.fedoraproject.org/>`_,
|
||||||
|
**Pagure Dist Git** and many other such elements.
|
||||||
|
|
||||||
|
The **Pagure Dist Git** API is used here and the base URL for ``/modules/`` of the
|
||||||
|
production deployment is mentioned of in the variable ``MODULES_SRC_URL``. The
|
||||||
|
said script can be found
|
||||||
|
`here <https://pagure.io/releng/blob/main/f/scripts/branching/modulepkg.py>`_.
|
||||||
|
|
||||||
|
``give-package.py``
|
||||||
|
""""
|
||||||
|
|
||||||
|
This script helps transfer a package in **Pagure Dist Git** from one user to
|
||||||
|
another or to orphan the said package by transferring the package to the
|
||||||
|
``orphan`` user using a privileged Pagure API access token.
|
||||||
|
|
||||||
|
The **Pagure Dist Git** API is used here and the base URL for the REST API of
|
||||||
|
the production deployment is mentioned of in the variable ``PAGURE_URL``. The
|
||||||
|
said script can be found
|
||||||
|
`here <https://pagure.io/releng/blob/main/f/scripts/distgit/give-package.py>`_.
|
||||||
|
|
||||||
|
``utilities.py``
|
||||||
|
""""
|
||||||
|
|
||||||
|
This script helps manipulate **Pagure Dist Git**. It can retry creation of a
|
||||||
|
session with the **Pagure Dist Git** API and transferring packages from one
|
||||||
|
user to another by using one of the helper scripts mentioned above.
|
||||||
|
|
||||||
|
The **Pagure Dist Git** API is used here and the base URL for the REST API of
|
||||||
|
the production deployment is mentioned of in the variable ``PAGURE_URL``. The
|
||||||
|
said script can be found
|
||||||
|
`here <https://pagure.io/releng/blob/main/f/scripts/distgit/utilities.py>`_.
|
||||||
|
|
||||||
|
``sig_policy.py``
|
||||||
|
""""
|
||||||
|
|
||||||
|
This script helps to enact the FESCo SIG policy. More about the FESCo SIG
|
||||||
|
policy can be found in
|
||||||
|
`this <https://docs.fedoraproject.org/en-US/fesco/SIG_policy/>`_ documentation.
|
||||||
|
|
||||||
|
The **Pagure Dist Git** API is used here and the base URL for the data elements
|
||||||
|
of the production deployment is mentioned of in the variable
|
||||||
|
``PAGURE_DIST_GIT_DATA_URL``. The said script can be found
|
||||||
|
`here <https://pagure.io/releng/blob/main/f/scripts/fesco/sig-policy/sig_policy.py>`_.
|
||||||
|
|
||||||
|
``find_unblocked_orphans.py``
|
||||||
|
""""
|
||||||
|
|
||||||
|
This script helps discover orphaned packages in Pagure that are unblocked in
|
||||||
|
Koji adnd to show what assets depend on those orphans. This script works across
|
||||||
|
various releases including EPEL and Fedora Linux repositories.
|
||||||
|
|
||||||
|
While a direct affiliation with **Pagure Dist Git** could not be drawn, the
|
||||||
|
object ``cache`` depends of a ``dogpile`` cache database named
|
||||||
|
``dist-git-orphans-cache.dbm``. The said script can be found
|
||||||
|
`here <https://pagure.io/releng/blob/main/f/scripts/find_unblocked_orphans.py>`_.
|
||||||
|
|
||||||
|
``insert_sla.py``
|
||||||
|
""""
|
||||||
|
|
||||||
|
This script helps choosing SLAs from a set specified by the Release Engineering
|
||||||
|
on the branches available on **Pagure Dist Git** to avoid having arbitrary
|
||||||
|
EOLs by working with the
|
||||||
|
`Product Definition Center <https://pdc.fedoraproject.org/>`_.
|
||||||
|
|
||||||
|
While a direct affiliation with **Pagure Dist Git** could not be drawn, the
|
||||||
|
changes made as a consequence of the execution of this script can lead to
|
||||||
|
changes on **Pagure Dist Git**. The said script can be found
|
||||||
|
`here <https://pagure.io/releng/blob/main/f/scripts/pdc/insert-sla.py>`_.
|
||||||
|
|
||||||
|
``check-retired-branches-pdc.py``
|
||||||
|
""""
|
||||||
|
|
||||||
|
This script helps collect all the packages from the **Pagure Dist Git** to
|
||||||
|
check if they are retired and queries
|
||||||
|
`Product Definition Center <https://pdc.fedoraproject.org/>`_ to confirm they
|
||||||
|
are not listed as active there.
|
||||||
|
|
||||||
|
The **Pagure Dist Git** API is used here and the base URL of the production
|
||||||
|
deployment is mentioned of in the variable ``dist_git_base``. The said script
|
||||||
|
can be found
|
||||||
|
`here <https://pagure.io/releng/blob/main/f/scripts/check-retired-branches-pdc.py>`_.
|
||||||
|
|
||||||
|
``orphan-listed-packages.py``
|
||||||
|
""""
|
||||||
|
|
||||||
|
This script helps orphaning listed packages in bulk when they fail to install
|
||||||
|
or fail to build without being addressed for some time. This is a better
|
||||||
|
solution than to perform the process once per package using the scripts.
|
||||||
|
|
||||||
|
The **Pagure Dist Git** API is used here and the base URL of the production
|
||||||
|
deployment is mentioned of in the variable ``BASE_URL``. The said script
|
||||||
|
can be found
|
||||||
|
`here <https://pagure.io/releng/blob/main/f/scripts/distgit/orphan-listed-packages.py>`_.
|
||||||
|
|
||||||
|
``check-unretirement.py``
|
||||||
|
""""
|
||||||
|
|
||||||
|
This script helps finding and listing the contributors of a Fedora Src Package
|
||||||
|
and check their SLAs from
|
||||||
|
`Product Definition Center <https://pdc.fedoraproject.org/>`_ to assert if the
|
||||||
|
file `dead.package` exists in the package for unretirement confirmation.
|
||||||
|
|
||||||
|
The **Pagure Dist Git** API is used here and the URL of the production
|
||||||
|
deployment for contributors and states are mentioned of in the variables
|
||||||
|
``contributors_url`` and ``state_url`` respectively. The said script can be
|
||||||
|
found
|
||||||
|
`here <https://pagure.io/releng/blob/main/f/scripts/check-unretirement.py>`_.
|
||||||
|
|
||||||
|
``orphan-all-packages.py``
|
||||||
|
""""
|
||||||
|
|
||||||
|
This script helps to orphan all the packages belonging to a certain set of
|
||||||
|
users if they wish to no longer package software on Fedora Linux. The package
|
||||||
|
is either transferred to a new owner from the committers or orphaned.
|
||||||
|
|
||||||
|
The **Pagure Dist Git** API is used here and the base URL of the production
|
||||||
|
deployment is mentioned of in the variable ``PAGURE_URL``. The said script
|
||||||
|
can be found
|
||||||
|
`here <https://pagure.io/releng/blob/main/f/scripts/distgit/orphan-all-packages.py>`_.
|
||||||
|
|
||||||
|
``fedretire``
|
||||||
|
""""
|
||||||
|
|
||||||
|
This script helps to retire packages that have either been orphaned for more
|
||||||
|
than six weeks, depended on a retried package that was orphaned or because the
|
||||||
|
said package was built and published.
|
||||||
|
|
||||||
|
The **Pagure Dist Git** API is used here and the base URL of the production
|
||||||
|
deployment is mentioned of in the variable ``PAGURE_URL``. The said script
|
||||||
|
can be found
|
||||||
|
`here <https://pagure.io/releng/blob/main/f/scripts/fedretire>`_.
|
||||||
|
|
||||||
|
``find_failures.py``
|
||||||
|
""""
|
||||||
|
|
||||||
|
This script helps discover failed builds in a given tag by delivering an output
|
||||||
|
in rough HTML format for a given package belonging to a certain packager.
|
||||||
|
|
||||||
|
The **Pagure Dist Git** API is used here and the URL for the owner data of the
|
||||||
|
production deployment is mentioned of in the variable ``ownerdataurl``. The
|
||||||
|
said script can be found
|
||||||
|
`here <https://pagure.io/releng/blob/main/f/scripts/find_failures.py>`_.
|
||||||
|
|
||||||
|
``follow-policy.py``
|
||||||
|
""""
|
||||||
|
|
||||||
|
This script helps confirm if a package follows policy by asserting its
|
||||||
|
installability and other checks, and in case it does not, it follows through
|
||||||
|
with orphaning the said package belonging to a certain packager.
|
||||||
|
|
||||||
|
The **Pagure Dist Git** API is used here and the URL for the package owner data
|
||||||
|
and confirming package orphaning of the production deployment is mentioned of
|
||||||
|
in the variable ``pkg_owners`` and function ``update_bugs``. The said script
|
||||||
|
can be found
|
||||||
|
`here <https://pagure.io/releng/blob/main/f/scripts/ftbfs-fti/follow-policy.py>`_.
|
||||||
|
|
||||||
|
Changes proposed
|
||||||
|
----
|
||||||
|
|
||||||
|
In order to switch out **Pagure Dist Git** for an alternative that provides a
|
||||||
|
relatively compatible API pattern, the following changes must be made in the
|
||||||
|
places mentioned below.
|
||||||
|
|
||||||
|
Necessary
|
||||||
|
^^^^
|
||||||
|
|
||||||
|
- Release Engineering scripts heavily rely on the **Pagure Dist Git** API and
|
||||||
|
hence it is important to find a replacement alternative that mimics the API
|
||||||
|
scheme of **Pagure Dist Git** and replace the URL to the production
|
||||||
|
deployment in the usages of the same in the scripts and documentations.
|
||||||
|
|
||||||
|
- In case the above is not possible, a translation layer (a meta service of
|
||||||
|
sorts) needs to be created and hosted on the same location where the
|
||||||
|
**Pagure Dist Git** is deployed and that can translate these requests into
|
||||||
|
the API schema that can be comprehended by the replacement alternative.
|
||||||
|
|
||||||
|
- In case the above is not possible, all the aforementioned scripts and
|
||||||
|
documentation will have to be rewritten to play well with the API schema of
|
||||||
|
the replacement alternative chosen and checked thoroughly to confirm that
|
||||||
|
the inter-working of the scripts are as robust as they are at the moment.
|
||||||
|
|
||||||
|
Unnecessary
|
||||||
|
^^^^
|
||||||
|
|
||||||
|
- None
|
||||||
Loading…
Add table
Add a link
Reference in a new issue