Add documentation about release engineering involvements
Signed-off-by: Akashdeep Dhar <akashdeep.dhar@gmail.com>
This commit is contained in:
Akashdeep Dhar
parent
3831828764
commit
8fb9b2fdf0
2 changed files with 301 additions and 0 deletions
|
|
@ -53,6 +53,7 @@ Index
|
|||
toddlers
|
||||
messaging
|
||||
monitorgating
|
||||
releng
|
||||
|
||||
Conclusions
|
||||
----
|
||||
|
|
|
|||
300
docs/dist-git-move/releng.rst
Normal file
300
docs/dist-git-move/releng.rst
Normal file
|
|
@ -0,0 +1,300 @@
|
|||
.. _releng:
|
||||
|
||||
Pagure Dist Git Interactions With Release Engineering Scripts
|
||||
====
|
||||
|
||||
Release Engineering Scripts (as the name suggests) contains scripts necessary
|
||||
for the Fedora Linux release processes.
|
||||
|
||||
Possible Interactions
|
||||
----
|
||||
|
||||
Documentation
|
||||
^^^^
|
||||
|
||||
The repository houses the following set of documentation that details on the
|
||||
interaction that the Release Engineering team has with the **Pagure Dist Git**
|
||||
platform.
|
||||
|
||||
Process fedora-scm-requests tickets
|
||||
""""
|
||||
|
||||
The documentation details on the process that is required to create new source
|
||||
control management repositories or a branch within it whenever those are
|
||||
requested by a packager. The said documentation can be found
|
||||
`here <https://pagure.io/releng/blob/main/f/docs/source/sop_process_dist_git_requests.rst>`_.
|
||||
|
||||
Remove dist-git branches
|
||||
""""
|
||||
|
||||
The documentation details on the process that is required to remove branches
|
||||
from the source control management repositories when they are requested for by
|
||||
the maintainers. The said documentation can be found
|
||||
`here <https://pagure.io/releng/blob/main/f/docs/source/sop_remote_dist_git_branches.rst>`_.
|
||||
|
||||
Mass branching
|
||||
""""
|
||||
|
||||
The documentation details on the process of branching the pending release away
|
||||
from ``devel/`` at the alpha freeze and how many elements (including
|
||||
**Pagure Dist Git**) are involved in making it happen. The said documentation
|
||||
can be found
|
||||
`here <https://pagure.io/releng/blob/main/f/docs/source/sop_mass_branching.rst>`_.
|
||||
|
||||
Fedora Layered Image Build System
|
||||
""""
|
||||
|
||||
The documentation describes how the Fedora Layered Image Build System works and
|
||||
how many elements (including **Pagure Dist Git**) are involved in making it
|
||||
happen. The said documentation can be found
|
||||
`here <https://pagure.io/releng/blob/main/f/docs/source/layered_image_build_service.rst>`_.
|
||||
|
||||
Adding a New Release Engineer
|
||||
""""
|
||||
|
||||
The documentation elucidates about the onboarding process for a new release
|
||||
engineer and how they must be provided access to various elements (including
|
||||
**Pagure Dist Git**) to be able to fulfill their purposes. The said
|
||||
documentation can be found
|
||||
`here <https://pagure.io/releng/blob/main/f/docs/source/sop_adding_new_release_engineer.rst>`_.
|
||||
|
||||
Mass Rebuild of Modules
|
||||
""""
|
||||
|
||||
The documentation explains about how periodical mass rebuilding of modules are
|
||||
performed in rawhide and how many elements (including **Pagure Dist Git**) are
|
||||
involved in the said process. The said documentation can be found
|
||||
`here <https://pagure.io/releng/blob/main/f/docs/source/sop_mass_rebuild_modules.rst>`_.
|
||||
|
||||
Unretiring a Package Branch
|
||||
""""
|
||||
|
||||
The documentation elaborates about how certain *retired packages* can be
|
||||
brought to an *unretired state* and how many elements (including
|
||||
**Pagure Dist Git**) are involved in the said process. The said documentation
|
||||
can be found
|
||||
`here <https://pagure.io/releng/blob/main/f/docs/source/sop_unretire.rst>`_.
|
||||
|
||||
Update Critical Path
|
||||
""""
|
||||
|
||||
The documentation explicates how certain applications can be added, updated or
|
||||
removed from the Fedora Linux release's critical path and how many elements
|
||||
(including **Pagure Dist Git**) are involved in making it happen. The said
|
||||
documentation can be found
|
||||
`here <https://pagure.io/releng/blob/main/f/docs/source/sop_update_critpath.rst>`_.
|
||||
|
||||
Troubleshooting
|
||||
""""
|
||||
|
||||
The documentation demonstrates the behaviour of many elements (including
|
||||
**Pagure Dist Git**) can be diagnosed and corrected in case those do not work
|
||||
how they are supposed to. The said documentation can be found
|
||||
`here <https://pagure.io/releng/blob/main/f/docs/source/troubleshooting.rst>`_.
|
||||
|
||||
Scripts
|
||||
^^^^
|
||||
|
||||
``retire_packagers.py``
|
||||
""""
|
||||
|
||||
This script helps retiring a packager by orphaning all the packages that the
|
||||
said packager is the main admin of, removing the access to the source control
|
||||
management repository for the said packager and resetting their watch status.
|
||||
|
||||
The Pagure Dist Git API is used here and the base URL of the production
|
||||
deployment is mentioned of in the variable ``dist_git_base``. The said script
|
||||
can be found
|
||||
`here <https://pagure.io/releng/blob/main/f/scripts/distgit/retire_packagers.py>`_.
|
||||
|
||||
|
||||
``modulepkg.py``
|
||||
""""
|
||||
|
||||
This script helps track modules that contain a specific package optionally
|
||||
limited to its branch involving various systems like
|
||||
`Product Definition Center <https://pdc.fedoraproject.org/>`_,
|
||||
**Pagure Dist Git** and many other such elements.
|
||||
|
||||
The **Pagure Dist Git** API is used here and the base URL for ``/modules/`` of the
|
||||
production deployment is mentioned of in the variable ``MODULES_SRC_URL``. The
|
||||
said script can be found
|
||||
`here <https://pagure.io/releng/blob/main/f/scripts/branching/modulepkg.py>`_.
|
||||
|
||||
``give-package.py``
|
||||
""""
|
||||
|
||||
This script helps transfer a package in **Pagure Dist Git** from one user to
|
||||
another or to orphan the said package by transferring the package to the
|
||||
``orphan`` user using a privileged Pagure API access token.
|
||||
|
||||
The **Pagure Dist Git** API is used here and the base URL for the REST API of
|
||||
the production deployment is mentioned of in the variable ``PAGURE_URL``. The
|
||||
said script can be found
|
||||
`here <https://pagure.io/releng/blob/main/f/scripts/distgit/give-package.py>`_.
|
||||
|
||||
``utilities.py``
|
||||
""""
|
||||
|
||||
This script helps manipulate **Pagure Dist Git**. It can retry creation of a
|
||||
session with the **Pagure Dist Git** API and transferring packages from one
|
||||
user to another by using one of the helper scripts mentioned above.
|
||||
|
||||
The **Pagure Dist Git** API is used here and the base URL for the REST API of
|
||||
the production deployment is mentioned of in the variable ``PAGURE_URL``. The
|
||||
said script can be found
|
||||
`here <https://pagure.io/releng/blob/main/f/scripts/distgit/utilities.py>`_.
|
||||
|
||||
``sig_policy.py``
|
||||
""""
|
||||
|
||||
This script helps to enact the FESCo SIG policy. More about the FESCo SIG
|
||||
policy can be found in
|
||||
`this <https://docs.fedoraproject.org/en-US/fesco/SIG_policy/>`_ documentation.
|
||||
|
||||
The **Pagure Dist Git** API is used here and the base URL for the data elements
|
||||
of the production deployment is mentioned of in the variable
|
||||
``PAGURE_DIST_GIT_DATA_URL``. The said script can be found
|
||||
`here <https://pagure.io/releng/blob/main/f/scripts/fesco/sig-policy/sig_policy.py>`_.
|
||||
|
||||
``find_unblocked_orphans.py``
|
||||
""""
|
||||
|
||||
This script helps discover orphaned packages in Pagure that are unblocked in
|
||||
Koji adnd to show what assets depend on those orphans. This script works across
|
||||
various releases including EPEL and Fedora Linux repositories.
|
||||
|
||||
While a direct affiliation with **Pagure Dist Git** could not be drawn, the
|
||||
object ``cache`` depends of a ``dogpile`` cache database named
|
||||
``dist-git-orphans-cache.dbm``. The said script can be found
|
||||
`here <https://pagure.io/releng/blob/main/f/scripts/find_unblocked_orphans.py>`_.
|
||||
|
||||
``insert_sla.py``
|
||||
""""
|
||||
|
||||
This script helps choosing SLAs from a set specified by the Release Engineering
|
||||
on the branches available on **Pagure Dist Git** to avoid having arbitrary
|
||||
EOLs by working with the
|
||||
`Product Definition Center <https://pdc.fedoraproject.org/>`_.
|
||||
|
||||
While a direct affiliation with **Pagure Dist Git** could not be drawn, the
|
||||
changes made as a consequence of the execution of this script can lead to
|
||||
changes on **Pagure Dist Git**. The said script can be found
|
||||
`here <https://pagure.io/releng/blob/main/f/scripts/pdc/insert-sla.py>`_.
|
||||
|
||||
``check-retired-branches-pdc.py``
|
||||
""""
|
||||
|
||||
This script helps collect all the packages from the **Pagure Dist Git** to
|
||||
check if they are retired and queries
|
||||
`Product Definition Center <https://pdc.fedoraproject.org/>`_ to confirm they
|
||||
are not listed as active there.
|
||||
|
||||
The **Pagure Dist Git** API is used here and the base URL of the production
|
||||
deployment is mentioned of in the variable ``dist_git_base``. The said script
|
||||
can be found
|
||||
`here <https://pagure.io/releng/blob/main/f/scripts/check-retired-branches-pdc.py>`_.
|
||||
|
||||
``orphan-listed-packages.py``
|
||||
""""
|
||||
|
||||
This script helps orphaning listed packages in bulk when they fail to install
|
||||
or fail to build without being addressed for some time. This is a better
|
||||
solution than to perform the process once per package using the scripts.
|
||||
|
||||
The **Pagure Dist Git** API is used here and the base URL of the production
|
||||
deployment is mentioned of in the variable ``BASE_URL``. The said script
|
||||
can be found
|
||||
`here <https://pagure.io/releng/blob/main/f/scripts/distgit/orphan-listed-packages.py>`_.
|
||||
|
||||
``check-unretirement.py``
|
||||
""""
|
||||
|
||||
This script helps finding and listing the contributors of a Fedora Src Package
|
||||
and check their SLAs from
|
||||
`Product Definition Center <https://pdc.fedoraproject.org/>`_ to assert if the
|
||||
file `dead.package` exists in the package for unretirement confirmation.
|
||||
|
||||
The **Pagure Dist Git** API is used here and the URL of the production
|
||||
deployment for contributors and states are mentioned of in the variables
|
||||
``contributors_url`` and ``state_url`` respectively. The said script can be
|
||||
found
|
||||
`here <https://pagure.io/releng/blob/main/f/scripts/check-unretirement.py>`_.
|
||||
|
||||
``orphan-all-packages.py``
|
||||
""""
|
||||
|
||||
This script helps to orphan all the packages belonging to a certain set of
|
||||
users if they wish to no longer package software on Fedora Linux. The package
|
||||
is either transferred to a new owner from the committers or orphaned.
|
||||
|
||||
The **Pagure Dist Git** API is used here and the base URL of the production
|
||||
deployment is mentioned of in the variable ``PAGURE_URL``. The said script
|
||||
can be found
|
||||
`here <https://pagure.io/releng/blob/main/f/scripts/distgit/orphan-all-packages.py>`_.
|
||||
|
||||
``fedretire``
|
||||
""""
|
||||
|
||||
This script helps to retire packages that have either been orphaned for more
|
||||
than six weeks, depended on a retried package that was orphaned or because the
|
||||
said package was built and published.
|
||||
|
||||
The **Pagure Dist Git** API is used here and the base URL of the production
|
||||
deployment is mentioned of in the variable ``PAGURE_URL``. The said script
|
||||
can be found
|
||||
`here <https://pagure.io/releng/blob/main/f/scripts/fedretire>`_.
|
||||
|
||||
``find_failures.py``
|
||||
""""
|
||||
|
||||
This script helps discover failed builds in a given tag by delivering an output
|
||||
in rough HTML format for a given package belonging to a certain packager.
|
||||
|
||||
The **Pagure Dist Git** API is used here and the URL for the owner data of the
|
||||
production deployment is mentioned of in the variable ``ownerdataurl``. The
|
||||
said script can be found
|
||||
`here <https://pagure.io/releng/blob/main/f/scripts/find_failures.py>`_.
|
||||
|
||||
``follow-policy.py``
|
||||
""""
|
||||
|
||||
This script helps confirm if a package follows policy by asserting its
|
||||
installability and other checks, and in case it does not, it follows through
|
||||
with orphaning the said package belonging to a certain packager.
|
||||
|
||||
The **Pagure Dist Git** API is used here and the URL for the package owner data
|
||||
and confirming package orphaning of the production deployment is mentioned of
|
||||
in the variable ``pkg_owners`` and function ``update_bugs``. The said script
|
||||
can be found
|
||||
`here <https://pagure.io/releng/blob/main/f/scripts/ftbfs-fti/follow-policy.py>`_.
|
||||
|
||||
Changes proposed
|
||||
----
|
||||
|
||||
In order to switch out **Pagure Dist Git** for an alternative that provides a
|
||||
relatively compatible API pattern, the following changes must be made in the
|
||||
places mentioned below.
|
||||
|
||||
Necessary
|
||||
^^^^
|
||||
|
||||
- Release Engineering scripts heavily rely on the **Pagure Dist Git** API and
|
||||
hence it is important to find a replacement alternative that mimics the API
|
||||
scheme of **Pagure Dist Git** and replace the URL to the production
|
||||
deployment in the usages of the same in the scripts and documentations.
|
||||
|
||||
- In case the above is not possible, a translation layer (a meta service of
|
||||
sorts) needs to be created and hosted on the same location where the
|
||||
**Pagure Dist Git** is deployed and that can translate these requests into
|
||||
the API schema that can be comprehended by the replacement alternative.
|
||||
|
||||
- In case the above is not possible, all the aforementioned scripts and
|
||||
documentation will have to be rewritten to play well with the API schema of
|
||||
the replacement alternative chosen and checked thoroughly to confirm that
|
||||
the inter-working of the scripts are as robust as they are at the moment.
|
||||
|
||||
Unnecessary
|
||||
^^^^
|
||||
|
||||
- None
|
||||
Loading…
Add table
Add a link
Reference in a new issue