297 lines
12 KiB
ReStructuredText
297 lines
12 KiB
ReStructuredText
.. _releng:
|
|
|
|
Pagure Dist Git Interactions With Release Engineering Scripts
|
|
=============================================================
|
|
|
|
Release Engineering Scripts (as the name suggests) contains scripts necessary
|
|
for the Fedora Linux release processes.
|
|
|
|
Possible Interactions
|
|
---------------------
|
|
|
|
Documentation
|
|
^^^^^^^^^^^^^
|
|
|
|
The repository houses the following set of documentation that details on the
|
|
interaction that the Release Engineering team has with the **Pagure Dist Git**
|
|
platform.
|
|
|
|
Remove dist-git branches
|
|
""""""""""""""""""""""""
|
|
|
|
The documentation details on the process that is required to remove branches
|
|
from the source control management repositories when they are requested for by
|
|
the maintainers. The said documentation can be found
|
|
`here <https://pagure.io/releng/blob/main/f/docs/source/sop_remote_dist_git_branches.rst>`__.
|
|
|
|
Mass branching
|
|
""""""""""""""
|
|
|
|
The documentation details on the process of branching the pending release away
|
|
from ``devel/`` at the alpha freeze and how many elements (including
|
|
**Pagure Dist Git**) are involved in making it happen. The said documentation
|
|
can be found
|
|
`here <https://pagure.io/releng/blob/main/f/docs/source/sop_mass_branching.rst>`__.
|
|
|
|
Fedora Layered Image Build System
|
|
"""""""""""""""""""""""""""""""""
|
|
|
|
The documentation describes how the Fedora Layered Image Build System works and
|
|
how many elements (including **Pagure Dist Git**) are involved in making it
|
|
happen. The said documentation can be found
|
|
`here <https://pagure.io/releng/blob/main/f/docs/source/layered_image_build_service.rst>`__.
|
|
|
|
Adding a New Release Engineer
|
|
"""""""""""""""""""""""""""""
|
|
|
|
The documentation elucidates about the onboarding process for a new release
|
|
engineer and how they must be provided access to various elements (including
|
|
**Pagure Dist Git**) to be able to fulfill their purposes. The said
|
|
documentation can be found
|
|
`here <https://pagure.io/releng/blob/main/f/docs/source/sop_adding_new_release_engineer.rst>`__.
|
|
|
|
Mass Rebuild of Modules
|
|
"""""""""""""""""""""""
|
|
|
|
The documentation explains about how periodical mass rebuilding of modules are
|
|
performed in rawhide and how many elements (including **Pagure Dist Git**) are
|
|
involved in the said process. The said documentation can be found
|
|
`here <https://pagure.io/releng/blob/main/f/docs/source/sop_mass_rebuild_modules.rst>`__.
|
|
|
|
Unretiring a Package Branch
|
|
"""""""""""""""""""""""""""
|
|
|
|
The documentation elaborates about how certain *retired packages* can be
|
|
brought to an *unretired state* and how many elements (including
|
|
**Pagure Dist Git**) are involved in the said process. The said documentation
|
|
can be found
|
|
`here <https://pagure.io/releng/blob/main/f/docs/source/sop_unretire.rst>`__.
|
|
|
|
Update Critical Path
|
|
""""""""""""""""""""
|
|
|
|
The documentation explicates how certain applications can be added, updated or
|
|
removed from the Fedora Linux release's critical path and how many elements
|
|
(including **Pagure Dist Git**) are involved in making it happen. The said
|
|
documentation can be found
|
|
`here <https://pagure.io/releng/blob/main/f/docs/source/sop_update_critpath.rst>`__.
|
|
|
|
Troubleshooting
|
|
"""""""""""""""
|
|
|
|
The documentation demonstrates the behaviour of many elements (including
|
|
**Pagure Dist Git**) can be diagnosed and corrected in case those do not work
|
|
how they are supposed to. The said documentation can be found
|
|
`here <https://pagure.io/releng/blob/main/f/docs/source/troubleshooting.rst>`__.
|
|
|
|
Scripts
|
|
^^^^^^^
|
|
|
|
``retire_packagers.py``
|
|
"""""""""""""""""""""""
|
|
|
|
This script helps retiring a packager by orphaning all the packages that the
|
|
said packager is the main admin of, removing the access to the source control
|
|
management repository for the said packager and resetting their watch status.
|
|
|
|
The Pagure Dist Git API is used here and the base URL of the production
|
|
deployment is mentioned of in the variable ``dist_git_base``. The said script
|
|
can be found
|
|
`here <https://pagure.io/releng/blob/main/f/scripts/distgit/retire_packagers.py>`__.
|
|
|
|
|
|
``modulepkg.py``
|
|
""""""""""""""""
|
|
|
|
This script helps track modules that contain a specific package optionally
|
|
limited to its branch involving various systems like
|
|
`Product Definition Center <https://pdc.fedoraproject.org/>`_,
|
|
**Pagure Dist Git** and many other such elements.
|
|
|
|
The **Pagure Dist Git** API is used here and the base URL for ``/modules/`` of the
|
|
production deployment is mentioned of in the variable ``MODULES_SRC_URL``. The
|
|
said script can be found
|
|
`here <https://pagure.io/releng/blob/main/f/scripts/branching/modulepkg.py>`__.
|
|
|
|
``give-package.py``
|
|
"""""""""""""""""""
|
|
|
|
This script helps transfer a package in **Pagure Dist Git** from one user to
|
|
another or to orphan the said package by transferring the package to the
|
|
``orphan`` user using a privileged Pagure API access token.
|
|
|
|
The **Pagure Dist Git** API is used here and the base URL for the REST API of
|
|
the production deployment is mentioned of in the variable ``PAGURE_URL``. The
|
|
said script can be found
|
|
`here <https://pagure.io/releng/blob/main/f/scripts/distgit/give-package.py>`__.
|
|
|
|
``utilities.py``
|
|
""""""""""""""""
|
|
|
|
This script helps manipulate **Pagure Dist Git**. It can retry creation of a
|
|
session with the **Pagure Dist Git** API and transferring packages from one
|
|
user to another by using one of the helper scripts mentioned above.
|
|
|
|
The **Pagure Dist Git** API is used here and the base URL for the REST API of
|
|
the production deployment is mentioned of in the variable ``PAGURE_URL``. The
|
|
said script can be found
|
|
`here <https://pagure.io/releng/blob/main/f/scripts/distgit/utilities.py>`__.
|
|
|
|
``sig_policy.py``
|
|
"""""""""""""""""
|
|
|
|
This script helps to enact the FESCo SIG policy. More about the FESCo SIG
|
|
policy can be found in
|
|
`this <https://docs.fedoraproject.org/en-US/fesco/SIG_policy/>`_ documentation.
|
|
|
|
The **Pagure Dist Git** API is used here and the base URL for the data elements
|
|
of the production deployment is mentioned of in the variable
|
|
``PAGURE_DIST_GIT_DATA_URL``. The said script can be found
|
|
`here <https://pagure.io/releng/blob/main/f/scripts/fesco/sig-policy/sig_policy.py>`__.
|
|
|
|
``find_unblocked_orphans.py``
|
|
"""""""""""""""""""""""""""""
|
|
|
|
This script helps discover orphaned packages in Pagure that are unblocked in
|
|
Koji adnd to show what assets depend on those orphans. This script works across
|
|
various releases including EPEL and Fedora Linux repositories.
|
|
|
|
While a direct affiliation with **Pagure Dist Git** could not be drawn, the
|
|
object ``cache`` depends of a ``dogpile`` cache database named
|
|
``dist-git-orphans-cache.dbm``. The said script can be found
|
|
`here <https://pagure.io/releng/blob/main/f/scripts/find_unblocked_orphans.py>`__.
|
|
|
|
``insert_sla.py``
|
|
"""""""""""""""""
|
|
|
|
This script helps choosing SLAs from a set specified by the Release Engineering
|
|
on the branches available on **Pagure Dist Git** to avoid having arbitrary
|
|
EOLs by working with the
|
|
`Product Definition Center <https://pdc.fedoraproject.org/>`_.
|
|
|
|
While a direct affiliation with **Pagure Dist Git** could not be drawn, the
|
|
changes made as a consequence of the execution of this script can lead to
|
|
changes on **Pagure Dist Git**. The said script can be found
|
|
`here <https://pagure.io/releng/blob/main/f/scripts/pdc/insert-sla.py>`__.
|
|
|
|
``check-retired-branches-pdc.py``
|
|
"""""""""""""""""""""""""""""""""
|
|
|
|
This script helps collect all the packages from the **Pagure Dist Git** to
|
|
check if they are retired and queries
|
|
`Product Definition Center <https://pdc.fedoraproject.org/>`_ to confirm they
|
|
are not listed as active there.
|
|
|
|
The **Pagure Dist Git** API is used here and the base URL of the production
|
|
deployment is mentioned of in the variable ``dist_git_base``. The said script
|
|
can be found
|
|
`here <https://pagure.io/releng/blob/main/f/scripts/check-retired-branches-pdc.py>`__.
|
|
|
|
``orphan-listed-packages.py``
|
|
"""""""""""""""""""""""""""""
|
|
|
|
This script helps orphaning listed packages in bulk when they fail to install
|
|
or fail to build without being addressed for some time. This is a better
|
|
solution than to perform the process once per package using the scripts.
|
|
|
|
The **Pagure Dist Git** API is used here and the base URL of the production
|
|
deployment is mentioned of in the variable ``BASE_URL``. The said script
|
|
can be found
|
|
`here <https://pagure.io/releng/blob/main/f/scripts/distgit/orphan-listed-packages.py>`__.
|
|
|
|
``check-unretirement.py``
|
|
"""""""""""""""""""""""""
|
|
|
|
This script helps finding and listing the contributors of a Fedora Src Package
|
|
and check their SLAs from
|
|
`Product Definition Center <https://pdc.fedoraproject.org/>`_ to assert if the
|
|
file `dead.package` exists in the package for unretirement confirmation.
|
|
|
|
The **Pagure Dist Git** API is used here and the URL of the production
|
|
deployment for contributors and states are mentioned of in the variables
|
|
``contributors_url`` and ``state_url`` respectively. The said script can be
|
|
found
|
|
`here <https://pagure.io/releng/blob/main/f/scripts/check-unretirement.py>`__.
|
|
|
|
``orphan-all-packages.py``
|
|
""""""""""""""""""""""""""
|
|
|
|
This script helps to orphan all the packages belonging to a certain set of
|
|
users if they wish to no longer package software on Fedora Linux. The package
|
|
is either transferred to a new owner from the committers or orphaned.
|
|
|
|
The **Pagure Dist Git** API is used here and the base URL of the production
|
|
deployment is mentioned of in the variable ``PAGURE_URL``. The said script
|
|
can be found
|
|
`here <https://pagure.io/releng/blob/main/f/scripts/distgit/orphan-all-packages.py>`__.
|
|
|
|
``fedretire``
|
|
"""""""""""""
|
|
|
|
This script helps to retire packages that have either been orphaned for more
|
|
than six weeks, depended on a retried package that was orphaned or because the
|
|
said package was built and published.
|
|
|
|
The **Pagure Dist Git** API is used here and the base URL of the production
|
|
deployment is mentioned of in the variable ``PAGURE_URL``. The said script
|
|
can be found
|
|
`here <https://pagure.io/releng/blob/main/f/scripts/fedretire>`__.
|
|
|
|
*It is important to note that this script is not used anymore and in its*
|
|
*existing state the script probes into an HTTP API endpoint*
|
|
``/api/0/packages`` *that no longer exists now. The script was also last*
|
|
*updated over a couple of years back.*
|
|
|
|
``find_failures.py``
|
|
""""""""""""""""""""
|
|
|
|
This script helps discover failed builds in a given tag by delivering an output
|
|
in rough HTML format for a given package belonging to a certain packager.
|
|
|
|
The **Pagure Dist Git** API is used here and the URL for the owner data of the
|
|
production deployment is mentioned of in the variable ``ownerdataurl``. The
|
|
said script can be found
|
|
`here <https://pagure.io/releng/blob/main/f/scripts/find_failures.py>`__.
|
|
|
|
``follow-policy.py``
|
|
""""""""""""""""""""
|
|
|
|
This script helps confirm if a package follows policy by asserting its
|
|
installability and other checks, and in case it does not, it follows through
|
|
with orphaning the said package belonging to a certain packager.
|
|
|
|
The **Pagure Dist Git** API is used here and the URL for the package owner data
|
|
and confirming package orphaning of the production deployment is mentioned of
|
|
in the variable ``pkg_owners`` and function ``update_bugs``. The said script
|
|
can be found
|
|
`here <https://pagure.io/releng/blob/main/f/scripts/ftbfs-fti/follow-policy.py>`__.
|
|
|
|
Changes proposed
|
|
----------------
|
|
|
|
In order to switch out **Pagure Dist Git** for an alternative that provides a
|
|
relatively compatible API pattern, the following changes must be made in the
|
|
places mentioned below.
|
|
|
|
Necessary
|
|
^^^^^^^^^
|
|
|
|
- Release Engineering scripts heavily rely on the **Pagure Dist Git** API and
|
|
hence it is important to find a replacement alternative that mimics the API
|
|
scheme of **Pagure Dist Git** and replace the URL to the production
|
|
deployment in the usages of the same in the scripts and documentations.
|
|
|
|
- In case the above is not possible, a translation layer (a meta service of
|
|
sorts) needs to be created and hosted on the same location where the
|
|
**Pagure Dist Git** is deployed and that can translate these requests into
|
|
the API schema that can be comprehended by the replacement alternative.
|
|
|
|
- In case the above is not possible, all the aforementioned scripts and
|
|
documentation will have to be rewritten to play well with the API schema of
|
|
the replacement alternative chosen and checked thoroughly to confirm that
|
|
the inter-working of the scripts are as robust as they are at the moment.
|
|
|
|
Unnecessary
|
|
^^^^^^^^^^^
|
|
|
|
- None
|