Infrastructure/arc
1
0
Fork 0
Code Issues 1 Pull requests Projects Releases Packages Wiki Activity Actions

Add preliminary design details for the Dist Git 2.0

Browse source 
Signed-off-by: Akashdeep Dhar <akashdeep.dhar@gmail.com>
This commit is contained in:
Akashdeep Dhar 2023-11-27 17:42:38 +05:30
parent 9d75197c03
commit 8c2beb489e
2 changed files with 214 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

3
docs/dist-git-move/index.rst
View file

@ -59,13 +59,14 @@ Interactions
monitorgating
releng
Summary
Index
-------
.. toctree::
:maxdepth: 1
summary
solution
Conclusions
-----------

212
docs/dist-git-move/solution.rst Normal file
View file

@ -0,0 +1,212 @@
.. _solution:
Designing An Alternative Replacement
====
As Pagure Dist Git is a comprehensive service that is used by over ten other
infrastructure services - it is important to ensure that the replacement for
it does not break compatibility with them. The applications dependent on
Pagure Dist Git mostly depend on its Git API, its HTTP API, its Lookaside
Cache, hosted HTTP resources as well as the messages sent to the Fedora
Messaging hub. Ensuring that these stream of information remain consistent
throughout the transition to the newer replacement.
It is also crucial to ensure that the solution is designed to be as agnostic
as possible in terms of the Git forge solution. This helps us avoid possible
decision biases that may arise as a consequence of a certain feature being
available or absent on a certain Git Forge solution. As Pagure Dist Git
provides a unique application around the good old Git workflow, availability
or absence of certain desired features similar to those from the existing
production deployment must be taken as a coincidence.
Related services
----
The list of related services working alongside Pagure Dist Git are distributed
in two categories to mark whether they are maintained within the team or are
maintained by someone else. This helps us to decide any additional steps that
are required to ensure compatibility for the projects that depend on Pagure
Dist Git but the team cannot control the developmental direction of. Alongside
those, the projects that are maintained in the team are expected to have
changes introduced to keep up with the alternative.
.. image:: ../_static/dist_git_system.drawio.svg
:target: ../_images/dist_git_system.drawio.svg
Maintained by the team
^^^^
1. **Monitor Gating**
Information about the dependence of the service on the production deployment
of Pagure Dist Git can be found
`here <https://fedora-arc.readthedocs.io/en/latest/dist-git-move/monitorgating.html>`_.
Depending on the investigations performed on the Monitor Gating project,
changes can be introduced to the project to ensure that the compatibility is
maintained with the alternative chosen for Pagure Dist Git.
2. **The New Hotness**
Information about the dependence of the service on the production deployment
of Pagure Dist Git can be found
`here <https://fedora-arc.readthedocs.io/en/latest/dist-git-move/hotness.html>`_.
Depending on the investigations performed on The New Hotness project,
changes can be introduced to the project to ensure that the compatibility is
maintained with the alternative chosen for Pagure Dist Git.
3. **Release Engineering Scripts**
Information about the dependence of the service on the production deployment
of Pagure Dist Git can be found
`here <https://fedora-arc.readthedocs.io/en/latest/dist-git-move/releng.html>`_.
Depending on the investigations performed on the Release Engineering
Scripts, changes can be introduced to the project to ensure that the
compatibility is maintained with the alternative chosen for Pagure Dist Git.
4. **Toddlers**
Information about the dependence of the service on the production deployment
of Pagure Dist Git can be found
`here <https://fedora-arc.readthedocs.io/en/latest/dist-git-move/toddlers.html>`_.
Depending on the investigations performed on the Toddlers project and the
scripts used, changes can be introduced to the project to ensure that the
compatibility is maintained with the alternative chosen for Pagure Dist Git.
5. **Fedora Notifications**
Information about the dependence of the service on the production deployment
of Pagure Dist Git can be found
`here <https://fedora-arc.readthedocs.io/en/latest/dist-git-move/notifications.html>`_.
Depending on the investigations performed on the Fedora Notifications
project, changes can be introduced to the project to ensure that the
compatibility is maintained with the alternative chosen for Pagure Dist Git.
6. **Datanommer**
Information about the dependence of the service on the production deployment
of Pagure Dist Git can be found
`here <https://fedora-arc.readthedocs.io/en/latest/dist-git-move/messaging.html>`_.
Depending on the investigations performed on the Fedora Messaging project,
changes can be introduced to the project to ensure that the compatibility is
maintained with the alternative chosen for Pagure Dist Git.
Maintained by other teams
^^^^
1. **Fedpkg**
Information about the dependence of the project on the production deployment
of Pagure Dist Git can be found
`here <https://fedora-arc.readthedocs.io/en/latest/dist-git-move/fedpkg.html>`_.
Depending on the investigations performed on the Fedpkg project, changes can
be recommended to the project maintainers to ensure that the compatibility
is maintained with the alternative chosen for Pagure Dist Git.
2. **COPR**
Information about the dependence of the service on the production deployment
of Pagure Dist Git can be found
`here <https://fedora-arc.readthedocs.io/en/latest/dist-git-move/copr.html>`_.
Depending on the investigations performed on the COPR project, changes can
be recommended to the project maintainers to ensure that the compatibility
is maintained with the alternative chosen for Pagure Dist Git.
3. **Fedora CI**
Information about the dependence of the service on the production deployment
of Pagure Dist Git can be found
`here <https://fedora-arc.readthedocs.io/en/latest/dist-git-move/ci.html>`_.
Depending on the investigations performed on the Fedora CI deployment,
changes can be recommended to the service maintainers to ensure that the
compatibility is maintained with the alternative chosen for Pagure Dist Git.
4. **Bodhi**
Information about the dependence of the service on the production deployment
of Pagure Dist Git can be found
`here <https://fedora-arc.readthedocs.io/en/latest/dist-git-move/ci.html>`_.
Depending on the investigations performed on the Bodhi project, changes can
be recommended to the project maintainers to ensure that the compatibility
is maintained with the alternative chosen for Pagure Dist Git.
5. **Packit**
Information about the dependence of the service on the production deployment
of Pagure Dist Git will be available soon. Depending on the investigations
performed on the Packit project, changes can be recommended to the project
maintainers to ensure that the compatibility is maintained with the
alternative chosen for Pagure Dist Git.
Interaction diagram
----
The following illustration explains how the Pagure Dist Git alternative would
work alongside other infrastructure applications and services. The replacement
would be a system chiefly consisting of three elements - **Messages Handler**,
**Dist Git Service** and **Compatibility Metaservice**.
Messages Handler
^^^^
.. image:: ../_static/dist_git_system_msgh.drawio.svg
:target: ../_images/dist_git_system_msgh.drawio.svg
This element is responsible for sending appropriately titled messages to the
Fedora Messaging bus by listening into events taking place in the Dist Git
Service deployment. It is important to ensure that the schema either more or
less the same or the changes to the schema are propagated across various
projects dependent on the messages emitted by the Dist Git Service to ensure
maximum compatibility. Some example of such projects are Fedora Notifications,
COPR, Fedora CI, Bodhi and Datanommer.
It is strongly recommended to pick a Git Forge solution that supports
*webhooks-based event execution* as this can significantly help save efforts
on writing polling conditions that lead to send of messages from the Dist Git
service to the Fedora Messaging bus. As such one of the previously suggested
projects i.e. Webhook To Fedora Messaging can prove to be extremely useful
here when implemented as then, this element would be reduced to being only a
special configuration of the said project.
Dist Git Service
^^^^
.. image:: ../_static/dist_git_system_main.drawio.svg
:target: ../_images/dist_git_system_main.drawio.svg
This element is primarily responsible for storing repository resources using
Git (for RPM specfiles and codebase assets) and Git LFS (for large-sized
binaries and source tarballs) as well as for providing means to interact with
the said resources using an interactive API accessible both via HTTPS and SSH.
An internet frontend is also necessary for use-cases like managing inter and
intra repository collaboration, showing package statistics and information from
various other relevant services etc.
It is strongly recommended to the pick a Git Forge solution that allows for
granular access control management in terms of both users and groups associated
with a repository. Having the access control of the said solution play well
with the Fedora Project's existing AAA stack is crucial to ensure that the
group accesses can be governed from Fedora Account System groups using SAML.
Also, having an HTTP API that allows for customizing its capabilities to match
those of Pagure Dist Git is necessary.
Compatibility Metaservice
^^^^
.. image:: ../_static/dist_git_system_meta.drawio.svg
:target: ../_images/dist_git_system_meta.drawio.svg
The services that depend on Pagure Dist Git and are maintained by the team can
easily have changes introduced to them to match the compatibility of the
changed Dist Git Service platform. The process can be slower for the projects
that depend on Pagure Dist Git and are not maintained by the team and hence it
is important to maintain an equivalent amount of compatibility while the
changes are introduced in the subsequent release by supporting the same API
calls while setting a obsolescence timeline.
The metaservice is expected to mimic the existing Pagure Dist Git API to help
applications and services like Fedpkg, COPR, Fedora CI, Bodhi and Packit obtain
the same results as they would when the older API would have been used. This is
done by hosting the metaservice in the same location as the Pagure Dist Git API
and letting it receive the requests from the clients. These requests are then
translated to work accordingly with the newer API schema to receive responses
which are again translated to the older schema.