From 8c2beb489e38840d75d2a86b22386471b3c43e34 Mon Sep 17 00:00:00 2001 From: Akashdeep Dhar Date: Mon, 27 Nov 2023 17:42:38 +0530 Subject: [PATCH] Add preliminary design details for the Dist Git 2.0 Signed-off-by: Akashdeep Dhar --- docs/dist-git-move/index.rst | 3 +- docs/dist-git-move/solution.rst | 212 ++++++++++++++++++++++++++++++++ 2 files changed, 214 insertions(+), 1 deletion(-) create mode 100644 docs/dist-git-move/solution.rst diff --git a/docs/dist-git-move/index.rst b/docs/dist-git-move/index.rst index e8e8230..8bb0d95 100644 --- a/docs/dist-git-move/index.rst +++ b/docs/dist-git-move/index.rst @@ -59,13 +59,14 @@ Interactions monitorgating releng -Summary +Index ------- .. toctree:: :maxdepth: 1 summary + solution Conclusions ----------- diff --git a/docs/dist-git-move/solution.rst b/docs/dist-git-move/solution.rst new file mode 100644 index 0000000..358b8cd --- /dev/null +++ b/docs/dist-git-move/solution.rst @@ -0,0 +1,212 @@ +.. _solution: + +Designing An Alternative Replacement +==== + +As Pagure Dist Git is a comprehensive service that is used by over ten other +infrastructure services - it is important to ensure that the replacement for +it does not break compatibility with them. The applications dependent on +Pagure Dist Git mostly depend on its Git API, its HTTP API, its Lookaside +Cache, hosted HTTP resources as well as the messages sent to the Fedora +Messaging hub. Ensuring that these stream of information remain consistent +throughout the transition to the newer replacement. + +It is also crucial to ensure that the solution is designed to be as agnostic +as possible in terms of the Git forge solution. This helps us avoid possible +decision biases that may arise as a consequence of a certain feature being +available or absent on a certain Git Forge solution. As Pagure Dist Git +provides a unique application around the good old Git workflow, availability +or absence of certain desired features similar to those from the existing +production deployment must be taken as a coincidence. + +Related services +---- + +The list of related services working alongside Pagure Dist Git are distributed +in two categories to mark whether they are maintained within the team or are +maintained by someone else. This helps us to decide any additional steps that +are required to ensure compatibility for the projects that depend on Pagure +Dist Git but the team cannot control the developmental direction of. Alongside +those, the projects that are maintained in the team are expected to have +changes introduced to keep up with the alternative. + +.. image:: ../_static/dist_git_system.drawio.svg + :target: ../_images/dist_git_system.drawio.svg + +Maintained by the team +^^^^ + +1. **Monitor Gating** + + Information about the dependence of the service on the production deployment + of Pagure Dist Git can be found + `here `_. + Depending on the investigations performed on the Monitor Gating project, + changes can be introduced to the project to ensure that the compatibility is + maintained with the alternative chosen for Pagure Dist Git. + +2. **The New Hotness** + + Information about the dependence of the service on the production deployment + of Pagure Dist Git can be found + `here `_. + Depending on the investigations performed on The New Hotness project, + changes can be introduced to the project to ensure that the compatibility is + maintained with the alternative chosen for Pagure Dist Git. + +3. **Release Engineering Scripts** + + Information about the dependence of the service on the production deployment + of Pagure Dist Git can be found + `here `_. + Depending on the investigations performed on the Release Engineering + Scripts, changes can be introduced to the project to ensure that the + compatibility is maintained with the alternative chosen for Pagure Dist Git. + +4. **Toddlers** + + Information about the dependence of the service on the production deployment + of Pagure Dist Git can be found + `here `_. + Depending on the investigations performed on the Toddlers project and the + scripts used, changes can be introduced to the project to ensure that the + compatibility is maintained with the alternative chosen for Pagure Dist Git. + +5. **Fedora Notifications** + + Information about the dependence of the service on the production deployment + of Pagure Dist Git can be found + `here `_. + Depending on the investigations performed on the Fedora Notifications + project, changes can be introduced to the project to ensure that the + compatibility is maintained with the alternative chosen for Pagure Dist Git. + +6. **Datanommer** + + Information about the dependence of the service on the production deployment + of Pagure Dist Git can be found + `here `_. + Depending on the investigations performed on the Fedora Messaging project, + changes can be introduced to the project to ensure that the compatibility is + maintained with the alternative chosen for Pagure Dist Git. + +Maintained by other teams +^^^^ + +1. **Fedpkg** + + Information about the dependence of the project on the production deployment + of Pagure Dist Git can be found + `here `_. + Depending on the investigations performed on the Fedpkg project, changes can + be recommended to the project maintainers to ensure that the compatibility + is maintained with the alternative chosen for Pagure Dist Git. + +2. **COPR** + + Information about the dependence of the service on the production deployment + of Pagure Dist Git can be found + `here `_. + Depending on the investigations performed on the COPR project, changes can + be recommended to the project maintainers to ensure that the compatibility + is maintained with the alternative chosen for Pagure Dist Git. + +3. **Fedora CI** + + Information about the dependence of the service on the production deployment + of Pagure Dist Git can be found + `here `_. + Depending on the investigations performed on the Fedora CI deployment, + changes can be recommended to the service maintainers to ensure that the + compatibility is maintained with the alternative chosen for Pagure Dist Git. + +4. **Bodhi** + + Information about the dependence of the service on the production deployment + of Pagure Dist Git can be found + `here `_. + Depending on the investigations performed on the Bodhi project, changes can + be recommended to the project maintainers to ensure that the compatibility + is maintained with the alternative chosen for Pagure Dist Git. + +5. **Packit** + + Information about the dependence of the service on the production deployment + of Pagure Dist Git will be available soon. Depending on the investigations + performed on the Packit project, changes can be recommended to the project + maintainers to ensure that the compatibility is maintained with the + alternative chosen for Pagure Dist Git. + +Interaction diagram +---- + +The following illustration explains how the Pagure Dist Git alternative would +work alongside other infrastructure applications and services. The replacement +would be a system chiefly consisting of three elements - **Messages Handler**, +**Dist Git Service** and **Compatibility Metaservice**. + +Messages Handler +^^^^ + +.. image:: ../_static/dist_git_system_msgh.drawio.svg + :target: ../_images/dist_git_system_msgh.drawio.svg + +This element is responsible for sending appropriately titled messages to the +Fedora Messaging bus by listening into events taking place in the Dist Git +Service deployment. It is important to ensure that the schema either more or +less the same or the changes to the schema are propagated across various +projects dependent on the messages emitted by the Dist Git Service to ensure +maximum compatibility. Some example of such projects are Fedora Notifications, +COPR, Fedora CI, Bodhi and Datanommer. + +It is strongly recommended to pick a Git Forge solution that supports +*webhooks-based event execution* as this can significantly help save efforts +on writing polling conditions that lead to send of messages from the Dist Git +service to the Fedora Messaging bus. As such one of the previously suggested +projects i.e. Webhook To Fedora Messaging can prove to be extremely useful +here when implemented as then, this element would be reduced to being only a +special configuration of the said project. + +Dist Git Service +^^^^ + +.. image:: ../_static/dist_git_system_main.drawio.svg + :target: ../_images/dist_git_system_main.drawio.svg + +This element is primarily responsible for storing repository resources using +Git (for RPM specfiles and codebase assets) and Git LFS (for large-sized +binaries and source tarballs) as well as for providing means to interact with +the said resources using an interactive API accessible both via HTTPS and SSH. +An internet frontend is also necessary for use-cases like managing inter and +intra repository collaboration, showing package statistics and information from +various other relevant services etc. + +It is strongly recommended to the pick a Git Forge solution that allows for +granular access control management in terms of both users and groups associated +with a repository. Having the access control of the said solution play well +with the Fedora Project's existing AAA stack is crucial to ensure that the +group accesses can be governed from Fedora Account System groups using SAML. +Also, having an HTTP API that allows for customizing its capabilities to match +those of Pagure Dist Git is necessary. + +Compatibility Metaservice +^^^^ + +.. image:: ../_static/dist_git_system_meta.drawio.svg + :target: ../_images/dist_git_system_meta.drawio.svg + +The services that depend on Pagure Dist Git and are maintained by the team can +easily have changes introduced to them to match the compatibility of the +changed Dist Git Service platform. The process can be slower for the projects +that depend on Pagure Dist Git and are not maintained by the team and hence it +is important to maintain an equivalent amount of compatibility while the +changes are introduced in the subsequent release by supporting the same API +calls while setting a obsolescence timeline. + +The metaservice is expected to mimic the existing Pagure Dist Git API to help +applications and services like Fedpkg, COPR, Fedora CI, Bodhi and Packit obtain +the same results as they would when the older API would have been used. This is +done by hosting the metaservice in the same location as the Pagure Dist Git API +and letting it receive the requests from the clients. These requests are then +translated to work accordingly with the newer API schema to receive responses +which are again translated to the older schema.