From daf8c47acf752dcabea6a1831f4b30eed78041fc Mon Sep 17 00:00:00 2001 From: Leonardo Rossetti Date: Tue, 22 Mar 2022 08:52:40 -0300 Subject: [PATCH] package automation --- docs/package-automation/index.rst | 97 +++++++++++++++++++ docs/package-automation/packit-and-pagure.rst | 31 ++++++ docs/package-automation/packit-triggers.rst | 37 +++++++ docs/package-automation/the-ci-platform.rst | 46 +++++++++ 4 files changed, 211 insertions(+) create mode 100644 docs/package-automation/index.rst create mode 100644 docs/package-automation/packit-and-pagure.rst create mode 100644 docs/package-automation/packit-triggers.rst create mode 100644 docs/package-automation/the-ci-platform.rst diff --git a/docs/package-automation/index.rst b/docs/package-automation/index.rst new file mode 100644 index 0000000..b987463 --- /dev/null +++ b/docs/package-automation/index.rst @@ -0,0 +1,97 @@ +Package Automation +================== + +Purpose +------- + +Not all Fedora applications are published as RPMs or are not always up to date to their latest code release, automating this workflow would fix that. + +Other teams would benefit from this work, such as those that build application containers or running integration tests based on RPM releases. + +Resources +--------- + +* Packit: https://packit.dev +* Bodhi: https://bodhi.fedoraproject.org/docs/ +* Packit Pagure support: https://github.com/packit/packit-service/issues/556 + + +Investigation +------------- + +The investigation was about testing several tools to accomplish this: Github actions, Jenkins, Packit and Bodhi. + +Packit does provide a great level of abstraction to both build and publish a package update, integrating with Bodhi and Koji in the way. + +Packit is also available in both Github and Gitlab as a CI application but not in Pagure (there is an open issue in their repository for implementing it). + +Here is the list of ideas/things we discussed/looked at: + +.. toctree:: + :maxdepth: 1 + + the-ci-platform + packit-triggers + packit-and-pagure + + +Conclusions +----------- + +Packit looks to be the path to take, specially because it can integrate with different tools, such as COPR and Bodhi. + +There is an open questions as how to deal with apps that are hosted in Pagure which are described in the "Proposed Roadmap" section. + +Proposed Roadmap +---------------- + +We propose the use of Packit CI Server, usually refered as Packit Service, because it can integrate with both Koji, COPR and Bodhi - it also means that +we don't need to maintain our own CI service to use packit. + +Packit CI is also available in Gitlab which means the migration to other SCMs will have minimal impact on this work. + +Github +------ + +* Enable/Install Packit Github app: https://packit.dev/docs/packit-service/#integrating-packit-as-a-service-into-your-project-or-organization-from-github-marketplace; +* Add a `.packit.yaml` file in each repository (can be done with `packit init`); +* Sample: https://github.com/fedora-infra/the-new-hotness/pull/432 + +Pagure +------ + +Pagure is not supported in Packit at the time of this writing. + +There is an open issue on the Packit team side to enable this in Pagure: https://github.com/packit/packit-service/issues/556. + +This could be an opportunity for CPE to work with the Packit team +to enable this in pagure, but that will require development effort on our side. + +We could split the initiative in two phases: + +* Phase 1: enable Packit in apps hosted in Github; +* Phase 2: Support Packit in Pagure and enable apps in Pagure to use Packit. + +Those do not depend on each other and could be worked in parallel but that would mean we need to dedicated more engineers to the initiative. + +Team +---- + +* A team of three engineers would be more than enough; +* One experienced engineer to lead the team. + +Skills +------ + +Github Apps +*********** + +* Git; +* YAML; +* Basic packit knowdlege; +* Basic build tools knowdlege (bodhi, koji, copr, etc); + +Packit + Pagure +*************** + +* All of the previous section + Python knowledge. diff --git a/docs/package-automation/packit-and-pagure.rst b/docs/package-automation/packit-and-pagure.rst new file mode 100644 index 0000000..bd8c2eb --- /dev/null +++ b/docs/package-automation/packit-and-pagure.rst @@ -0,0 +1,31 @@ +Packit and Pagure +================= + +Resources +---------- + +* https://github.com/packit/packit-service/issues/556 + +Discussion +---------- + +This discussion is about enabling Pagure support in Packit, what needs to be done and the amount of effort to do so. + +What would be needed +-------------------- + +Packit team estimates that would be 2-3 months implementation time but prioritization on their side is needed. + +The CPE team could contribute to this integration as the Packit codebase is structured in a way that a "Pagure Handler" +can be worked in parallel with other Packit code changes. + +Packit codebase is written in Python: https://github.com/packit/packit-service. + +Conclusion +---------- + +The CPE could work on this and enable Pagure support, if we really want to invest in Pagure support. + +The topic of centralizing events in fedora-messaging for all SCMs showed up during this discussion. +there were some talks about it but nobody never took the time to work/design it - it is whole new topic that +should not be mixed in this work but something to consider for a future initiative. diff --git a/docs/package-automation/packit-triggers.rst b/docs/package-automation/packit-triggers.rst new file mode 100644 index 0000000..f0f3438 --- /dev/null +++ b/docs/package-automation/packit-triggers.rst @@ -0,0 +1,37 @@ +Packit Triggers +=============== + +Resources +---------- + +* Packit Triggers: https://packit.dev/docs/configuration/#packit-service-jobs + +Discussion +---------- + +The discussion was about when and where to push package changes. + +Packit has three event triggers: + +* Pull Requests +* Commits +* Releases + +The topic showed up because pushing package changes on every and single Pull Requests sound unreasonable in some cases. + +What would be needed +-------------------- + +* PR CI testing is done using COPR, we can centralize those builds in a specific repository if we want to; +* Use feature branches to automatically push updates to specific Koji tags (manual releases or commits); + +Everything is configured in the packit yaml file. + +Conclusion +---------- + +The conclusion is that we all triggers for different scenarios: + +* Pull Requests triggers using COPR for CI RPM test; +* Merging of the main branch can push updates to rawhide; +* Github releases and/or commits to feature branches could trigger updates in "production koji targets". diff --git a/docs/package-automation/the-ci-platform.rst b/docs/package-automation/the-ci-platform.rst new file mode 100644 index 0000000..c41c3ed --- /dev/null +++ b/docs/package-automation/the-ci-platform.rst @@ -0,0 +1,46 @@ +The CI Platform +=============== + +Resources +---------- + +* Packit + Github: https://packit.dev/docs/packit-service/#integrating-packit-as-a-service-into-your-project-or-organization-from-github-marketplace +* Packit + Gitlab: https://packit.dev/docs/faq/#can-we-use-packit-with-gitlab + + +Discussion +---------- + +This discussion was about which CI platform to use, self-hosted (Jenkins/Zuul), Packit, Github actions etc. + + +What would be needed +-------------------- + +Github +****** + +* Add a `.packit.yaml` file in each repository (those can be created by running `packit init`); +* A spec file in each repository; +* Organization needs to have packit enabled (we already do). +* Around 31 repositories in total but we are not sure how many of those we can skip. + +Pagure +****** + +There is no Pagure support in Packit at this moment, which means we need some workarounds to make it happen: + +* Create mirrors in Github for those repositories; +* Use Jenkins/Zuul to run packit commands; +* Ignore it until we move those apps to something else. + +Conclusion +---------- + +Packit is the right choice for us: + +* Packit is available as a "CI Plataform" in both Github and Gitlab; +* It supports multiple "build-systems", such as Koji and COPR and it also supports Bodhi for package updates; +* All we need is to enable Packit in groups/organizations (already enabled in fedora-infra Github org) and a yaml file; +* It has a CLI which enables us to re-produce its behavior locally; +* It is not maintained by CPE but the team is close enough to us.