From 23d915946b09edd45ca8752b7b9f04a7edb30b6d Mon Sep 17 00:00:00 2001 From: Akashdeep Dhar Date: Tue, 15 Nov 2022 08:08:55 +0530 Subject: [PATCH] github2fedmsg : add docs to improve dev env Signed-off-by: Akashdeep Dhar --- .../enhancement_of_repository.rst | 16 -- .../enhancing_the_development_environment.rst | 170 ++++++++++++++++++ docs/github2fedmsg/index.rst | 2 +- 3 files changed, 171 insertions(+), 17 deletions(-) delete mode 100644 docs/github2fedmsg/enhancement_of_repository.rst create mode 100644 docs/github2fedmsg/enhancing_the_development_environment.rst diff --git a/docs/github2fedmsg/enhancement_of_repository.rst b/docs/github2fedmsg/enhancement_of_repository.rst deleted file mode 100644 index 8ce4de2..0000000 --- a/docs/github2fedmsg/enhancement_of_repository.rst +++ /dev/null @@ -1,16 +0,0 @@ -.. _enhancement_of_repository: - -Enhancement of dev env for github2fedmsg -======================================== - -There is plenty of tech debt regarding the repository. This document -will try to address all the current tech debt tied to kerneltest repository. - -Investigation -------------- - -* CI - missing -* Development environment - Just README instructions for virtual env -* Automatic dependency updater - recommending renovate -* When moving to OpenShift we can consolidate adding automatic deployment with - GitHub webhooks diff --git a/docs/github2fedmsg/enhancing_the_development_environment.rst b/docs/github2fedmsg/enhancing_the_development_environment.rst new file mode 100644 index 0000000..8b8d564 --- /dev/null +++ b/docs/github2fedmsg/enhancing_the_development_environment.rst @@ -0,0 +1,170 @@ +.. _enhancing_the_development_environment: + +Enhancing the Development Environment +======================================== + +The current state of the development environment of the github2fedmsg project +leaves a lot to be desired. This document attempts to address all the existing +tech debts bound to the github2fedmsg project repository. + +Investigations +---- + +**Continuous Integration** + 1. Apparently, there has been a continuous integration configured for the + repository, which was last active on Jun 17, 2020 on + `PR #27 `_. + This continuous integration was set up on a Jenkins instance of Fedora + Infrastructure on the CentOS CI namespace (as deduced by the `details + URL `_) + but is either no longer available or can only be accessed via the Red Hat + VPN. If the former is the case, we would want to reconfigure the + continuous integration in the project repository but if latter is the + case, we would want to ensure that the continuous integration is + accessible to the wider community to promote contributor participation in + the project. + 2. In order to set up a continuous integration in the project repository, + `GitHub Actions `_ can be put into + use as the project repository itself is hosted on GitHub, thus allowing + for integration and support to a greater extent. It is also possible to + use Fedora Linux containers in GitHub Actions, as evidenced by their use + in `Meetbot Logs `_, + `CentOS Duffy `_, + `Fedora Messaging Notifier `_, + `MDAPI `_ + and many more such projects, which would help replicate the production + environment accurately for the relevant build and test jobs. Other + continuous integration options like `Zuul CI `_, + `Travis CI `_ etc. can also be considered on + the basis of particular needs. + +**Development Environment** + 1. A small amount of information about setting up a virtual environment and + installing the project using + `setuptools `_ + is currently available on the + `README.rst document `_ + in the project repository. Although, the current setup works fine in a + specific environment having (now EOLed) Python 2.7.x, it can be useful to + abstract the use of setuptools, using dependency management tools like + `poetry `_, + `pipenv `_ etc. + to steer away from the unnecessary attention in creating and updating + the setuptools configuration, facilitating for automatic dependency + updates and maintaining a list of dependencies specific to development, + testing and production scenarios. + 2. Maintaining codebase to uphold established quality standards will be of + great help, both, for the convenience of existing contributors and to + help onboard potential ones. In the current setup, + `flake8 `_ is + `used `_ + for linting the codebase. In addition to flake8, + `isort `_ and + `black `_ can + be used for sorting of dependency imports and enforcing code styles + wherever necessary. This has been of great help in maintaining code + quality, as evidenced by their uses in + `Meetbot Logs `_, + `CentOS Duffy `_, + `Fedora Messaging Notifier `_, + `MDAPI `_ + and many more such projects. + +**Automatic Dependency Updates** + 1. Projects such as + `Meetbot Logs `_, + `CentOS Duffy `_ and + `MDAPI `_ + use `GitHub Dependabot `_ to help + automatically update the dependency lockfile as and when new updates and + vulnerability fixes come out for the unexcluded dependencies, matching + the version requirement mentioned in the project configuration file. + While the former case requires having a valid GitHub Dependabot + configuration in the repository, the latter does not and will work + regardless of having it or not. Being GitHub's own security worker bot, + it has a greater extent of integration with the project repositories + available on GitHub. + 2. Projects such as + `Fedora Messaging Notifier `_ + and `Anitya `_ + use `Renovate `_ to help automatically + update the dependency lockfile as and when new updates or vulnerability + fixes come out for the unexcluded dependencies, matching the version + requirement mentioned in the project configuration file. Renovate is + highly flexible with its configuration, allows for checking of dependency + lockfile for verifying checksums, organizing custom schedules for updates + and also supports auto-merging of the dependency update pull requests. + +**Outdated dependencies** + 1. The project makes use of a certain set of dependencies which are either + left in a pre-release state or are no longer actively maintained. While + these dependencies can be put to use in the project in its current state, + it is very likely that a lot of security vulnerabilities must have flown + under the radar unpatched, in the absence of features updates, security + patches and bug fixes. Apart from that, being older libraries/frameworks - + the availability of updated documentation is likely to be scarce for + these, thus making the development using these dependencies more + challenging for the maintainers. + 2. Moving forward, we would want to look into either possible replacing the + following dependencies or avoiding their use entirely. + + 1. `Velruse `_, had its last + update almost *9 years back* in + `Aug 30, 2013 `_. + 2. `WebError `_, had its last update + almost *6 years back* in + `Apr 10, 2016 `_ and the + `project documentation states `_ + that the project is no longer actively maintained. + 3. `pyramid-mako `_ had its last + update almost *3 years back* in + `Aug 19, 2019 `_. + 4. `transaction `_ had its last + update almost *2 years back* in + `Dec 11, 2020 `_. + +**Contributor's Quality of Life** + 1. Certain additional documentations such as CODE OF CONDUCT, CONTRIBUTING + GUIDELINES etc. can either be added or linked in, to set the expectations + right for both existing and potential contributors. The wiki feature + present in the project repository can be preferred to be put into use + for keeping documentation like these which are less likely to change + across multiple releases. These documentation are available in + repositories such as MDAPI + `[1] `_ + `[2] `_, + Meetbot Logs + `[1] `_ + `[2] `_, + CentOS Duffy + `[1] `_ etc. + and can be included here too. + 2. Tests and code coverage should be added to ensure that the project code + remains functional over time across the addition of features, fixing of + bugs, patching of security vulnerabilities etc. The + `tests `_ + currently available in the project repository have a limited scope and + are not automated using continuous integration to test out the code + `on pull requests `_, + `on pushes `_ + and + `before deployment workflows `_. + These tests and code coverage have been implemented in + `Meetbot Logs `_, + `CentOS Duffy `_, + `Fedora Messaging Notifier `_, + `MDAPI `_ + and many more such repositories and are automated using GitHub Actions. + +**Automating Deployments to OpenShift** + 1. The project currently makes use of semantic versioning for its releases + and requires manual intervention for deploying every distinct release to + both `staging `_ and + `production `_ + environments. The effort can be reduced by the use of webhooks which + would help monitor distinct release on the project repository and then + automatically run deployment scripts to put the new version up on + the staging environment. For this, GitHub webhooks can be configured to + be used on the said OpenShift cluster and the production deployment is + suggested to require manual intervention to make way for additional due + diligence before deployment. diff --git a/docs/github2fedmsg/index.rst b/docs/github2fedmsg/index.rst index e5cd493..3294753 100644 --- a/docs/github2fedmsg/index.rst +++ b/docs/github2fedmsg/index.rst @@ -55,7 +55,7 @@ Here is the list of ideas/things we discussed/looked at: revitalization rewrite - enhancement_of_repository + enhancing_the_development_environment Conclusions