Infrastructure/arc
1
0
Fork 0
Code Issues 1 Pull requests Projects Releases Packages Wiki Activity Actions

fix parsing errors and sphinx warnings

Browse source 
Signed-off-by: Ryan Lerch <rlerch@redhat.com>
This commit is contained in:
Ryan Lercho 2023-11-16 08:02:56 +10:00 • committed by zlopez
parent 8fb9b2fdf0
commit ba720c3d77
98 changed files with 4799 additions and 4788 deletions
Download patch file Download diff file Expand all files Collapse all files

298
docs/github2fedmsg/enhancing_the_development_environment.rst
View file

@ -1,170 +1,166 @@
.. _enhancing_the_development_environment:
Enhancing the Development Environment
========================================
=====================================
The current state of the development environment of the github2fedmsg project
leaves a lot to be desired. This document attempts to address all the existing
tech debts bound to the github2fedmsg project repository.
The current state of the development environment of the github2fedmsg project leaves a
lot to be desired. This document attempts to address all the existing tech debts bound
to the github2fedmsg project repository.
Investigations
----
--------------
**Continuous Integration**
1. Apparently, there has been a continuous integration configured for the
repository, which was last active on Jun 17, 2020 on
`PR #27 <https://github.com/fedora-infra/github2fedmsg/pull/27>`_.
This continuous integration was set up on a Jenkins instance of Fedora
Infrastructure on the CentOS CI namespace (as deduced by the `details
URL <https://jenkins-fedora-apps.apps.ci.centos.org/job/github2fedmsg/job/PR-27/1/display/redirect>`_)
but is either no longer available or can only be accessed via the Red Hat
VPN. If the former is the case, we would want to reconfigure the
continuous integration in the project repository but if latter is the
case, we would want to ensure that the continuous integration is
accessible to the wider community to promote contributor participation in
the project.
2. In order to set up a continuous integration in the project repository,
`GitHub Actions <https://github.com/features/actions>`_ can be put into
use as the project repository itself is hosted on GitHub, thus allowing
for integration and support to a greater extent. It is also possible to
use Fedora Linux containers in GitHub Actions, as evidenced by their use
in `Meetbot Logs <https://github.com/fedora-infra/mote/blob/main/.github/workflows/main.yml#L10>`_,
`CentOS Duffy <https://github.com/CentOS/duffy/blob/dev/.github/workflows/main.yaml#L8>`_,
`Fedora Messaging Notifier <https://github.com/fedora-infra/fmn/blob/fmn-next/.github/workflows/main.yaml#L8>`_,
`MDAPI <https://github.com/fedora-infra/mdapi/blob/develop/.github/workflows/main.yml#L9>`_
and many more such projects, which would help replicate the production
environment accurately for the relevant build and test jobs. Other
continuous integration options like `Zuul CI <https://zuul-ci.org/>`_,
`Travis CI <https://www.travis-ci.com/>`_ etc. can also be considered on
the basis of particular needs.
1. Apparently, there has been a continuous integration configured for the
repository, which was last active on Jun 17, 2020 on `PR #27
<https://github.com/fedora-infra/github2fedmsg/pull/27>`_. This continuous
integration was set up on a Jenkins instance of Fedora Infrastructure on the
CentOS CI namespace (as deduced by the `details URL
<https://jenkins-fedora-apps.apps.ci.centos.org/job/github2fedmsg/job/PR-27/1/display/redirect>`_)
but is either no longer available or can only be accessed via the Red Hat VPN. If
the former is the case, we would want to reconfigure the continuous integration
in the project repository but if latter is the case, we would want to ensure that
the continuous integration is accessible to the wider community to promote
contributor participation in the project.
2. In order to set up a continuous integration in the project repository, `GitHub
Actions <https://github.com/features/actions>`_ can be put into use as the
project repository itself is hosted on GitHub, thus allowing for integration and
support to a greater extent. It is also possible to use Fedora Linux containers
in GitHub Actions, as evidenced by their use in `Meetbot Logs
<https://github.com/fedora-infra/mote/blob/main/.github/workflows/main.yml#L10>`__,
`CentOS Duffy
<https://github.com/CentOS/duffy/blob/dev/.github/workflows/main.yaml#L8>`_,
`Fedora Messaging Notifier
<https://github.com/fedora-infra/fmn/blob/fmn-next/.github/workflows/main.yaml#L8>`__,
`MDAPI
<https://github.com/fedora-infra/mdapi/blob/develop/.github/workflows/main.yml#L9>`_
and many more such projects, which would help replicate the production
environment accurately for the relevant build and test jobs. Other continuous
integration options like `Zuul CI <https://zuul-ci.org/>`_, `Travis CI
<https://www.travis-ci.com/>`_ etc. can also be considered on the basis of
particular needs.
**Development Environment**
1. A small amount of information about setting up a virtual environment and
installing the project using
`setuptools <https://setuptools.pypa.io/en/latest/>`_
is currently available on the
`README.rst document <https://github.com/fedora-infra/github2fedmsg/blob/develop/README.rst>`_
in the project repository. Although, the current setup works fine in a
specific environment having (now EOLed) Python 2.7.x, it can be useful to
abstract the use of setuptools, using dependency management tools like
`poetry <https://python-poetry.org/>`_,
`pipenv <https://pipenv.pypa.io/en/latest/>`_ etc.
to steer away from the unnecessary attention in creating and updating
the setuptools configuration, facilitating for automatic dependency
updates and maintaining a list of dependencies specific to development,
testing and production scenarios.
2. Maintaining codebase to uphold established quality standards will be of
great help, both, for the convenience of existing contributors and to
help onboard potential ones. In the current setup,
`flake8 <https://flake8.pycqa.org/en/latest/>`_ is
`used <https://github.com/fedora-infra/github2fedmsg/blob/develop/tox.ini#L13>`_
for linting the codebase. In addition to flake8,
`isort <https://isort.readthedocs.io/en/latest/>`_ and
`black <https://black.readthedocs.io/en/stable/>`_ can
be used for sorting of dependency imports and enforcing code styles
wherever necessary. This has been of great help in maintaining code
quality, as evidenced by their uses in
`Meetbot Logs <https://github.com/fedora-infra/mote/blob/main/tox.ini#L27-L56>`_,
`CentOS Duffy <https://github.com/CentOS/duffy/blob/dev/tox.ini#L25-L41>`_,
`Fedora Messaging Notifier <https://github.com/fedora-infra/fmn/blob/fmn-next/tox.ini#L16-L33>`_,
`MDAPI <https://github.com/fedora-infra/mdapi/blob/develop/tox.ini#L26-L54>`_
and many more such projects.
1. A small amount of information about setting up a virtual environment and
installing the project using `setuptools
<https://setuptools.pypa.io/en/latest/>`_ is currently available on the
`README.rst document
<https://github.com/fedora-infra/github2fedmsg/blob/develop/README.rst>`_ in the
project repository. Although, the current setup works fine in a specific
environment having (now EOLed) Python 2.7.x, it can be useful to abstract the use
of setuptools, using dependency management tools like `poetry
<https://python-poetry.org/>`_, `pipenv <https://pipenv.pypa.io/en/latest/>`_
etc. to steer away from the unnecessary attention in creating and updating the
setuptools configuration, facilitating for automatic dependency updates and
maintaining a list of dependencies specific to development, testing and
production scenarios.
2. Maintaining codebase to uphold established quality standards will be of great
help, both, for the convenience of existing contributors and to help onboard
potential ones. In the current setup, `flake8
<https://flake8.pycqa.org/en/latest/>`__ is `used
<https://github.com/fedora-infra/github2fedmsg/blob/develop/tox.ini#L13>`__ for
linting the codebase. In addition to flake8, `isort
<https://isort.readthedocs.io/en/latest/>`__ and `black
<https://black.readthedocs.io/en/stable/>`__ can be used for sorting of
dependency imports and enforcing code styles wherever necessary. This has been of
great help in maintaining code quality, as evidenced by their uses in `Meetbot
Logs <https://github.com/fedora-infra/mote/blob/main/tox.ini#L27-L56>`__, `CentOS
Duffy <https://github.com/CentOS/duffy/blob/dev/tox.ini#L25-L41>`__, `Fedora
Messaging Notifier
<https://github.com/fedora-infra/fmn/blob/fmn-next/tox.ini#L16-L33>`__, `MDAPI
<https://github.com/fedora-infra/mdapi/blob/develop/tox.ini#L26-L54>`__ and many
more such projects.
**Automatic Dependency Updates**
1. Projects such as
`Meetbot Logs <https://github.com/fedora-infra/mote/blob/main/.github/dependabot.yml>`_,
`CentOS Duffy <https://github.com/CentOS/duffy/blob/dev/.github/dependabot.yml>`_ and
`MDAPI <https://github.com/fedora-infra/mdapi/blob/develop/.github/dependabot.yml>`_
use `GitHub Dependabot <https://github.com/dependabot>`_ to help
automatically update the dependency lockfile as and when new updates and
vulnerability fixes come out for the unexcluded dependencies, matching
the version requirement mentioned in the project configuration file.
While the former case requires having a valid GitHub Dependabot
configuration in the repository, the latter does not and will work
regardless of having it or not. Being GitHub's own security worker bot,
it has a greater extent of integration with the project repositories
available on GitHub.
2. Projects such as
`Fedora Messaging Notifier <https://github.com/fedora-infra/fmn/blob/fmn-next/renovate.json>`_
and `Anitya <https://github.com/fedora-infra/anitya/blob/master/renovate.json>`_
use `Renovate <https://github.com/apps/renovate>`_ to help automatically
update the dependency lockfile as and when new updates or vulnerability
fixes come out for the unexcluded dependencies, matching the version
requirement mentioned in the project configuration file. Renovate is
highly flexible with its configuration, allows for checking of dependency
lockfile for verifying checksums, organizing custom schedules for updates
and also supports auto-merging of the dependency update pull requests.
1. Projects such as `Meetbot Logs
<https://github.com/fedora-infra/mote/blob/main/.github/dependabot.yml>`__,
`CentOS Duffy
<https://github.com/CentOS/duffy/blob/dev/.github/dependabot.yml>`__ and `MDAPI
<https://github.com/fedora-infra/mdapi/blob/develop/.github/dependabot.yml>`__
use `GitHub Dependabot <https://github.com/dependabot>`__ to help automatically
update the dependency lockfile as and when new updates and vulnerability fixes
come out for the unexcluded dependencies, matching the version requirement
mentioned in the project configuration file. While the former case requires
having a valid GitHub Dependabot configuration in the repository, the latter does
not and will work regardless of having it or not. Being GitHub's own security
worker bot, it has a greater extent of integration with the project repositories
available on GitHub.
2. Projects such as `Fedora Messaging Notifier
<https://github.com/fedora-infra/fmn/blob/fmn-next/renovate.json>`__ and `Anitya
<https://github.com/fedora-infra/anitya/blob/master/renovate.json>`_ use
`Renovate <https://github.com/apps/renovate>`_ to help automatically update the
dependency lockfile as and when new updates or vulnerability fixes come out for
the unexcluded dependencies, matching the version requirement mentioned in the
project configuration file. Renovate is highly flexible with its configuration,
allows for checking of dependency lockfile for verifying checksums, organizing
custom schedules for updates and also supports auto-merging of the dependency
update pull requests.
**Outdated dependencies**
1. The project makes use of a certain set of dependencies which are either
left in a pre-release state or are no longer actively maintained. While
these dependencies can be put to use in the project in its current state,
it is very likely that a lot of security vulnerabilities must have flown
under the radar unpatched, in the absence of features updates, security
patches and bug fixes. Apart from that, being older libraries/frameworks -
the availability of updated documentation is likely to be scarce for
these, thus making the development using these dependencies more
challenging for the maintainers.
2. Moving forward, we would want to look into either possible replacing the
following dependencies or avoiding their use entirely.
1. The project makes use of a certain set of dependencies which are either left in a
pre-release state or are no longer actively maintained. While these dependencies
can be put to use in the project in its current state, it is very likely that a
lot of security vulnerabilities must have flown under the radar unpatched, in the
absence of features updates, security patches and bug fixes. Apart from that,
being older libraries/frameworks - the availability of updated documentation is
likely to be scarce for these, thus making the development using these
dependencies more challenging for the maintainers.
2. Moving forward, we would want to look into either possible replacing the
following dependencies or avoiding their use entirely.
1. `Velruse <https://pypi.org/project/velruse/>`_, had its last
update almost *9 years back* in
`Aug 30, 2013 <https://pypi.org/project/velruse/1.1.1/>`_.
2. `WebError <https://pypi.org/project/WebError/>`_, had its last update
almost *6 years back* in
`Apr 10, 2016 <https://pypi.org/project/WebError/0.13.1/>`_ and the
`project documentation states <https://github.com/Pylons/weberror/blob/e395f6d0f0becb299b905197c4577d3b74ad40ae/README.rst#notice>`_
that the project is no longer actively maintained.
3. `pyramid-mako <https://pypi.org/project/pyramid-mako/>`_ had its last
update almost *3 years back* in
`Aug 19, 2019 <https://pypi.org/project/pyramid-mako/1.1.0/>`_.
4. `transaction <https://pypi.org/project/transaction/>`_ had its last
update almost *2 years back* in
`Dec 11, 2020 <https://pypi.org/project/transaction/3.0.1/>`_.
1. `Velruse <https://pypi.org/project/velruse/>`_, had its last update almost *9
years back* in `Aug 30, 2013 <https://pypi.org/project/velruse/1.1.1/>`_.
2. `WebError <https://pypi.org/project/WebError/>`_, had its last update almost
*6 years back* in `Apr 10, 2016 <https://pypi.org/project/WebError/0.13.1/>`_
and the `project documentation states
<https://github.com/Pylons/weberror/blob/e395f6d0f0becb299b905197c4577d3b74ad40ae/README.rst#notice>`_
that the project is no longer actively maintained.
3. `pyramid-mako <https://pypi.org/project/pyramid-mako/>`_ had its last update
almost *3 years back* in `Aug 19, 2019
<https://pypi.org/project/pyramid-mako/1.1.0/>`_.
4. `transaction <https://pypi.org/project/transaction/>`_ had its last update
almost *2 years back* in `Dec 11, 2020
<https://pypi.org/project/transaction/3.0.1/>`_.
**Contributor's Quality of Life**
1. Certain additional documentations such as CODE OF CONDUCT, CONTRIBUTING
GUIDELINES etc. can either be added or linked in, to set the expectations
right for both existing and potential contributors. The wiki feature
present in the project repository can be preferred to be put into use
for keeping documentation like these which are less likely to change
across multiple releases. These documentation are available in
repositories such as MDAPI
`[1] <https://github.com/fedora-infra/mdapi/blob/develop/CODE_OF_CONDUCT.md>`_
`[2] <https://github.com/fedora-infra/mdapi/blob/develop/CONTRIBUTING.md>`_,
Meetbot Logs
`[1] <https://github.com/fedora-infra/mote/blob/main/CODE_OF_CONDUCT.md>`_
`[2] <https://github.com/fedora-infra/mote/blob/main/CONTRIBUTING.md>`_,
CentOS Duffy
`[1] <https://github.com/CentOS/duffy/blob/dev/CONTRIBUTING.md>`_ etc.
and can be included here too.
2. Tests and code coverage should be added to ensure that the project code
remains functional over time across the addition of features, fixing of
bugs, patching of security vulnerabilities etc. The
`tests <https://github.com/fedora-infra/github2fedmsg/blob/develop/github2fedmsg/tests.py>`_
currently available in the project repository have a limited scope and
are not automated using continuous integration to test out the code
`on pull requests <https://docs.github.com/en/developers/webhooks-and-events/webhooks/webhook-events-and-payloads#pull_request>`_,
`on pushes <https://docs.github.com/en/developers/webhooks-and-events/webhooks/webhook-events-and-payloads#push>`_
and
`before deployment workflows <https://docs.github.com/en/developers/webhooks-and-events/webhooks/webhook-events-and-payloads#workflow_dispatch>`_.
These tests and code coverage have been implemented in
`Meetbot Logs <https://github.com/fedora-infra/mote/blob/main/tox.ini#L7-L19>`_,
`CentOS Duffy <https://github.com/CentOS/duffy/blob/dev/tox.ini#L7-L18>`_,
`Fedora Messaging Notifier <https://github.com/fedora-infra/fmn/blob/fmn-next/tox.ini#L7-L14>`_,
`MDAPI <https://github.com/fedora-infra/mdapi/blob/develop/tox.ini#L7-L17>`_
and many more such repositories and are automated using GitHub Actions.
1. Certain additional documentations such as CODE OF CONDUCT, CONTRIBUTING
GUIDELINES etc. can either be added or linked in, to set the expectations right
for both existing and potential contributors. The wiki feature present in the
project repository can be preferred to be put into use for keeping documentation
like these which are less likely to change across multiple releases. These
documentation are available in repositories such as MDAPI `[1]
<https://github.com/fedora-infra/mdapi/blob/develop/CODE_OF_CONDUCT.md>`__ `[2]
<https://github.com/fedora-infra/mdapi/blob/develop/CONTRIBUTING.md>`__, Meetbot
Logs `[1] <https://github.com/fedora-infra/mote/blob/main/CODE_OF_CONDUCT.md>`__
`[2] <https://github.com/fedora-infra/mote/blob/main/CONTRIBUTING.md>`__, CentOS
Duffy `[1] <https://github.com/CentOS/duffy/blob/dev/CONTRIBUTING.md>`__ etc. and
can be included here too.
2. Tests and code coverage should be added to ensure that the project code remains
functional over time across the addition of features, fixing of bugs, patching of
security vulnerabilities etc. The `tests
<https://github.com/fedora-infra/github2fedmsg/blob/develop/github2fedmsg/tests.py>`_
currently available in the project repository have a limited scope and are not
automated using continuous integration to test out the code `on pull requests
<https://docs.github.com/en/developers/webhooks-and-events/webhooks/webhook-events-and-payloads#pull_request>`_,
`on pushes
<https://docs.github.com/en/developers/webhooks-and-events/webhooks/webhook-events-and-payloads#push>`_
and `before deployment workflows
<https://docs.github.com/en/developers/webhooks-and-events/webhooks/webhook-events-and-payloads#workflow_dispatch>`_.
These tests and code coverage have been implemented in `Meetbot Logs
<https://github.com/fedora-infra/mote/blob/main/tox.ini#L7-L19>`__, `CentOS Duffy
<https://github.com/CentOS/duffy/blob/dev/tox.ini#L7-L18>`__, `Fedora Messaging
Notifier <https://github.com/fedora-infra/fmn/blob/fmn-next/tox.ini#L7-L14>`__,
`MDAPI <https://github.com/fedora-infra/mdapi/blob/develop/tox.ini#L7-L17>`__ and
many more such repositories and are automated using GitHub Actions.
**Automating Deployments to OpenShift**
1. The project currently makes use of semantic versioning for its releases
and requires manual intervention for deploying every distinct release to
both `staging <https://apps.fedoraproject.org/github2fedmsg>`_ and
`production <https://apps.stg.fedoraproject.org/github2fedmsg>`_
environments. The effort can be reduced by the use of webhooks which
would help monitor distinct release on the project repository and then
automatically run deployment scripts to put the new version up on
the staging environment. For this, GitHub webhooks can be configured to
be used on the said OpenShift cluster and the production deployment is
suggested to require manual intervention to make way for additional due
diligence before deployment.
1. The project currently makes use of semantic versioning for its releases and
requires manual intervention for deploying every distinct release to both
`staging <https://apps.fedoraproject.org/github2fedmsg>`_ and `production
<https://apps.stg.fedoraproject.org/github2fedmsg>`_ environments. The effort can
be reduced by the use of webhooks which would help monitor distinct release on
the project repository and then automatically run deployment scripts to put the
new version up on the staging environment. For this, GitHub webhooks can be
configured to be used on the said OpenShift cluster and the production deployment
is suggested to require manual intervention to make way for additional due
diligence before deployment.

54
docs/github2fedmsg/index.rst
View file

@ -5,10 +5,10 @@ Purpose
-------
This investigation's goal is to address tech debt of github2fedmsg application.
github2fedmsg application is used to translate events emitted by github to fedora messaging
messages using github webhooks. It also has database of projects to watch and provides web
interface to add new repositories. It authenticates with FAS and Github, linking accounts
together.
github2fedmsg application is used to translate events emitted by github to fedora
messaging messages using github webhooks. It also has database of projects to watch and
provides web interface to add new repositories. It authenticates with FAS and Github,
linking accounts together.
The app has git-hub2fedmsg bot on GitHub that is being used to link the FAS account to
Github.
@ -16,34 +16,35 @@ Github.
Resources
---------
* github2fedmsg initiative proposal: https://pagure.io/cpe/initiatives-proposal/issue/24
* github2fedmsg repo: https://github.com/fedora-infra/github2fedmsg
* github2fedmsg web: https://apps.fedoraproject.org/github2fedmsg
* github2fedmsg ansible: https://pagure.io/fedora-infra/ansible/blob/main/f/roles/github2fedmsg
* github2fedmsg SOP: https://docs.fedoraproject.org/en-US/infra/sysadmin_guide/github2fedmsg/
- github2fedmsg initiative proposal: https://pagure.io/cpe/initiatives-proposal/issue/24
- github2fedmsg repo: https://github.com/fedora-infra/github2fedmsg
- github2fedmsg web: https://apps.fedoraproject.org/github2fedmsg
- github2fedmsg ansible:
https://pagure.io/fedora-infra/ansible/blob/main/f/roles/github2fedmsg
- github2fedmsg SOP:
https://docs.fedoraproject.org/en-US/infra/sysadmin_guide/github2fedmsg/
Requirements
------------
* Porting to Flask and authlib
* Linking GitHub account to FAS account
* Listening to GitHub events
* Translating GitHub events to Fedora messages
* Add/remove repositories to watch
* CI
* Documentation
* OpenShift deployment
- Porting to Flask and authlib
- Linking GitHub account to FAS account
- Listening to GitHub events
- Translating GitHub events to Fedora messages
- Add/remove repositories to watch
- CI
- Documentation
- OpenShift deployment
What Needs To Be Addressed?
---------------------------
* Create message schema
* python2 -> python3
* migration to OpenShift
* Documentation - only README and SOPs are available
* Update development environment
* There is no staging environment
- Create message schema
- python2 -> python3
- migration to OpenShift
- Documentation - only README and SOPs are available
- Update development environment
- There is no staging environment
Investigation
-------------
@ -57,12 +58,11 @@ Here is the list of ideas/things we discussed/looked at:
rewrite
enhancing_the_development_environment
Conclusions
-----------
Investigation team recommends to not work on this app, but instead work
on :doc:`../webhook2fedmsg/index`.
Investigation team recommends to not work on this app, but instead work on
:doc:`../webhook2fedmsg/index`.
Proposed Roadmap
----------------

76
docs/github2fedmsg/revitalization.rst
View file

@ -3,57 +3,71 @@
Revitalization of github2fedmsg application
===========================================
This document will investigate the possibility of updating the code base we currently have.
This document will investigate the possibility of updating the code base we currently
have.
What needs to be addressed (from our index)
-------------------------------------------
**Python 2 -> Python 3**
| *Current situation*: Written in Python 2.7.x which is now `EOLed <https://www.python.org/doc/sunset-python-2/>`_.
| *Suggested outcome*: Python 3.10.x (and above).
| *Reason(s)*: Benefits from performance improvements, bug fixes, security patches etc. of the language.
| *Current situation*: Written in Python 2.7.x which is now `EOLed
<https://www.python.org/doc/sunset-python-2/>`_.
| *Suggested outcome*: Python 3.10.x (and above).
| *Reason(s)*: Benefits from performance improvements, bug fixes, security patches
etc. of the language.
**Migration to OpenShift**
| *Current situation*: The app is running in a Virtual Machine (VM).
| *Suggested outcome*: Deploy it as an OpenShift service.
| *Reason(s)*: Automated deployments and we already have an OpenShift cluster.
| *Current situation*: The app is running in a Virtual Machine (VM).
| *Suggested outcome*: Deploy it as an OpenShift service.
| *Reason(s)*: Automated deployments and we already have an OpenShift cluster.
**Documentation**
| *Current situation*: Only a brief README file and `SOP <https://docs.fedoraproject.org/en-US/infra/sysadmin_guide/github2fedmsg/>`_ are available.
| *Suggested outcome*: It would be great if the documentation was more extensive. It should include instructions about setting up the development environment, contributing guidelines, code of conduct, etc. This is also covered in the `dev environment enhancement doc <https://fedora-arc.readthedocs.io/en/latest/github2fedmsg/enhancing_the_development_environment.html>`_.
| *Reason(s)*: Easier onboarding and setting expectations for contributors.
| *Current situation*: Only a brief README file and `SOP
<https://docs.fedoraproject.org/en-US/infra/sysadmin_guide/github2fedmsg/>`_ are
available.
| *Suggested outcome*: It would be great if the documentation was more extensive.
It should include instructions about setting up the development environment,
contributing guidelines, code of conduct, etc. This is also covered in the `dev
environment enhancement doc
<https://fedora-arc.readthedocs.io/en/latest/github2fedmsg/enhancing_the_development_environment.html>`_.
| *Reason(s)*: Easier onboarding and setting expectations for contributors.
**Porting to Authlib**
| *Current outcome*: The authentication library used is `Velruse <https://velruse.readthedocs.io/en/latest/usage.html>`_.
| *Suggested using*: `Authlib <https://authlib.org/>`_.
| *Reason(s)*: Velruse hasn't been updated in 9 years and Authlib is in active development.
| *Current outcome*: The authentication library used is `Velruse
<https://velruse.readthedocs.io/en/latest/usage.html>`_.
| *Suggested using*: `Authlib <https://authlib.org/>`_.
| *Reason(s)*: Velruse hasn't been updated in 9 years and Authlib is in active
development.
**Development environment**
| *Current situation*: This is covered in depth in our other document about enhancing the development environment.
| *Current situation*: This is covered in depth in our other document about
enhancing the development environment.
Notes
-----
* setuptools>=58 `breaks support for 2to3 <https://setuptools.pypa.io/en/latest/history.html#v58-0-0>`_.
* 2to3 is `deprecated since Python 3.11 and will be removed in Python 3.13 <https://docs.python.org/3/library/2to3.html>`_.
- setuptools>=58 `breaks support for 2to3
<https://setuptools.pypa.io/en/latest/history.html#v58-0-0>`_.
- 2to3 is `deprecated since Python 3.11 and will be removed in Python 3.13
<https://docs.python.org/3/library/2to3.html>`_.
The Good and the Bad
--------------------
**Advantages**
1. The project is written in (now EOLed) Python 2.7.x
and as Python 3 is currently in active development, the project can
benefit from the performance improvements, bug fixes, and security patches
included in the language itself.
2. The project makes use of a certain set of dependencies, the support of
which has not been ported from Python 2 to Python 3. Porting into
Python 3 would necessitate the use of newer dependencies which would be
more updated, secure and maintained.
1. The project is written in (now EOLed) Python 2.7.x and as Python 3 is currently
in active development, the project can benefit from the performance improvements,
bug fixes, and security patches included in the language itself.
2. The project makes use of a certain set of dependencies, the support of which has
not been ported from Python 2 to Python 3. Porting into Python 3 would
necessitate the use of newer dependencies which would be more updated, secure and
maintained.
**Disadvantages**
1. The move from Python 2 to Python 3 would require special attention into
finding the alternatives for the dependencies which either support only
Python 2 or are unmaintained as of 2022. Inability to find those would
render the rewrite unsuccessful.
2. We will continue using the `Pyramid web framework <https://trypyramid.com/>`_,
however the team lacks the expertise with it. This makes the app difficult
to maintain and team members would have to upskill in this area.
1. The move from Python 2 to Python 3 would require special attention into finding
the alternatives for the dependencies which either support only Python 2 or are
unmaintained as of 2022. Inability to find those would render the rewrite
unsuccessful.
2. We will continue using the `Pyramid web framework <https://trypyramid.com/>`_,
however the team lacks the expertise with it. This makes the app difficult to
maintain and team members would have to upskill in this area.

180
docs/github2fedmsg/rewrite.rst
View file

@ -1,111 +1,135 @@
.. _rewrite:
Complete Rewrite of github2fedmsg Application
====
=============================================
This document will investigate the possibility of rewriting github2fedmsg from scratch.
Elements that could use our attention
----
-------------------------------------
**Project configuration**
| *Currently uses*: Setuptools (`setup.py <https://github.com/fedora-infra/github2fedmsg/blob/develop/setup.py>`_)
| *Suggested using*: Poetry (`pyproject.toml <https://python-poetry.org/docs/pyproject/>`_)
| *Reason(s)*: Easier dependency management, project configuration, testing information etc.
| *Currently uses*: Setuptools (`setup.py
<https://github.com/fedora-infra/github2fedmsg/blob/develop/setup.py>`_)
| *Suggested using*: Poetry (`pyproject.toml
<https://python-poetry.org/docs/pyproject/>`_)
| *Reason(s)*: Easier dependency management, project configuration, testing
information etc.
**Backend language**
| *Currently uses*: Python 2.7.x (now `EOLed <https://www.python.org/doc/sunset-python-2/>`_)
| *Suggested using*: Python 3.10.x (and above)
| *Reason(s)*: Benefit from performance improvements, bug fixes, security patches etc. on the language
| *Currently uses*: Python 2.7.x (now `EOLed
<https://www.python.org/doc/sunset-python-2/>`_)
| *Suggested using*: Python 3.10.x (and above)
| *Reason(s)*: Benefit from performance improvements, bug fixes, security patches
etc. on the language
**Base framework**
| *Currently uses*: `Pyramid <https://trypyramid.com/>`_
| *Suggested using*: `Flask <https://flask.palletsprojects.com/en/2.2.x/>`_ or `FastAPI <https://fastapi.tiangolo.com/>`_
| *Reason(s)*: More knowledge about the suggested ones in the team, streamlined development, more features etc.
| *Currently uses*: `Pyramid <https://trypyramid.com/>`_
| *Suggested using*: `Flask <https://flask.palletsprojects.com/en/2.2.x/>`_ or
`FastAPI <https://fastapi.tiangolo.com/>`_
| *Reason(s)*: More knowledge about the suggested ones in the team, streamlined
development, more features etc.
**Templating implementation**
| *Currently uses*: `Mako <https://www.makotemplates.org/>`_
| *Suggested using*: `Jinja <https://jinja.palletsprojects.com/en/3.1.x/>`_
| *Reason(s)*: More knowledge about the suggested ones in the team, standardized templating etc.
| *Currently uses*: `Mako <https://www.makotemplates.org/>`_
| *Suggested using*: `Jinja <https://jinja.palletsprojects.com/en/3.1.x/>`_
| *Reason(s)*: More knowledge about the suggested ones in the team, standardized
templating etc.
**Authentication library**
| *Currently uses*: `Velruse <https://velruse.readthedocs.io/en/latest/usage.html>`_
| *Suggested using*: `Authlib <https://authlib.org/>`_
| *Reason(s)*: Authlib is actively maintained while Velruse had its last update `9 years ago <https://pypi.org/project/velruse/1.1.1/>`_.
| *Currently uses*: `Velruse
<https://velruse.readthedocs.io/en/latest/usage.html>`_
| *Suggested using*: `Authlib <https://authlib.org/>`_
| *Reason(s)*: Authlib is actively maintained while Velruse had its last update `9
years ago <https://pypi.org/project/velruse/1.1.1/>`_.
**Frontend CSS library**
| *Currently uses*: `Bootstrap 3.1.1 (Fedora) <https://github.com/fedora-infra/github2fedmsg/tree/develop/github2fedmsg/static/bootstrap-3.1.1-fedora>`_
| *Suggested using*: Updated version of Bootstrap for eg. `Boostrap 5 <https://getbootstrap.com/docs/5.0/getting-started/introduction/>`_
| *Reason(s)*: Web interface looks/feels dated and is not responsive to variable widths.
| *Currently uses*: `Bootstrap 3.1.1 (Fedora)
<https://github.com/fedora-infra/github2fedmsg/tree/develop/github2fedmsg/static/bootstrap-3.1.1-fedora>`_
| *Suggested using*: Updated version of Bootstrap for eg. `Boostrap 5
<https://getbootstrap.com/docs/5.0/getting-started/introduction/>`_
| *Reason(s)*: Web interface looks/feels dated and is not responsive to variable
widths.
**Notification popups**
| *Currently uses*: `Messenger 1.4.1 <https://github.com/fedora-infra/github2fedmsg/tree/develop/github2fedmsg/static/messenger-1.4.1>`_
| *Suggested using*: Updated version of Messenger or any maintained alternative
| *Reason(s)*: Library was last added `8 years back <https://github.com/fedora-infra/github2fedmsg/commit/cb98f6c7b74bb50c7cb8ee58aff81033c23300ee>`_ and could use an update.
| *Currently uses*: `Messenger 1.4.1
<https://github.com/fedora-infra/github2fedmsg/tree/develop/github2fedmsg/static/messenger-1.4.1>`_
| *Suggested using*: Updated version of Messenger or any maintained alternative
| *Reason(s)*: Library was last added `8 years back
<https://github.com/fedora-infra/github2fedmsg/commit/cb98f6c7b74bb50c7cb8ee58aff81033c23300ee>`_
and could use an update.
**Interface interactivity**
| *Currently uses*: `JQuery 1.11.0 <https://github.com/fedora-infra/github2fedmsg/blob/develop/github2fedmsg/static/jquery-1.11.0.min.js>`_
| *Suggested using*: Updated version of JQuery for eg. `JQuery 3.6.1 <https://code.jquery.com/jquery-3.6.1.min.js>`_
| *Reason(s)*: Library was last added `9 years back <https://github.com/fedora-infra/github2fedmsg/commit/8985732f1e22a565dfd3ce9964896e9e4f86657e>`_ and could use an update.
| *Currently uses*: `JQuery 1.11.0
<https://github.com/fedora-infra/github2fedmsg/blob/develop/github2fedmsg/static/jquery-1.11.0.min.js>`_
| *Suggested using*: Updated version of JQuery for eg. `JQuery 3.6.1
<https://code.jquery.com/jquery-3.6.1.min.js>`_
| *Reason(s)*: Library was last added `9 years back
<https://github.com/fedora-infra/github2fedmsg/commit/8985732f1e22a565dfd3ce9964896e9e4f86657e>`_
and could use an update.
**API modelling**
| *Currently uses*: In-place `creation of data structures <https://github.com/fedora-infra/github2fedmsg/blob/develop/github2fedmsg/views/webhooks.py#L247-L253>`_ in worker functions
| *Suggested using*: Data validation and typing libraries like `Pydantic <https://pydantic-docs.helpmanual.io/>`_
| *Reason(s)*: Better codebase modularity and readability, auto-validation of API data types etc.
| *Currently uses*: In-place `creation of data structures
<https://github.com/fedora-infra/github2fedmsg/blob/develop/github2fedmsg/views/webhooks.py#L247-L253>`_
in worker functions
| *Suggested using*: Data validation and typing libraries like `Pydantic
<https://pydantic-docs.helpmanual.io/>`_
| *Reason(s)*: Better codebase modularity and readability, auto-validation of API
data types etc.
**Command line parsing**
| *Currently uses*: None. Just making do with `stdio` and `stdout`.
| *Suggested using*: `Click <https://click.palletsprojects.com/en/8.1.x/>`_ or any other alternatives
| *Reason(s)*: Easy-to-use command line creation, handling of options and commands with auto-generated help etc.
| *Currently uses*: None. Just making do with `stdio` and `stdout`.
| *Suggested using*: `Click <https://click.palletsprojects.com/en/8.1.x/>`_ or any
other alternatives
| *Reason(s)*: Easy-to-use command line creation, handling of options and commands
with auto-generated help etc.
The good and the bad
----
--------------------
**Advantages**
1. In its current state, the project is written in (now EOLed) Python 2.7.x
and as Python 3 is currently in active development, the project can
benefit from the performance improvements, bug fixes and security patches
included in the language itself.
2. The project makes use of a certain set of dependencies, the support of
which, have not been ported from Python 2 to Python 3. Porting into
Python 3 would necessitate the use of newer dependencies which would be
more updated, secure and maintained.
3. In its current state, the project makes use of the
`Pyramid <https://trypyramid.com/>`_ web framework, the expertise of which
the team currently lacks, making it unmaintainable right now. A rewrite
using libraries and frameworks that the team is acquainted with would help.
4. The web interface makes use of non-default
`Mako <https://www.makotemplates.org/>`_-based
`templates <https://github.com/fedora-infra/github2fedmsg/tree/develop/github2fedmsg/templates>`_
that work right now but due to their obscure nature, it can become
difficult to debug the templates later. A rewrite of templates using
a standardized
`Jinja <https://jinja.palletsprojects.com/en/3.1.x/>`_ format would go a
long way.
5. The web interface could use updated versions (or replacements) of
unmaintained JavaScript and CSS3 dependencies (for now, it has
`Bootstrap 3.1.1 <https://github.com/fedora-infra/github2fedmsg/tree/develop/github2fedmsg/static/bootstrap-3.1.1-fedora>`_,
`Messenger 1.4.1 <https://github.com/fedora-infra/github2fedmsg/tree/develop/github2fedmsg/static/messenger-1.4.1>`_,
`JQuery 1.11.0 <https://github.com/fedora-infra/github2fedmsg/blob/develop/github2fedmsg/static/jquery-1.11.0.min.js>`_
etc.) in order to introduce responsiveness and provide a better look and
feel.
6. The API model can be standardized with the use of typing and data
validation libraries such as
`Pydantic <https://pydantic-docs.helpmanual.io/>`_ instead of
`forming data structures <https://github.com/fedora-infra/github2fedmsg/blob/develop/github2fedmsg/views/webhooks.py#L247-L253>`_
in the worker function itself, making the code more modular and
maintainable in the long run.
1. In its current state, the project is written in (now EOLed) Python 2.7.x and as
Python 3 is currently in active development, the project can benefit from the
performance improvements, bug fixes and security patches included in the language
itself.
2. The project makes use of a certain set of dependencies, the support of which,
have not been ported from Python 2 to Python 3. Porting into Python 3 would
necessitate the use of newer dependencies which would be more updated, secure and
maintained.
3. In its current state, the project makes use of the `Pyramid
<https://trypyramid.com/>`_ web framework, the expertise of which the team
currently lacks, making it unmaintainable right now. A rewrite using libraries
and frameworks that the team is acquainted with would help.
4. The web interface makes use of non-default `Mako
<https://www.makotemplates.org/>`_-based `templates
<https://github.com/fedora-infra/github2fedmsg/tree/develop/github2fedmsg/templates>`_
that work right now but due to their obscure nature, it can become difficult to
debug the templates later. A rewrite of templates using a standardized `Jinja
<https://jinja.palletsprojects.com/en/3.1.x/>`_ format would go a long way.
5. The web interface could use updated versions (or replacements) of unmaintained
JavaScript and CSS3 dependencies (for now, it has `Bootstrap 3.1.1
<https://github.com/fedora-infra/github2fedmsg/tree/develop/github2fedmsg/static/bootstrap-3.1.1-fedora>`_,
`Messenger 1.4.1
<https://github.com/fedora-infra/github2fedmsg/tree/develop/github2fedmsg/static/messenger-1.4.1>`_,
`JQuery 1.11.0
<https://github.com/fedora-infra/github2fedmsg/blob/develop/github2fedmsg/static/jquery-1.11.0.min.js>`_
etc.) in order to introduce responsiveness and provide a better look and feel.
6. The API model can be standardized with the use of typing and data validation
libraries such as `Pydantic <https://pydantic-docs.helpmanual.io/>`_ instead of
`forming data structures
<https://github.com/fedora-infra/github2fedmsg/blob/develop/github2fedmsg/views/webhooks.py#L247-L253>`_
in the worker function itself, making the code more modular and maintainable in
the long run.
**Disadvantages**
1. The move from Python 2 to Python 3 would require special attention into
the finding alternatives for the dependencies which either support only
Python 2 or are unmaintained as of 2022. Inability to find those would
render the rewrite unsuccessful.
2. In the wake of new ways to interact with an API, some features (eg. web
interface, when API is suggested to be used by itself) of the current
version have become redundant and a 1:1 rewrite would not help as it
would frontport those features too.
3. While reworking the database model, it is possible that the existing dump
cannot be fully imported due to the changes introduced in the database
schema to better work the newer reimplementation, thereby potentially
causing data loss.
1. The move from Python 2 to Python 3 would require special attention into the
finding alternatives for the dependencies which either support only Python 2 or
are unmaintained as of 2022. Inability to find those would render the rewrite
unsuccessful.
2. In the wake of new ways to interact with an API, some features (eg. web
interface, when API is suggested to be used by itself) of the current version
have become redundant and a 1:1 rewrite would not help as it would frontport
those features too.
3. While reworking the database model, it is possible that the existing dump cannot
be fully imported due to the changes introduced in the database schema to better
work the newer reimplementation, thereby potentially causing data loss.