Update the developer doc

Mention the cookiecutter template.

Signed-off-by: Aurélien Bompard <aurelien@bompard.org>

modules/developer_guide/pages/auth.adoc

@@ -78,9 +78,20 @@ Pyramid extension that uses OAuthlib. It does not appear to be actively
 maintained, but it is a reasonable starting point for our few Pyramid
 applications.
 
+==== Authlib
+
+https://authlib.org/[Authlib] is a Python library to build OAuth and
+OpenID Connect servers and clients. It has integration layers for Flask
+and Django as protocol servers and clients, as well as client integration
+layers for Requests, HTTPX, Starlette and FastAPI. As a client app
+authenticating with OpenID Connect, it is most useful if several
+authentication providers are to be supported (such as social networks).
+
 ==== Flask-OIDC
 
-link:#flask-oidc[Flask-OIDC] is a Flask extension.
+https://github.com/fedora-infra/flask-oidc/[Flask-OIDC] is a Flask
+extension that makes using Authlib easier when using Flask with only one
+authentication provider (such as Fedora's identity provider).
 
 ==== Mozilla-Django-OIDC
 

modules/developer_guide/pages/code-style.adoc

@@ -9,7 +9,7 @@ head as they move from project to project.
 We follow the https://www.python.org/dev/peps/pep-0008/[PEP8] style
 guide for Python. Projects should make it easy to check new code for
 PEP8 violations preferably by
-https://pypi.python.org/pypi/flake8[flake8]. It is up to the individual
+https://docs.astral.sh/ruff/[Ruff]. It is up to the individual
 project to choose an enforcement method, but it should be clearly
 documented and continuous integration tests should ensure code is
 correctly styled before merging pull requests.
@@ -21,70 +21,18 @@ example, the maximum line length might vary. The test suite should
 enforce this.
 ====
 
-==== Enforcement
+==== Linting
 
-Projects should automatically enforce code style. How a project does so
+Ruff is capable of running a lot of rules to lint the source code.
-is up to the maintainers, but several good options are documented here.
+The categories of rules to be run can be selecting by
+https://docs.astral.sh/ruff/configuration/[configuring ruff] in the
+`pyproject.toml` file.
 
-===== Tox
+A reasonable set of rules and configuration can be found in the
+Cookiecutter template's
+https://github.com/fedora-infra/cookiecutter-python-app/blob/main/%7B%7B%20cookiecutter.slug%20%7D%7D/pyproject.toml#L77[pyproject.toml]
+file.
 
-`tox-config` is an excellent way to test style. `flake8` looks in, among
-other places, `tox.ini` for its configuration so if you're already using
-tox this is a good place to place your configuration. For example,
-adding the following snippet to your `tox.ini` file will run `flake8` as
-part of your test suite:
-
-....
-[testenv:lint]
-deps =
-    flake8 > 3.0
-commands =
-    python -m flake8 {posargs}
-
-[flake8]
-show-source = True
-max-line-length = 100
-exclude = .git,.tox,dist,*egg
-....
-
-===== Unit Test
-
-If you're not using `tox`, you can add a unit test to your test suite:
-
-....
-"""This module runs flake8 on a subset of the code base"""
-import os
-import subprocess
-import unittest
-
-# Adjust this as necessary to ensure REPO_PATH resolves to the root of your
-# repository.
-REPO_PATH = os.path.abspath(os.path.dirname(os.path.join(os.path.dirname(__file__), '../')))
-
-class TestStyle(unittest.TestCase):
-    """Run flake8 on the repository directory as part of the unit tests."""
-
-    def test_code_with_flake8(self):
-        """Assert the code is PEP8-compliant"""
-
-        # enforced_paths = [
-        #     'mypackage/pep8subpackage/',
-        #     'mypackage/a_module.py',
-        # ]
-        # enforced_paths = [os.path.join(REPO_PATH, p) for p in enforced_paths]
-
-        # If your entire codebase is not already PEP8 compliant, you can enforce
-        # the style incrementally using ``enforced_paths``.
-        #
-        # flake8_command = ['flake8', '--max-line-length', '100'] + enforced_paths
-        flake8_command = ['flake8', '--max-line-length', '100', REPO_PATH]
-
-        self.assertEqual(subprocess.call(flake8_command), 0)
-
-
-if __name__ == '__main__':
-    unittest.main()
-....
 
 ==== Auto formatting
 
@@ -107,18 +55,56 @@ Your text editor is very likely to have a plugin to run Black on file
 saving. The documentation has instructions to set it up in Vim and in VS
 Code (and in Emacs).
 
-You can check that your code is properly formatted according to Black's
+
-settings by adding the following snippet to your `tox.ini` file:
+==== Enforcement
+
+Projects should automatically enforce code style. How a project does so
+is up to the maintainers, but several good options are documented here.
+
+===== Tox
+
+https://tox.wiki[Tox] is an excellent way to test style. For example,
+adding the following snippet to your `tox.ini` file will run `ruff` as
+part of your test suite:
 
 ....
+[testenv:lint]
+deps =
+    ruff
+commands =
+    python -m ruff check {posargs:.}
+
 [testenv:format]
 deps =
     black
 commands =
     python -m black --check {posargs:.}
+
 ....
 
-Remember to add `format` to your Tox `envlist`.
+Remember to add `lint` and `format` to your Tox `envlist`.
+
+
+===== Pre-Commit
+
+https://pre-commit.com/[Pre-commit] is a git hook that will run before Git
+stores a commit. It has the ability to run a set of tools against the files
+that are to be committed, and thus will let the developer know of errors and
+violations early in the development process. Install the `pre-commit` package
+and run `pre-commit install` to setup the hook.
+
+The tools to be run are listed in a `.pre-commit-config.yaml` file at the top
+of the repository. You can find an example in our CookieCutter template's
+https://github.com/fedora-infra/cookiecutter-python-app/blob/main/%7B%7B%20cookiecutter.slug%20%7D%7D/.pre-commit-config.yaml[.pre-commit-config.yaml]
+file.
+
+
+It is also possible and recommended to run the pre-commit checks as part of
+the test suite in tox, see for example the `checks` section in our
+CookieCutter template's
+https://github.com/fedora-infra/cookiecutter-python-app/blob/5ad9adb75600ac13fc65bb9fd847e5091a5d983b/%7B%7B%20cookiecutter.slug%20%7D%7D/tox.ini#L34[tox.ini]
+file.
+
 
 === Javascript
 

modules/developer_guide/pages/documentation.adoc

@@ -108,51 +108,35 @@ not conflict by writing in the same changelog file, and a link to the
 issue, the pull request or the commit is automatically inserted.
 
 In your project root, add a `pyproject.toml` file with a
-`tool.towncrier` section similar to the one in
+`tool.towncrier` section similar to the one in our CookieCutter template's
-https://raw.githubusercontent.com/fedora-infra/fedora-messaging/master/pyproject.toml[fedora-messaging].
+https://github.com/fedora-infra/cookiecutter-python-app/blob/main/%7B%7B%20cookiecutter.slug%20%7D%7D/pyproject.toml[pyproject.toml]
+file.
 
-Create a `news` directory where the news items will be written, and in
-there create a `_template.rst` file with a content similar to the one in
-https://raw.githubusercontent.com/fedora-infra/fedora-messaging/master/news/_template.rst[fedora-messaging].
 
-Of course, replace `fedora_messaging` and the project URL by yours,
+Create a `changelog.d` directory where the changelog items will be written,
-where applicable.
+and inthere create a `_template.md` file with a content similar to the one in
+https://raw.githubusercontent.com/fedora-infra/cookiecutter-python-app/main/%7B%7B%20cookiecutter.slug%20%7D%7D/changelog.d/_template.md[our template].
+
+Of course, replace the first section (the `reference` macro) with the
+project's Github URL. Or just use the CookieCutter template.
 
 Then create a `docs/changelog.rst` file (location configured in the
-`pyproject.toml` file) with the follwing content:
+`pyproject.toml` file) with a content similar to:
 
 ....
-=============
+# Release Notes
-Release Notes
-=============
 
-.. towncrier release notes start
+<!-- towncrier release notes start -->
 ....
 
-Then each commit can add a file in the `news` folder to document the
+Then each commit can add a file in the `changelog.d` folder to document the
-change. The file has the `source.type` name format, where `source` is one
+change. The format is described in our CookieCutter template's
-of:
+https://github.com/fedora-infra/cookiecutter-python-app/blob/main/%7B%7B%20cookiecutter.slug%20%7D%7D/docs/contributing.rst#release-notes[contributing]
-
+document.
-* `42` when the change is described in issue `42`
-* `PR42` when the change has been implemented in pull request `42`, and
-there is no associated issue
-* `Cabcdef` when the change has been implemented in changeset `abcdef`,
-and there is no associated issue or pull request.
-* `username` for contributors (`author` extention). It should be the
-username part of their commits' email address.
-
-And where the `type` part of the filename is:
-
-* `feature`: for new features
-* `bug`: for bug fixes
-* `api`: for API changes
-* `dev`: for development-related changes
-* `author`: for contributor names
-* `other`: for other changes
 
 A preview of the release notes can be generated with
-`towncrier --draft`.
+`towncrier build --draft`.
 
-When running `towncrier`, the tool will write the changelog file and
+When running `towncrier build`, the tool will write the changelog file and
-remove the individual news fragments. These changes can then be
+remove the individual changelog item files. These changes can then be
 committed as part of the release commit.

modules/developer_guide/pages/frameworks.adoc

@@ -33,6 +33,12 @@ with SQLAlchemy. It will configure a scoped session for you, set up a
 declarative base class, and provide a convenient
 `flask_sqlalchemy.BaseQuery` sub-class for you.
 
+===== SQLAlchemy Helpers
+
+https://github.com/fedora-infra/sqlalchemy-helpers/[SQLAlchemy-Helpers] is an
+alternative to using Flask-SQLAlchemy if you need your models to be usable
+from outside Flask.
+
 ==== SQLAlchemy
 
 http://www.sqlalchemy.org/[SQLAlchemy] is an SQL toolkit and Object

modules/developer_guide/pages/security_policy.adoc

@@ -26,6 +26,10 @@ configuration. Any exclusion lines that appear in the codebase MUST be
 sufficiently explained. Code that is only executed during test suite
 runs MAY be exempted from this check.
 
+Note that https://docs.astral.sh/ruff/[Ruff] is also capable of running
+security checks equivalent to Bandit's. Make sure you are selecting the
+`S` category in `pyproject.toml`.
+
 === Authentication
 
 The application MUST use OpenID Connect to authenticate users. The

modules/developer_guide/pages/writing-tests.adoc

@@ -85,61 +85,13 @@ interpreters. It also allows you to define arbitrary test environments,
 so it's an excellent place to run the code style tests and to ensure the
 project's documentation builds without errors or warnings.
 
-Here's an example `tox.ini` file that runs a project's unit tests in
+You can find an example in our CookieCutter template's
-Python 2.7, Python 3.4, Python 3.5, and Python 3.6. It also runs
+https://github.com/fedora-infra/cookiecutter-python-app/blob/main/%7B%7B%20cookiecutter.slug%20%7D%7D/tox.ini[tox.ini].
-https://pypi.python.org/pypi/flake8[flake8] on the entire codebase and
+It runs the test suite in various versions of Python, reports the coverage,
-builds the documentation with the `warnings treated as errors` Sphinx
+runs the pre-commit checks, makes sure we're not depending on software with
-flag enabled. Finally, it enforces 100% coverage on lines edited by new
+forbidden licenses, and builds the documentation with the `warnings treated
-patches using https://pypi.org/project/diff-cover/[diff-cover]:
+as errors` Sphinx flag enabled.
 
-....
-[tox]
-envlist = py27,py34,py35,py36,lint,diff-cover,docs
-# If the user is missing an interpreter, don't fail
-skip_missing_interpreters = True
-
-[testenv]
-deps =
-    -rtest-requirements.txt
-# Substitute your test runner of choice
-commands =
-    py.test
-# When running in OpenShift you don't have a username, so expanduser
-# won't work. If you are running your tests in CentOS CI, this line is
-# important so the tests can pass there, otherwise tox will fail to find
-# a home directory when looking for configuration files.
-passenv = HOME
-
-[testenv:diff-cover]
-deps =
-    diff-cover
-commands =
-    diff-cover coverage.xml --compare-branch=origin/master --fail-under=100
-
-[testenv:docs]
-changedir = docs
-deps =
-    sphinx
-    sphinxcontrib-httpdomain
-    -rrequirements.txt
-whitelist_externals =
-    mkdir
-    sphinx-build
-commands=
-    mkdir -p _static
-    sphinx-build -W -b html -d {envtmpdir}/doctrees .  _build/html
-
-[testenv:lint]
-deps =
-    flake8 > 3.0
-commands =
-    python -m flake8 {posargs}
-
-[flake8]
-show-source = True
-max-line-length = 100
-exclude = .git,.tox,dist,*egg
-....
 
 === Coverage
 
@@ -147,53 +99,39 @@ https://pypi.python.org/pypi/coverage/[coverage] is a good way to
 collect test coverage statistics.
 http://docs.pytest.org/en/latest/contents.html[pytest] has a
 https://pypi.python.org/pypi/pytest-cov[pytest-cov] plugin that
-integrates with https://pypi.python.org/pypi/coverage/[coverage] and
+integrates with https://pypi.python.org/pypi/coverage/[coverage].
-https://pypi.python.org/pypi/nose-cov[nose-cov] provides integration for
+https://pypi.org/project/diff-cover/[Diff-Cover] can be used to ensure
-the https://nose.readthedocs.io/en/latest/[nose] test runner.
-https://pypi.org/project/diff-cover/[diff-cover] can be used to ensure
 that all lines edited in a patch have coverage.
 
 It's possible (and recommended) to have the test suite fail if the
-coverage percentage goes down. This example `.coveragerc`:
+coverage is below 100%. See the template's
+https://github.com/fedora-infra/cookiecutter-python-app/blob/main/%7B%7B%20cookiecutter.slug%20%7D%7D/pyproject.toml#L98[pyproject.toml]
+file for an example.
+
+An alternative to having 100% coverage is to forbid new changes from bringing
+the coverage down. This can be checked by `diff-cover` with the following
+`tox.ini` snippet:
 
 ....
-[run]
+[tox]
-# Track what conditional branches are covered.
+envlist = pyXX...,diff-cover,...
-branch = True
+# If the user is missing an interpreter, don't fail
-include =
+skip_missing_interpreters = True
-    my_python_package/*
 
-[report]
+[testenv:diff-cover]
-# Fail if the coverage is not 100%
+deps =
-fail_under = 100
+    diff-cover
-# Display results with up 1/100th of a percent accuracy.
+commands =
-precision = 2
+    diff-cover coverage.xml --compare-branch=origin/develop --fail-under=100
-exclude_lines =
-    pragma: no cover
-
-    # Don't complain if tests don't hit defensive assertion code
-    raise AssertionError
-    raise NotImplementedError
-
-    if __name__ == .__main__.:
-omit =
-    my_python_package/tests/*
 ....
 
-To configure `pytest` to collect coverage data on your project, edit
+You must then make sure that the `pytest` command line contains
-`setup.cfg` and add this block, substituting `yourpackage` with the name
+`--cov-report xml` to produce the `coverage.xml` file that `diff-cover` will
-of the Python package you are measuring coverage on:
+analyze.
 
-....
+New projects should enforce 100% test coverage. Existing projects should
-[tool:pytest]
+ensure test coverage does not drop to accept a pull request and should
-addopts = --cov-config .coveragerc --cov=yourpackage --cov-report term --cov-report xml --cov-report html
+increase the minimum test coverage until it is 100%.
-....
-
-This causes coverage (and any test running plugins using coverage) to fail if
-the coverage level is not 100%. New projects should enforce 100% test
-coverage. Existing projects should ensure test coverage does not drop to
-accept a pull request and should increase the minimum test coverage
-until it is 100%.
 
 [NOTE]
 ====
@@ -214,46 +152,20 @@ dependencies are checked recursively.
 
 The licenses are validated against a set of acceptable licenses that you
 define in a file called `.license_strategy.ini` in your project
-directory. Here is an example of such a file, that would accept Free
+directory. To avoid having to maintain and sync the same file in every
-licenses:
+project, a shared file is available
+https://raw.githubusercontent.com/fedora-infra/shared/main/liccheck-strategy.ini[in the shared repo].
 
-....
+The CookieCutter template contains
-[Licenses]
+https://github.com/fedora-infra/cookiecutter-python-app/blob/main/%7B%7B%20cookiecutter.slug%20%7D%7D/devel/run-liccheck.sh[a shell script]
-authorized_licenses:
+that will download the shared file and check the dependencies' licenses
-        bsd
+against it.
-        new bsd
-        simplified bsd
-        apache
-        apache 2.0
-        apache software
-        gnu lgpl
-        gpl v2
-        gpl v3
-        lgpl with exceptions or zpl
-        isc
-        isc license (iscl)
-        mit
-        python software foundation
-        zpl 2.1
-....
 
-The verification is case-insensitive, and is done on both the `license`
+If you want to add a license to the list, feel free to open a Pull Request
-and the `classifiers` metadata fields. See
+against https://github.com/fedora-infra/shared[the shared repo].
-https://pypi.org/project/liccheck/[liccheck]'s documentation for more
-details.
 
-You can automate the license check with the following snippet in your
+You can automate the license check by running the shell script in Tox.
-`tox.ini` file:
 
-....
-[testenv:licenses]
-deps =
-    liccheck
-commands =
-    liccheck -s .license_strategy.ini
-....
-
-Remember to add `licenses` to your Tox `envlist`.
 
 === Security
 
@@ -271,3 +183,7 @@ commands =
 ....
 
 Remember to add `bandit` to your Tox `envlist`.
+
+Note that https://docs.astral.sh/ruff/[Ruff] is also capable of running
+security checks equivalent to Bandit's. Make sure you are selecting the
+`S` category in `pyproject.toml`.