Update DNF Counting SOP for new mirrors-countme

Fixes: https://github.com/fedora-infra/mirrors-countme/issues/84

Signed-off-by: Nils Philippsen <nils@redhat.com>

modules/sysadmin_guide/pages/dnf-counting.adoc

@@ -8,17 +8,15 @@ installations.
 Owner::
   Fedora Infrastructure Team
 Contact::
-  #fedora-admin, #fedora-noc
+  #fedora-admin, #fedora-noc,
-Initiative Representatives::
+  admin@fedoraproject.org
-  * Nils Philippsen (nphilipp)
-  * Adam Saleh (asaleh)
 Servers::
   log01, proxy0*
 Purpose::
   Give interested parties information about the number of Fedora
   installations.
 Repositories::
-  * https://pagure.io/mirrors-countme
+  * https://github.com/fedora-infra/mirrors-countme
   * https://pagure.io/fedora-infra/ansible/blob/main/f/roles/web-data-analysis
 
 == What it is
@@ -51,10 +49,36 @@ log data) and b) shell scripts in the `web-data-analysis` role in Ansible:
   including proxies to `log01`.
 * `combineHttpLogs.sh` (Ansible) combines the logs for the different web sites
   which are scattered across the proxy hosts.
-* `condense-mirrorlogs.sh` & `mirrorlist.py` (Ansible) extract hosts from the
+* `condense-mirrorlogs.sh` & `mirrorlist.py` (Ansible) extracts hosts from the
   combined log data.
 * `countme-update.sh` (Ansible) drives `countme-update-rawdb.sh` &
-  `countme-update-totals.sh` (`mirrors-countme`) which generate statistics.
+  `countme-update-totals.sh` (`mirrors-countme`) which generates statistics.
+* `countme-trim-raw` (`mirrors-countme`) to trim the intermediary database file
+  (``raw.db``).
+
+== Changes implemented in the Q2/2023 DNF mirrors-countme initiative
+
+* The “traditional“ statistics which were done before DNF
+  learned about the `countme` variable were reimplemented: Count any
+  individual IP sighted, no matter if with or without `countme`. This is
+  necessary to count systems which don’t have that feature in their DNF or
+  YUM, and – while giving different numbers – gives us an idea how things
+  develop when compared to the same numbers for more modern OSes.
+* The ``countme-trim-raw`` tool was implemented, to trim the intermediary
+  database ``raw.db`` which contains necessary information gleamed from
+  parsing the merged log files. This database grows steadily and – with the
+  brought back counting of any individual IP sighted – quickly, so once these
+  data have been safely turned into the final statistics, we wanted a way to
+  remove them so that the local volume were it is stored doesn’t fill up
+  completely.
+* The project repository was cleaned up, i.e. large data files used in
+  integration tests were removed because they made cloning the repository
+  unnecessarily slow, for a couple hundred KB of code, the repo was more than
+  300 MB in size. In the context, the repository was moved from Pagure to
+  GitHub.
+* Unused code was removed, the remaining code was refactored and condensed
+  to remove redundancies and comprehensive unit tests were added so that the
+  barrier to contributing is lower and changes are less risky.
 
 == Changes implemented in the Q3/2021 DNF Counting Initiative
 
@@ -137,7 +161,7 @@ scripts and re-running them.
 
 == Where are the docs?
 
-Here :) and at https://pagure.io/mirrors-countme/blob/main/f/README.md
+Here :) and at https://github.com/fedora-infra/mirrors-countme
 
 == Is there data that needs to be backed up?
 
@@ -160,30 +184,31 @@ branch of `mirrors-countme`.
 +
 From a clone of the upstream repository:
 +
-.. In `countme/version.py`, bump `__version__` (e.g. to `0.0.7`) and
+.. In `pyproject.toml`, bump `tool.poetry.version` (e.g. to `0.1.2`) and
 commit the change, e.g.:
 +
 ....
-git commit -s -m "Bump version to 0.0.7" -- countme/version.py
+git commit -s -m "Version 0.1.2" -- pyproject.toml
 ....
 .. Tag the previous change with a GPG-signed tag:
 +
 ....
-git tag -s 0.0.7
+git tag -s 0.1.2
 ....
 .. Push both the change and the tag:
 +
 ....
-git push origin main 0.0.7
+git push origin main 0.1.2
 ....
 .. Create a source tarball (this will be created as e.g.
-`dist/mirrors-countme-0.0.7.tar.gz`):
+`dist/mirrors_countme-0.1.2.tar.gz`):
 +
 ....
-python setup.py sdist
+poetry build
 ....
-.. https://pagure.io/mirrors-countme/upload[Upload this tarball] to the
+From the https://github.com/fedora-infra/mirrors-countme/tags[list of tags],
-`mirrors-countme` releases folder
+select “Create release” in the menu for the respective tag, and attach the
+created tarball and wheel files to the created release.
 . Update and Build the `python-mirrors-countme` Fedora Package
 +
 From a clone of the Fedora package repository, in the `rawhide` branch:
@@ -203,12 +228,12 @@ rpmspectool get python-mirrors-countme.spec
 .. Upload the source tarball to the lookaside cache:
 +
 ....
-fedpkg new-sources mirrors-countme-0.0.7.tar.gz
+fedpkg new-sources mirrors_countme-0.1.2.tar.gz
 ....
 .. Commit the changes to the repository, e.g.:
 +
 ....
-git commit -s -m "Version 0.0.7" -- python-mirrors-countme.spec
+git commit -s -m "Version 0.1.2" -- python-mirrors-countme.spec
 ....
 .. Push the changes and build:
 +
@@ -244,7 +269,7 @@ koji tag-build epel8-infra-candidate
 more than a few minutes), e.g.:
 +
 ....
-koji buildinfo python-mirrors-countme-0.0.7-1.el8
+koji buildinfo python-mirrors-countme-0.1.2-1.el8
 ....
 +
 The build must be tagged with the corresponding `*-infra-stg` tag.
@@ -281,9 +306,10 @@ All of this runs on `log01.iad2.fedoraproject.org` and is deployed through the
 `web-data-analysis` role and the `groups/logserver.yml` playbook,
 respectively.
 
-The `mirrors-countme` upstream project publishes source tarballs here:
+The `mirrors-countme` upstream project publishes source tarballs to their
+corresponding releases in the repository on GitHub:
 
-https://releases.pagure.org/mirrors-countme/
+https://github.com/fedora-infra/mirrors-countme/releases
 
 These are packaged in Fedora as the `python-mirrors-countme` (SRPM) and
 `python3-mirrors-countme` (RPM) packages.
@@ -303,7 +329,7 @@ The same as anything else that deals with log data.
 
 Report bugs with `mirrors-countme` at its upstream project:
 
-https://pagure.io/mirrors-countme/new_issue
+https://github.com/fedora-infra/mirrors-countme/issues/new
 
 Anything concerning the cron jobs or other scripts should probably go into our
 Infrastructure tracker: