= datanommer SOP

Consume fedmsg bus activity and stuff it in a postgresql db.

== Contact Information

Owner::
  Messaging SIG, Fedora Infrastructure Team
Contact::
  #fedora-apps, #fedora-fedmsg, #fedora-admin, #fedora-noc
Servers::
  busgateway01
Purpose::
  Save fedmsg bus activity

== Description

datanommer is a set of three modules:

python-datanommer-models::
  Schema definition and API for storing new items and querying existing
  items
python-datanommer-consumer::
  A plugin for the fedmsg-hub that actively listens to the bus and
  stores events.
datanommer-commands::
  A set of CLI tools for querying the DB.

datanommer will one day serve as a backend for future web services like
datagrepper and dataviewer.

Source: https://github.com/fedora-infra/datanommer/

Plan: https://fedoraproject.org/wiki/User:Ianweller/statistics_plus_plus

== CLI tools

Dump the db into a file as json:

....
$ datanommer-dump > datanommer-dump.json
....

When was the last bodhi message?:

....
$ # It was 678 seconds ago
$ datanommer-latest --category bodhi --timesince
[678]
....

When was the last bodhi message in more readable terms?:

....
$ # It was 12 minutes and 43 seconds ago
$ datanommer-latest --category bodhi --timesince --human
[0:12:43.087949]
....

What was that last bodhi message?:

....
$ datanommer-latest --category bodhi
[{"bodhi": {
  "topic": "org.fedoraproject.stg.bodhi.update.comment", 
  "msg": {
    "comment": {
      "group": null, 
      "author": "ralph", 
      "text": "Testing for latest datanommer.", 
      "karma": 0, 
      "anonymous": false, 
      "timestamp": 1360349639.0, 
      "update_title": "xmonad-0.10-10.fc17"
    }, 
    "agent": "ralph"
  }, 
}}]
....

Show me stats on datanommer messages by topic:

....
$ datanommer-stats --topic
org.fedoraproject.stg.fas.group.member.remove has 10 entries
org.fedoraproject.stg.logger.log has 76 entries
org.fedoraproject.stg.bodhi.update.comment has 5 entries
org.fedoraproject.stg.busmon.colorized-messages has 10 entries
org.fedoraproject.stg.fas.user.update has 10 entries
org.fedoraproject.stg.wiki.article.edit has 106 entries
org.fedoraproject.stg.fas.user.create has 3 entries
org.fedoraproject.stg.bodhitest.testing has 4 entries
org.fedoraproject.stg.fedoratagger.tag.create has 9 entries
org.fedoraproject.stg.fedoratagger.user.rank.update has 5 entries
org.fedoraproject.stg.wiki.upload.complete has 1 entries
org.fedoraproject.stg.fas.group.member.sponsor has 6 entries
org.fedoraproject.stg.fedoratagger.tag.update has 1 entries
org.fedoraproject.stg.fas.group.member.apply has 17 entries
org.fedoraproject.stg.__main__.testing has 1 entries
....

== Upgrading the DB Schema

datanommer uses "python-alembic" to manage its schema. When developers
want to add new columns or features, these should/must be tracked in
alembic and shipped with the RPM.

In order to run upgrades on our stg/prod dbs:

[arabic]
. ssh to busgateway01\{.stg}
. `cd /usr/share/datanommer.models/`
. Run:
+
....
$ alembic upgrade +1
....

____
Over and over again until the db is fully upgraded.
____