These gnuplot logs are needing to be updated to cover the newer
dates. This will probably be the last set of updates as newer graphs
should be in the pipeline.
Signed-off-by: Stephen Smoogen <ssmoogen@redhat.com>
This removes osbs and allmost all it's associated playbooks and files.
It served long and well, but we no longer need it.
flatpaks are building with a koji-flatpak plugin.
base/minimal/toolbox containers are building with kiwi.
We aren't building any other containers right now, and we did they could
be added to kiwi.
This is the end of an era... I look with nostolga on
ansible-ansible-openshift-ansible (a role to setup ansible on a control
host and run it from our ansible).
Good bye osbs!
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
Previously, an expired certificate on log01 made sending messages out of
the scripts fail, which caused a couple days worth of lost logs.
Signed-off-by: Nils Philippsen <nils@redhat.com>
Matthew Miller found that running the awk script over multiple days
caused newer releases than F33 would go up forever. This fix should
zero out all the new variables.
Signed-off-by: Stephen Smoogen <ssmoogen@redhat.com>
When a corrupt date is found in the log file, I have the program
default of 1970-01-01 but because there is a lookup used it needed to
be 1970-Jan-01 which would then get replaced by 1970-01-01.
Signed-off-by: Stephen Smoogen <ssmoogen@redhat.com>
This prevents sending out unnecessary mails when run from the related
cron job:
condense-mirrorlogs.cron
-> condense-mirrorlogs.sh > /dev/null
-> mirrorlist.py
Additionally, report the failing file name in the case of an error.
Signed-off-by: Nils Philippsen <nils@redhat.com>
Previously, the script was very talkative by default. Make the default
to be silent for log levels < WARNING and allow logging (at different
level) to syslog. Additionally, configure the cronjob to log everything
of levels >= INFO to syslog.
Signed-off-by: Nils Philippsen <nils@redhat.com>
This lets users of simple_message_to_bus predefine items which should be
present in all message bodies this way:
export MSGBODY_PRESET="key1=value1 key2=value2"
This doesn't work with spaces in either keys or values, any quotation
will be used verbatim.
Signed-off-by: Nils Philippsen <nils@redhat.com>
Callers of simple_message_to_bus need to set and export MSGTOPIC_PREFIX
explicitly.
This decouples the fedora-messaging-utils and web-data-analysis roles.
Additionally, don't assume /bin/sh is /bin/bash.
Signed-off-by: Nils Philippsen <nils@redhat.com>
This should prevent race conditions of the form that logs are attempted
to be combined while syncing those of individual hosts hasn't finished.
Signed-off-by: Nils Philippsen <nils@redhat.com>
This is to enable running the syncing and combining scripts in
series rather than from independently scheduled cron jobs.
Signed-off-by: Nils Philippsen <nils@redhat.com>
When renaming a file over another which is the same hard link, the
rename is a no-op. This left many temporary files in /var/log/hosts
because a file is attempted to be synced (and thus hard-linked between
dated and undated file names) over a couple of days. The solution to
this is how the `ln` command does it: rename, then unlink the temporary
file.
Signed-off-by: Nils Philippsen <nils@redhat.com>
The previous one synced all hosts serially and ran rsync for each log
file. This reimplements the shell script in Python, with these changes:
- Run rsync on whole directories of log files, with much reduced
overhead.
- Use a pool of five workers which process hosts in parallel.
Additionally, remove download-rdu01.vpn.fedoraproject.org from the list
of synced hosts.
Signed-off-by: Nils Philippsen <nils@redhat.com>
