Commit graph

2 commits

Author SHA1 Message Date
Nils Philippsen
6e62fcbe69 Don't drop temporary files all over the place
When renaming a file over another which is the same hard link, the
rename is a no-op. This left many temporary files in /var/log/hosts
because a file is attempted to be synced (and thus hard-linked between
dated and undated file names) over a couple of days. The solution to
this is how the `ln` command does it: rename, then unlink the temporary
file.

Signed-off-by: Nils Philippsen <nils@redhat.com>
2021-08-12 09:45:49 +00:00
Nils Philippsen
f703e7a771 Add and use optimized http log syncing script
The previous one synced all hosts serially and ran rsync for each log
file. This reimplements the shell script in Python, with these changes:

- Run rsync on whole directories of log files, with much reduced
  overhead.
- Use a pool of five workers which process hosts in parallel.

Additionally, remove download-rdu01.vpn.fedoraproject.org from the list
of synced hosts.

Signed-off-by: Nils Philippsen <nils@redhat.com>
2021-08-05 16:44:47 +00:00