Add and use optimized http log syncing script
The previous one synced all hosts serially and ran rsync for each log file. This reimplements the shell script in Python, with these changes: - Run rsync on whole directories of log files, with much reduced overhead. - Use a pool of five workers which process hosts in parallel. Additionally, remove download-rdu01.vpn.fedoraproject.org from the list of synced hosts. Signed-off-by: Nils Philippsen <nils@redhat.com>
This commit is contained in:
parent
ea1c19d522
commit
f703e7a771
4 changed files with 282 additions and 140 deletions
|
@ -151,9 +151,30 @@
|
|||
- web-data
|
||||
- cron
|
||||
|
||||
- name: install a sync httpd logs cron script only on log01
|
||||
copy: src=syncHttpLogs.sh dest=/etc/cron.daily/syncHttpLogs.sh mode=0755
|
||||
- name: remove old syncHttpLogs.sh cron script only on log01
|
||||
file:
|
||||
path: /etc/cron.daily/syncHttpLogs.sh
|
||||
state: absent
|
||||
when: inventory_hostname.startswith('log01')
|
||||
tags:
|
||||
- web-data
|
||||
- cron
|
||||
|
||||
- name: write configuration file for script to sync httpd logs
|
||||
template:
|
||||
src: sync-http-logs.yaml.j2
|
||||
dest: /etc/sync-http-logs.yaml
|
||||
when: inventory_hostname.startswith('log01')
|
||||
tags:
|
||||
- web-data
|
||||
- config
|
||||
|
||||
- name: install a script to sync httpd logs via cron only on log01
|
||||
copy:
|
||||
src: sync-http-logs.py
|
||||
dest: /etc/cron.daily/sync-http-logs.py
|
||||
mode: 0755
|
||||
when: inventory_hostname.startswith('log01')
|
||||
tags:
|
||||
- web-data
|
||||
- cron
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue