Pavel Raiskup
f4538051df
s3-mirror: sync s3-sync-path script with ideas from s3.sh
...
1. sync everything except for repomd.xml
2. then sync repomd.xml files only, and invalidate caches
3. gently wait a bit to give current downloads a chance
4. delete outdated RPMs and metadata, shouldn't be needed
Also make the sleep/cache configurable.
2020-04-24 21:34:28 +02:00
Kevin Fenzi
96190cec94
Revert "Revert "s3-mirror: Split things into 2 sync runs, one without repodata and delete, the other with both.""
...
This reverts commit 0798930ebea639871194525789061161b2c19bf9.
Got some +1s, so pushing this now.
2020-04-24 21:34:26 +02:00
Kevin Fenzi
76fb2e0871
Revert "s3-mirror: Split things into 2 sync runs, one without repodata and delete, the other with both."
...
This reverts commit 626c976f8ce616750a56cd0de7847beeaf3924f7.
2020-04-24 21:34:26 +02:00
Kevin Fenzi
899d1767eb
s3-mirror: Split things into 2 sync runs, one without repodata and delete, the other with both.
...
In order to make sure the s3 mirror always is consistent, split out the commands
to make it sync without repodata and delete, then do another run with those, then finally
invalidate all the repodata/* files.
Some of the cron jobs are adjusted to allow the repodata invalidation.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2020-04-24 21:34:26 +02:00
Rick Elrod
50b7f1d22c
s3-mirror: Stop echoing the command we are running
...
Signed-off-by: Rick Elrod <relrod@redhat.com>
2020-04-24 21:34:11 +02:00
Rick Elrod
139af32030
s3-mirror: No reason not to sync releases/test
...
Signed-off-by: Rick Elrod <relrod@redhat.com>
2020-04-24 21:34:10 +02:00
Rick Elrod
97adecb9b4
s3-mirror: stop excluding development/*, just exclude development/rawhide
...
Signed-off-by: Rick Elrod <relrod@redhat.com>
2020-04-24 21:34:08 +02:00
Rick Elrod
3f1d2cc9f7
s3-mirror: remove them here too
...
Signed-off-by: Rick Elrod <relrod@redhat.com>
2020-04-24 21:34:07 +02:00
Rick Elrod
ca768c3000
send invalidation stdout to /dev/null for now to shut up cron
...
Signed-off-by: Rick Elrod <relrod@redhat.com>
2019-05-31 15:28:10 +00:00
Rick Elrod
ca07bf43c8
s3-mirror: try adding --only-show-errors to the sync part
...
Signed-off-by: Rick Elrod <relrod@redhat.com>
2019-05-30 15:51:06 +00:00
Rick Elrod
580d5c2347
ok, awscli, have it your way
...
Signed-off-by: Rick Elrod <relrod@redhat.com>
2019-05-29 22:28:22 +00:00
Rick Elrod
50279dac19
add a more general s3 sync script that takes a path from /pub/ and attempts to sync it
...
Signed-off-by: Rick Elrod <relrod@redhat.com>
2019-05-29 19:03:44 +00:00
