Infrastructure/ansible
1
0
Fork 0
Code Issues Pull requests 6 Projects Releases Packages Wiki Activity Actions

Switch 'Fedora Archive' category to scan-primary-mirror

Browse source 
Scanning 'Fedora Archive' with the Python based scanner produces
following error for some time now:

/srv/pub/archive/fullfiletimelist-archive has changed since last run. Running umdl for Fedora Archive at Thu Jan 13 12:15:01 UTC 2022
Traceback (most recent call last):
  File "/usr/bin/mm2_update-master-directory-list", line 1122, in <module>
    sys.exit(main())
  File "/usr/bin/mm2_update-master-directory-list", line 1105, in main
    session, config, i['path'], category)
  File "/usr/bin/mm2_update-master-directory-list", line 970, in sync_directories_from_fullfiletimelist
    sync_category_directory(session, config, category, dirname, ctimes[dirname], file_dict, is_repo)
  File "/usr/bin/mm2_update-master-directory-list", line 617, in sync_category_directory
    session, config, relativeDName, D, category, 'repomd.xml')
  File "/usr/lib/python2.7/site-packages/mirrormanager2/lib/umdl.py", line 294, in make_repo_file_details
    yumrepo = rpmmd.repoMDObject.RepoMD('repoid', absolutepath)
  File "/usr/lib/python2.7/site-packages/rpmmd/repoMDObject.py", line 145, in __init__
    self.parse(srcfile)
  File "/usr/lib/python2.7/site-packages/rpmmd/repoMDObject.py", line 165, in parse
    for event, elem in parser:
  File "<string>", line 100, in next
  File "/usr/lib/python2.7/site-packages/rpmmd/misc.py", line 150, in read
    return self.checksums.read(self._fo, size)
  File "/usr/lib/python2.7/site-packages/rpmmd/misc.py", line 104, in read
    self.update(data)
  File "/usr/lib/python2.7/site-packages/rpmmd/misc.py", line 100, in update
    sumalgo.update(data.encode('utf-8'))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 273: ordinal not in range(128)
umdl for Fedora Archive returned non-zero. Something failed. Please check umdl.log. Thu Jan 13 12:17:45 UTC 2022

The new tool, scan-primary-mirror, does not have this problem and
I wanted to switch to it for some time already.

With this the error is gone and the new tool is used.

Signed-off-by: Adrian Reber <adrian@lisas.de>
This commit is contained in:
Adrian Reber 2022-01-13 14:10:56 +01:00
parent 2ea3cebe2b
commit df910b3b23
No known key found for this signature in database
GPG key ID: 82C9378ED3C4906A
2 changed files with 37 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

1
roles/mirrormanager/backend/files/umdl-required
View file

@ -28,6 +28,7 @@ elif [ "${1}" == "fedora-secondary" ]; then
CATEGORY="Fedora Secondary Arches"
elif [ "${1}" == "archive" ]; then
CATEGORY="Fedora Archive"
SCANNER="/usr/local/bin/scan-primary-mirror"
elif [ "${1}" == "codecs" ]; then
CATEGORY="Fedora Codecs"
SCANNER="/usr/local/bin/scan-primary-mirror"

37
roles/mirrormanager/backend/templates/scan-primary-mirror.toml
View file

@ -1,7 +1,7 @@
max_propagation_days = 2
max_stale_days = 3
excludes=[".*\\.snapshot", ".*/\\.~tmp~"]
skip_paths_for_version=["pub/alt"]
skip_paths_for_version=["pub/alt", "pub/archive"]
test_paths=["/test/", "/stage/"]
skip_repository_paths = ["Cloud", "Workstation", "Server", "drpms" ]
do_not_display_paths = ["_Beta"]
@ -94,3 +94,38 @@ url="/srv/codecs.fedoraproject.org/"
regex="codecs.fedoraproject.org/openh264/[\\.\\d]+/.*"
prefix="fedora-cisco-openh264"
### Fedora Archive ###
[[category]]
name="Fedora Archive"
type="directory"
url="/srv/pub/archive/"
[[repository_mapping]]
regex="^pub/archive/fedora(-secondary)?(/linux)?/(core|releases|development)/(rawhide|[\\.\\d]+)/.*"
prefix="fedora"
[[repository_mapping]]
regex="^pub/archive/fedora(-secondary)?(/linux)?/updates/[\\.\\d]+/Everything/.*"
prefix="updates-released"
[[repository_mapping]]
regex="^pub/archive/fedora(-secondary)?(/linux)?/updates/[\\.\\d]+/Modular/.*"
prefix="updates-released-modular"
[[repository_mapping]]
regex="^pub/archive/fedora(-secondary)?(/linux)?/updates/testing/[\\.\\d]+/Everything/.*"
prefix="updates-testing"
[[repository_mapping]]
regex="^pub/archive/fedora(-secondary)?(/linux)?/updates/testing/[\\.\\d]+/Modular/.*"
prefix="updates-testing-modular"
# pre modular structure
[[repository_mapping]]
regex="^pub/archive/fedora(-secondary)?(/linux)?/updates/testing/([\\.\\d]+)/[^ME].*"
prefix="updates-testing"
[[repository_mapping]]
regex="^pub/archive/fedora(-secondary)?(/linux)?/updates/([\\.\\d]+)/[^ME].*"
prefix="updates-released"