Infrastructure/fedora-infrastructure

Fork 0

Failure when attempted to build cheese flatpak from module #7846

New issue

Closed

opened 2019-05-28 15:04:37 +00:00 by amigadave · 11 comments

amigadave commented

2019-05-28 15:04:37 +00:00

I have built cheese as a module in koji:

https://koji.fedoraproject.org/koji/buildinfo?buildID=1271291
https://koji.fedoraproject.org/koji/buildinfo?buildID=1271297

When trying to build that module into a flatpak in koji (with fedpkg flatpak-build, which succeeds locally) there is a build failure:

https://koji.fedoraproject.org/koji/taskinfo?taskID=35111655

I asked Patrick Uiterwijk on #fedora-admin and he said that the cause is an upload failure, but to file a ticket to track the root cause.

I have built cheese as a module in koji: https://koji.fedoraproject.org/koji/buildinfo?buildID=1271291 https://koji.fedoraproject.org/koji/buildinfo?buildID=1271297 When trying to build that module into a flatpak in koji (with `fedpkg flatpak-build`, which succeeds locally) there is a build failure: https://koji.fedoraproject.org/koji/taskinfo?taskID=35111655 I asked Patrick Uiterwijk on #fedora-admin and he said that the cause is an upload failure, but to file a ticket to track the root cause.

kevin commented

2019-05-28 23:23:23 +00:00

Is this a constant issue? or if you retry it works?

kevin commented

2019-05-28 23:23:24 +00:00

Metadata Update from @kevin:

Issue priority set to: Waiting on Reporter (was: Needs Review)

**Metadata Update from @kevin**: - Issue priority set to: Waiting on Reporter (was: Needs Review)

kalev commented

2019-05-29 07:20:03 +00:00

All flatpak builds I've tried (since https://pagure.io/fedora-infrastructure/issue/7833 got fixed) have run into the same issue with log uploading, e.g.:

$ fedpkg flatpak-build
Created task: 35121309
Task info: https://koji.fedoraproject.org/koji/taskinfo?taskID=35121309
Watching tasks (this may be safely interrupted)...
35121309 buildContainer (noarch): open (buildvm-07.phx2.fedoraproject.org)
35121309 buildContainer (noarch): open (buildvm-07.phx2.fedoraproject.org) -> FAILED: Fault: <Fault 2001: "Image build failed. Error in plugin koji_import: GenericError('File upload failed: koji-upload/1559113182.4967813.rUJYQGJl/orchestrator.log'). OSBS build id: flatpak-runtime-f30-be1bb-11">
  0 free  0 open  0 done  1 failed

35121309 buildContainer (noarch) failed

All flatpak builds I've tried (since https://pagure.io/fedora-infrastructure/issue/7833 got fixed) have run into the same issue with log uploading, e.g.: ``` $ fedpkg flatpak-build Created task: 35121309 Task info: https://koji.fedoraproject.org/koji/taskinfo?taskID=35121309 Watching tasks (this may be safely interrupted)... 35121309 buildContainer (noarch): open (buildvm-07.phx2.fedoraproject.org) 35121309 buildContainer (noarch): open (buildvm-07.phx2.fedoraproject.org) -> FAILED: Fault: <Fault 2001: "Image build failed. Error in plugin koji_import: GenericError('File upload failed: koji-upload/1559113182.4967813.rUJYQGJl/orchestrator.log'). OSBS build id: flatpak-runtime-f30-be1bb-11"> 0 free 0 open 0 done 1 failed 35121309 buildContainer (noarch) failed ```

amigadave commented

2019-05-29 11:54:34 +00:00

Author

Is this a constant issue? or if you retry it works?

Constant for me, and the same issue that @kalev is seeing, it seems.

> Is this a constant issue? or if you retry it works? Constant for me, and the same issue that @kalev is seeing, it seems.

mizdebsk commented

2019-05-29 11:55:43 +00:00

Metadata Update from @mizdebsk:

Issue priority set to: Waiting on Assignee (was: Waiting on Reporter)
Issue tagged with: OSBS

**Metadata Update from @mizdebsk**: - Issue priority set to: Waiting on Assignee (was: Waiting on Reporter) - Issue tagged with: OSBS

cverna commented

2019-05-29 12:20:47 +00:00

So on OSBS receives a exception from Koji when trying to upload the log files, the file upload is successful but we still get the exception from Koji.

@mizdebsk pointed out #7838 but as far as I can tell OSBS is already fast upload.

2019-05-28 14:16:17,657 platform:- - atomic_reactor.plugin - DEBUG - Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/atomic_reactor/plugin.py", line 239, in run
    plugin_response = plugin_instance.run()
  File "/usr/lib/python3.7/site-packages/atomic_reactor/plugins/exit_koji_import.py", line 471, in run
    self.upload_file(self.session, output, server_dir)
  File "/usr/lib/python3.7/site-packages/atomic_reactor/plugins/exit_koji_import.py", line 448, in upload_file
    callback=upload_logger.callback, **kwargs)
  File "/usr/lib/python3.7/site-packages/atomic_reactor/koji_util.py", line 64, in call_with_catch
    return session_attr(*a, **kw)
  File "/usr/lib/python3.7/site-packages/koji/__init__.py", line 2760, in uploadWrapper
    self.fastUpload(localfile, path, name, callback, blocksize, overwrite, volume=volume)
  File "/usr/lib/python3.7/site-packages/koji/__init__.py", line 2698, in fastUpload
    raise GenericError("File upload failed: %s/%s" % (path, name))
koji.GenericError: File upload failed: koji-upload/1559052594.307929.ggNepdsC/x86_64.log
2019-05-28 14:16:17,657 platform:- - atomic_reactor.plugin - WARNING - plugin 'koji_import' raised an exception: GenericError('File upload failed: koji-upload/1559052594.307929.ggNepdsC/x86_64.log')

So on OSBS receives a exception from Koji when trying to upload the log files, the file upload is successful but we still get the exception from Koji. @mizdebsk pointed out #7838 but as far as I can tell OSBS is already fast upload. ``` 2019-05-28 14:16:17,657 platform:- - atomic_reactor.plugin - DEBUG - Traceback (most recent call last): File "/usr/lib/python3.7/site-packages/atomic_reactor/plugin.py", line 239, in run plugin_response = plugin_instance.run() File "/usr/lib/python3.7/site-packages/atomic_reactor/plugins/exit_koji_import.py", line 471, in run self.upload_file(self.session, output, server_dir) File "/usr/lib/python3.7/site-packages/atomic_reactor/plugins/exit_koji_import.py", line 448, in upload_file callback=upload_logger.callback, **kwargs) File "/usr/lib/python3.7/site-packages/atomic_reactor/koji_util.py", line 64, in call_with_catch return session_attr(*a, **kw) File "/usr/lib/python3.7/site-packages/koji/__init__.py", line 2760, in uploadWrapper self.fastUpload(localfile, path, name, callback, blocksize, overwrite, volume=volume) File "/usr/lib/python3.7/site-packages/koji/__init__.py", line 2698, in fastUpload raise GenericError("File upload failed: %s/%s" % (path, name)) koji.GenericError: File upload failed: koji-upload/1559052594.307929.ggNepdsC/x86_64.log 2019-05-28 14:16:17,657 platform:- - atomic_reactor.plugin - WARNING - plugin 'koji_import' raised an exception: GenericError('File upload failed: koji-upload/1559052594.307929.ggNepdsC/x86_64.log') ```

cverna commented

2019-05-30 14:32:23 +00:00

The expection is raise if the kojihub checkUpload method does not return anything. Using the following script shows that the file was correctly uploaded.

import koji
c = koji.ClientSession("https://koji.fedoraproject.org/kojihub")
c.checkUpload("koji-upload/1559052594.307929.ggNepdsC/", "x86_64.log")
{'mtime': 1559052977.617066, 'size': 323288}

So I am not sure what is going on here, maybe a race condition.

The expection is raise if the kojihub `checkUpload` method does not return anything. Using the following script shows that the file was correctly uploaded. ``` import koji c = koji.ClientSession("https://koji.fedoraproject.org/kojihub") c.checkUpload("koji-upload/1559052594.307929.ggNepdsC/", "x86_64.log") {'mtime': 1559052977.617066, 'size': 323288} ``` So I am not sure what is going on here, maybe a race condition.

cverna commented

2019-06-03 09:41:33 +00:00

Ok that should be fixed --> https://koji.fedoraproject.org/koji/taskinfo?taskID=35246024

I had to hotfix OSBS code to use fastUpload, I ll submit a patch upstream to allow OSBS to use fastUpload or not.

Ok that should be fixed --> https://koji.fedoraproject.org/koji/taskinfo?taskID=35246024 I had to hotfix OSBS code to use `fastUpload`, I ll submit a patch upstream to allow OSBS to use `fastUpload` or not.

cverna commented

2019-06-03 09:41:45 +00:00

Metadata Update from @cverna: