MBS is frozen #9653

New issue

Closed

opened 2021-02-12 11:31:05 +00:00 by ppisar · 11 comments

ppisar commented

2021-02-12 11:31:05 +00:00

MBS task stopped advancing. Example:

https://mbs.fedoraproject.org/module-build-service/2/module-builds/11322 is frozen in "init" phase for 25 minutes.
https://mbs.fedoraproject.org/module-build-service/2/module-builds/11302 is in "wait" state for 4 hours without attempting to build any package.
There are plenty of task like that.

I tried canceling the task, pushing a commit and submitting it again to recover from a possible MBS restart. No success. I think MBS got stuck.

MBS task stopped advancing. Example: https://mbs.fedoraproject.org/module-build-service/2/module-builds/11322 is frozen in "init" phase for 25 minutes. https://mbs.fedoraproject.org/module-build-service/2/module-builds/11302 is in "wait" state for 4 hours without attempting to build any package. There are plenty of task like that. I tried canceling the task, pushing a commit and submitting it again to recover from a possible MBS restart. No success. I think MBS got stuck.

smooge commented

2021-02-12 15:11:34 +00:00

I do not have any background on this service but have tried a kick in the pants. I doubt it fixed whatever the real issue is.

smooge commented

2021-02-12 15:11:34 +00:00

Metadata Update from @smooge:

Issue priority set to: Waiting on Assignee (was: Needs Review)
Issue tagged with: high-trouble, mbs, medium-gain, ops

**Metadata Update from @smooge**: - Issue priority set to: Waiting on Assignee (was: Needs Review) - Issue tagged with: high-trouble, mbs, medium-gain, ops

ppisar commented

2021-02-15 09:19:23 +00:00

Author

It helped. The build successfully finished.

ppisar commented

2021-02-15 09:19:24 +00:00

Author

Metadata Update from @ppisar:

Issue close_status updated to: Fixed
Issue status updated to: Closed (was: Open)

**Metadata Update from @ppisar**: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)

ppisar commented

2021-02-15 11:12:33 +00:00

Author

Metadata Update from @ppisar:

Issue status updated to: Open (was: Closed)

**Metadata Update from @ppisar**: - Issue status updated to: Open (was: Closed)

ppisar commented

2021-02-15 11:16:56 +00:00

Author

I built another 3 modules successfully and then I got another MBS failure. E.g. https://mbs.fedoraproject.org/module-build-service/2/module-builds/11347 reports:

$ fedpkg module-build-watch $(seq 11347 11352)
[Build #11347] perl-App-cpanminus-1.7044-3420210211143957-3a2830b3 is in "failed" state.
  Koji tag: module-perl-App-cpanminus-1.7044-3420210211143957-3a2830b3
  Link: https://mbs.fedoraproject.org/module-build-service/2/module-builds/11347
  Components: 0 done, 0 failed
  Reason: [Errno 12] Cannot allocate memory

Or https://mbs.fedoraproject.org/module-build-service/2/module-builds/11360:

[Build #11360] perl-FCGI-0.79-3520210211145428-e12b6f3a is in "failed" state.
  Koji tag: None
  Link: https://mbs.fedoraproject.org/module-build-service/2/module-builds/11360
  Components: 0 done, 0 failed
  Reason: Failed to get the latest commit for git+https://src.fedoraproject.org/rpms/perl-FCGI#f34

I think that MBS experiences a memory leak. Please restart it.

I built another 3 modules successfully and then I got another MBS failure. E.g. https://mbs.fedoraproject.org/module-build-service/2/module-builds/11347 reports: ~~~~ $ fedpkg module-build-watch $(seq 11347 11352) [Build #11347] perl-App-cpanminus-1.7044-3420210211143957-3a2830b3 is in "failed" state. Koji tag: module-perl-App-cpanminus-1.7044-3420210211143957-3a2830b3 Link: https://mbs.fedoraproject.org/module-build-service/2/module-builds/11347 Components: 0 done, 0 failed Reason: [Errno 12] Cannot allocate memory ~~~~ Or https://mbs.fedoraproject.org/module-build-service/2/module-builds/11360: ~~~~ [Build #11360] perl-FCGI-0.79-3520210211145428-e12b6f3a is in "failed" state. Koji tag: None Link: https://mbs.fedoraproject.org/module-build-service/2/module-builds/11360 Components: 0 done, 0 failed Reason: Failed to get the latest commit for git+https://src.fedoraproject.org/rpms/perl-FCGI#f34 ~~~~ I think that MBS experiences a memory leak. Please restart it.