Closed Bug 1625403 Opened 4 years ago Closed 4 years ago

notarization-poller task got stuck

Categories

(Release Engineering :: Release Automation: Signing, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED DUPLICATE of bug 1620697

People

(Reporter: jcristau, Unassigned)

Details

The shippable-l10n-notarization-poller-macosx64-shippable-11/opt task for firefox 75.0b10 ran for about 6 hours. Johan ended up killing and re-running it, hopefully that helps, but there seems to be something wrong there...

https://firefox-ci-tc.services.mozilla.com/tasks/bRJOUtzXT3KURXn9DDGbSg/runs/0

The problem was worked around for the 75.0b10 release. :sfraser and I both investigated the issue. I tried to connect on right machine[1], but I needed a password. It's located at the usual place, but I didn't have access to it (Simon did). By looking at logs under /builds/scriptworker/poller [2], Simon found:

2020-03-27T09:49:47 INFO - Running ['xcrun', 'altool', '--notarization-info', '2635b110-f444-4937-bece-6067a2d08c8d', '-u', 'release_notarization@mozilla.com', '--password', '********'] in /builds/scriptworker/poller ...
2020-03-27T09:49:47 INFO - 2020-03-27 09:49:47.900 altool[45124:28630842] No errors getting notarization info.
2020-03-27T09:49:47 INFO - 
2020-03-27T09:49:47 INFO -    RequestUUID: 2635b110-f444-4937-bece-6067a2d08c8d
2020-03-27T09:49:47 INFO -           Date: 2020-03-27 02:26:45 +0000
2020-03-27T09:49:47 INFO -         Status: unknown (-3)
2020-03-27T09:49:47 INFO -     LogFileURL: (null)

It turns out Apple "forgot" about a specific UUID. Simon found a workaround called out in Slack[3]: it's a matter of cancelling the polling task and rerunning the upstream submission task. It took us about 15 minutes to get both tasks green.

[1] https://github.com/mozilla-releng/scriptworker-scripts/wiki/machines
[2] https://github.com/mozilla-releng/scriptworker-scripts/wiki/manual-rollout-poller
[3] https://mozilla.slack.com/archives/CFFLZM7L3/p1584918137076100

Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → DUPLICATE
You need to log in before you can comment on or make changes to this bug.