Closed Bug 1593816 Opened 6 years ago Closed 6 years ago

Perma signingscript.exceptions.SigningServerError: Cannot retrieve signing token from any signing server.

Categories

(Release Engineering :: Release Automation, defect, P1)

Product:
Release Engineering
Component:
Release Automation
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: intermittent-bug-filer, Assigned: mtabara)

References

Details

(Keywords: intermittent-failure)

Filed by: rmaries [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=274473764&repo=mozilla-release
Full log: https://queue.taskcluster.net/v1/task/VQTfw3YDQ66PBpKfO5q4Ew/runs/0/artifacts/public/logs/live_backing.log

2019-11-04 20:40:12,418 - asyncio - DEBUG - Using selector: EpollSelector
2019-11-04 20:40:12,421 - signingscript.utils - INFO - Loading signing server config from /app/configs/passwords.json
2019-11-04 20:40:12,421 - signingscript.utils - INFO - Signing server config loaded from /app/configs/passwords.json
2019-11-04 20:40:12,421 - signingscript.script - INFO - getting signingserver token
2019-11-04 20:40:12,422 - scriptworker.client - ERROR - Failed to run async_main
Traceback (most recent call last):
File "/app/lib/python3.7/site-packages/scriptworker/client.py", line 205, in _handle_asyncio_loop
await async_main(context)
File "/app/src/signingscript/script.py", line 68, in async_main
all_signing_formats,
File "/app/src/signingscript/task.py", line 191, in get_token
"Cannot retrieve signing token from any signing server."
signingscript.exceptions.SigningServerError: Cannot retrieve signing token from any signing server.
exit code: 5

Assignee: nobody → mtabara
Regressed by: 1542819

I've uplifted the GCP patches (all of the workers) from beta to release in the light of upcoming TCW 9th of Nov.
For some reason, the {win32,64} on-push signing jobs are failing with the error from comment 0.

Tom suggested we might need to adjust the format.

Bumping priority as this is CI on mozilla-release and it's also blocking the migration.
If we can't find the fix in good time, we could also backout the patch.

Priority: P5 → P1
No longer regressed by: 1542819
See Also: → 1542819

Turns out I forgot to also uplift patches from bug 1540276. Thanks @aki and @tomprince for letting me know.
Grafting those now too to fix the mozilla-release

The follow-up uplifts fixed the broken signing jobs: https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=2d7eef3ae040515738746128dcbf718b8cfee1bb

Feel free to reopen this if I missed anything.

Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Component: Release Automation: Signing → Release Automation
You need to log in before you can comment on or make changes to this bug.