1751808 - Intermittent iscript.exceptions.IScriptError: ('%s in %s exited %s!\n%s', ['productsign', '--keychain', '/builds/scriptworker/certs/nightly-signing.keychain', '--sign', '43AQ936H96', '/builds/scriptworker/work/1/Firefox Nightly.tmp2.pkg', '/builds/scriptw

Status: RESOLVED FIXED

(Release Engineering :: Release Automation, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: mlaza [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=365297263&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/bbTT8arkSNuJpPKNjZv79g/runs/0/artifacts/public/logs/live_backing.log

---

2022-01-25 00:26:48,884 - scriptworker_client.utils - INFO - productsign: adding certificate "Apple Root CA"
2022-01-25 00:26:48,884 - scriptworker_client.utils - INFO - productsign: Wrote signed product archive to /builds/scriptworker/work/4/Firefox Nightly.pkg
2022-01-25 00:26:48,886 - scriptworker_client.utils - INFO - ['productsign', '--keychain', '/builds/scriptworker/certs/nightly-signing.keychain', '--sign', '43AQ936H96', '/builds/scriptworker/work/4/Firefox Nightly.tmp2.pkg', '/builds/scriptworker/work/4/Firefox Nightly.pkg'] in /builds/scriptworker/work/4 exited 0
2022-01-25 00:26:48,886 - scriptworker_client.client - ERROR - Failed to run async_main; exiting 1
Traceback (most recent call last):
  File "/builds/scriptworker/virtualenv/lib/python3.8/site-packages/scriptworker_client-1.0.0-py3.8.egg/scriptworker_client/client.py", line 182, in _handle_asyncio_loop
    await async_main(config, task)
  File "/builds/scriptworker/virtualenv/lib/python3.8/site-packages/iscript-1.0.1-py3.8.egg/iscript/script.py", line 39, in async_main
    await notarize_1_behavior(config, task)
  File "/builds/scriptworker/virtualenv/lib/python3.8/site-packages/iscript-1.0.1-py3.8.egg/iscript/mac.py", line 1278, in notarize_1_behavior
    await create_pkg_files(config, sign_config, all_paths, requirements_plist_path=requirements_plist_path)
  File "/builds/scriptworker/virtualenv/lib/python3.8/site-packages/iscript-1.0.1-py3.8.egg/iscript/mac.py", line 1067, in create_pkg_files
    await raise_future_exceptions(futures)
  File "/builds/scriptworker/virtualenv/lib/python3.8/site-packages/scriptworker_client-1.0.0-py3.8.egg/scriptworker_client/aio.py", line 65, in raise_future_exceptions
    raise exceptions[0]
  File "/builds/scriptworker/virtualenv/lib/python3.8/site-packages/scriptworker_client-1.0.0-py3.8.egg/scriptworker_client/aio.py", line 91, in semaphore_wrapper
    return await coro
  File "/builds/scriptworker/virtualenv/lib/python3.8/site-packages/scriptworker_client-1.0.0-py3.8.egg/scriptworker_client/utils.py", line 259, in run_command
    raise exception(
iscript.exceptions.IScriptError: ('%s in %s exited %s!\n%s', ['productsign', '--keychain', '/builds/scriptworker/certs/nightly-signing.keychain', '--sign', '43AQ936H96', '/builds/scriptworker/work/1/Firefox Nightly.tmp2.pkg', '/builds/scriptworker/work/1/Firefox Nightly.pkg'], '/builds/scriptworker/work/1', -11, '')
exit code: 1

Comment 1

[Aki Sasaki (not active)]

We probably need to wrap the productsign call in a retry_async since it looks like we may be hitting an external network resource (timestamp server).

Comment 2

[Intermittent Failures Robot]

1 failures in 3445 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform and build breakdown:

• osx-shippable: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1751808&startday=2022-01-24&endday=2022-01-30&tree=all

Comment 3

[Heitor Neiva [:hneiva]]

Added a retry in: https://github.com/mozilla-releng/scriptworker-scripts/pull/476

Comment 4

[Julien Cristau [:jcristau]]

Attachment: GitHub Pull Request