1689486 - Intermittent addonscript.exceptions.SignatureError: Expected 1 file. Got (0) full response: {...}

Status: RESOLVED INACTIVE

(Release Engineering :: Release Automation, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: malexandru [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=328124252&repo=mozilla-beta
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/YPVDiPXzSqW1jrpkySz0rw/runs/0/artifacts/public/logs/live_backing.log

---

2021-01-28 21:42:09,174 - scriptworker.utils - DEBUG - retry_async: get_signed_addon_url: sleeping 120 seconds before retry
2021-01-28 21:42:15,604 - addonscript.utils - DEBUG - Calling amo_get() with URL "https://addons.mozilla.org/api/v4/addons/[e-mail]/versions/86.0buildid20210128185743/uploads/907b71ccadda45efaa59e291630063af/"
2021-01-28 21:42:15,787 - addonscript.utils - DEBUG - amo_get() for URL "https://addons.mozilla.org/api/v4/addons/[e-mail]/versions/86.0buildid20210128185743/uploads/907b71ccadda45efaa59e291630063af/" returned HTTP status code: 200
2021-01-28 21:42:15,787 - addonscript.utils - DEBUG - amo_get() for URL "https://addons.mozilla.org/api/v4/addons/[e-mail]/versions/86.0buildid20210128185743/uploads/907b71ccadda45efaa59e291630063af/" returned: {'guid': '[e-mail]', 'active': True, 'automated_signing': False, 'url': 'https://addons.mozilla.org/api/v4/addons/[e-mail]/versions/86.0buildid20210128185743/uploads/907b71ccadda45efaa59e291630063af/', 'files': [{'download_url': 'https://addons.mozilla.org/api/v4/file/3717042/taqbaylit_language_pack-86.0buildid20210128185743-fx.xpi', 'hash': 'sha256:03c6b618b9fe3be2069b74120d49c47bbdcf79ad38a385ef497b8de277182bf7', 'signed': True}], 'passed_review': True, 'pk': '907b71ccadda45efaa59e291630063af', 'processed': True, 'reviewed': True, 'valid': True, 'validation_results': {'errors': 0, 'warnings': 0, 'notices': 1, 'success': True, 'compatibility_summary': {'warnings': 0, 'errors': 0, 'notices': 0}, 'metadata': {'listed': True, 'identified_files': {}, 'is_webextension': True, 'id': '[e-mail]', 'manifestVersion': 2, 'name': 'Taqbaylit Language Pack', 'type': 1, 'version': '86.0buildid20210128185743', 'firefoxMinVersion': '86.0', 'totalScannedFileSize': 1410561, 'emptyFiles': [], 'jsLibs': {}, 'unknownMinifiedFiles': []}, 'messages': [{'message': 'The "version" property uses a Firefox-specific format.', 'description': 'See http://mzl.la/1kXIADa (MDN Docs) for more information.', 'file': 'manifest.json', 'uid': '099e4864e19a450ba2bfc004c95b6d83', 'type': 'notice', 'id': ['PROP_VERSION_TOOLKIT_ONLY'], 'tier': 1}], 'message_tree': {}, 'ending_tier': 5}, 'validation_url': 'https://addons.mozilla.org/en-US/developers/upload/907b71ccadda45efaa59e291630063af', 'version': '86.0buildid20210128185743'}
2021-01-28 21:42:15,788 - addonscript.utils - DEBUG - Calling amo_download() with URL "https://addons.mozilla.org/api/v4/file/3717042/taqbaylit_language_pack-86.0buildid20210128185743-fx.xpi"
2021-01-28 21:42:16,063 - addonscript.utils - DEBUG - amo_download() for URL "https://addons.mozilla.org/api/v4/file/3717042/taqbaylit_language_pack-86.0buildid20210128185743-fx.xpi" returned HTTP status code: 200
2021-01-28 21:42:16,063 - addonscript.utils - DEBUG - Writing content at URL "https://addons.mozilla.org/api/v4/file/3717042/taqbaylit_language_pack-86.0buildid20210128185743-fx.xpi" to file "/app/artifacts/public/build/kab/target.langpack.xpi"
2021-01-28 21:44:09,256 - addonscript.utils - DEBUG - Calling amo_get() with URL "https://addons.mozilla.org/api/v4/addons/[e-mail]/versions/86.0buildid20210128185743/uploads/ddd1988db88341f0991d4cff260ff44f/"
2021-01-28 21:44:09,644 - addonscript.utils - DEBUG - amo_get() for URL "https://addons.mozilla.org/api/v4/addons/[e-mail]/versions/86.0buildid20210128185743/uploads/ddd1988db88341f0991d4cff260ff44f/" returned HTTP status code: 200
2021-01-28 21:44:09,644 - addonscript.utils - DEBUG - amo_get() for URL "https://addons.mozilla.org/api/v4/addons/[e-mail]/versions/86.0buildid20210128185743/uploads/ddd1988db88341f0991d4cff260ff44f/" returned: {'guid': '[e-mail]', 'active': False, 'automated_signing': False, 'url': 'https://addons.mozilla.org/api/v4/addons/[e-mail]/versions/86.0buildid20210128185743/uploads/ddd1988db88341f0991d4cff260ff44f/', 'files': [], 'passed_review': False, 'pk': 'ddd1988db88341f0991d4cff260ff44f', 'processed': True, 'reviewed': False, 'valid': False, 'validation_results': {'errors': 1, 'warnings': 0, 'notices': 0, 'success': False, 'compatibility_summary': {'notices': 0, 'errors': 0, 'warnings': 0}, 'metadata': {'listed': True, 'is_webextension': True}, 'messages': [{'id': ['validator', 'unexpected_exception'], 'message': "Sorry, we couldn't load your WebExtension.", 'description': ['Validation was unable to complete successfully due to an unexpected error.', 'Check https://developer.mozilla.org/en-US/Add-ons/WebExtensions to ensure your webextension is valid or file a bug at http://bit.ly/1POrYYU'], 'type': 'error', 'fatal': True, 'tier': 1, 'for_appversions': None, 'uid': '35432f419340461897aa8362398339c4'}], 'message_tree': {}, 'ending_tier': 5}, 'validation_url': 'https://addons.mozilla.org/en-US/developers/upload/ddd1988db88341f0991d4cff260ff44f', 'version': '86.0buildid20210128185743'}
2021-01-28 21:44:09,645 - scriptworker.utils - DEBUG - retry_async: get_signed_addon_url: sleeping 120 seconds before retry
2021-01-28 21:46:09,733 - addonscript.utils - DEBUG - Calling amo_get() with URL "https://addons.mozilla.org/api/v4/addons/[e-mail]/versions/86.0buildid20210128185743/uploads/ddd1988db88341f0991d4cff260ff44f/"
2021-01-28 21:46:10,028 - addonscript.utils - DEBUG - amo_get() for URL "https://addons.mozilla.org/api/v4/addons/[e-mail]/versions/86.0buildid20210128185743/uploads/ddd1988db88341f0991d4cff260ff44f/" returned HTTP status code: 200
2021-01-28 21:46:10,029 - addonscript.utils - DEBUG - amo_get() for URL "https://addons.mozilla.org/api/v4/addons/[e-mail]/versions/86.0buildid20210128185743/uploads/ddd1988db88341f0991d4cff260ff44f/" returned: {'guid': '[e-mail]', 'active': False, 'automated_signing': False, 'url': 'https://addons.mozilla.org/api/v4/addons/[e-mail]/versions/86.0buildid20210128185743/uploads/ddd1988db88341f0991d4cff260ff44f/', 'files': [], 'passed_review': False, 'pk': 'ddd1988db88341f0991d4cff260ff44f', 'processed': True, 'reviewed': False, 'valid': False, 'validation_results': {'errors': 1, 'warnings': 0, 'notices': 0, 'success': False, 'compatibility_summary': {'notices': 0, 'errors': 0, 'warnings': 0}, 'metadata': {'listed': True, 'is_webextension': True}, 'messages': [{'id': ['validator', 'unexpected_exception'], 'message': "Sorry, we couldn't load your WebExtension.", 'description': ['Validation was unable to complete successfully due to an unexpected error.', 'Check https://developer.mozilla.org/en-US/Add-ons/WebExtensions to ensure your webextension is valid or file a bug at http://bit.ly/1POrYYU'], 'type': 'error', 'fatal': True, 'tier': 1, 'for_appversions': None, 'uid': '35432f419340461897aa8362398339c4'}], 'message_tree': {}, 'ending_tier': 5}, 'validation_url': 'https://addons.mozilla.org/en-US/developers/upload/ddd1988db88341f0991d4cff260ff44f', 'version': '86.0buildid20210128185743'}
2021-01-28 21:46:10,029 - scriptworker.utils - DEBUG - retry_async: get_signed_addon_url: sleeping 120 seconds before retry
2021-01-28 21:48:10,089 - addonscript.utils - DEBUG - Calling amo_get() with URL "https://addons.mozilla.org/api/v4/addons/[e-mail]/versions/86.0buildid20210128185743/uploads/ddd1988db88341f0991d4cff260ff44f/"
2021-01-28 21:48:10,319 - addonscript.utils - DEBUG - amo_get() for URL "https://addons.mozilla.org/api/v4/addons/[e-mail]/versions/86.0buildid20210128185743/uploads/ddd1988db88341f0991d4cff260ff44f/" returned HTTP status code: 200
2021-01-28 21:48:10,320 - addonscript.utils - DEBUG - amo_get() for URL "https://addons.mozilla.org/api/v4/addons/[e-mail]/versions/86.0buildid20210128185743/uploads/ddd1988db88341f0991d4cff260ff44f/" returned: {'guid': '[e-mail]', 'active': False, 'automated_signing': False, 'url': 'https://addons.mozilla.org/api/v4/addons/[e-mail]/versions/86.0buildid20210128185743/uploads/ddd1988db88341f0991d4cff260ff44f/', 'files': [], 'passed_review': False, 'pk': 'ddd1988db88341f0991d4cff260ff44f', 'processed': True, 'reviewed': False, 'valid': False, 'validation_results': {'errors': 1, 'warnings': 0, 'notices': 0, 'success': False, 'compatibility_summary': {'notices': 0, 'errors': 0, 'warnings': 0}, 'metadata': {'listed': True, 'is_webextension': True}, 'messages': [{'id': ['validator', 'unexpected_exception'], 'message': "Sorry, we couldn't load your WebExtension.", 'description': ['Validation was unable to complete successfully due to an unexpected error.', 'Check https://developer.mozilla.org/en-US/Add-ons/WebExtensions to ensure your webextension is valid or file a bug at http://bit.ly/1POrYYU'], 'type': 'error', 'fatal': True, 'tier': 1, 'for_appversions': None, 'uid': '35432f419340461897aa8362398339c4'}], 'message_tree': {}, 'ending_tier': 5}, 'validation_url': 'https://addons.mozilla.org/en-US/developers/upload/ddd1988db88341f0991d4cff260ff44f', 'version': '86.0buildid20210128185743'}
2021-01-28 21:48:10,321 - scriptworker.utils - DEBUG - retry_async: get_signed_addon_url: sleeping 120 seconds before retry
2021-01-28 21:50:10,409 - addonscript.utils - DEBUG - Calling amo_get() with URL "https://addons.mozilla.org/api/v4/addons/[e-mail]/versions/86.0buildid20210128185743/uploads/ddd1988db88341f0991d4cff260ff44f/"
2021-01-28 21:50:10,707 - addonscript.utils - DEBUG - amo_get() for URL "https://addons.mozilla.org/api/v4/addons/[e-mail]/versions/86.0buildid20210128185743/uploads/ddd1988db88341f0991d4cff260ff44f/" returned HTTP status code: 200
2021-01-28 21:50:10,708 - addonscript.utils - DEBUG - amo_get() for URL "https://addons.mozilla.org/api/v4/addons/[e-mail]/versions/86.0buildid20210128185743/uploads/ddd1988db88341f0991d4cff260ff44f/" returned: {'guid': '[e-mail]', 'active': False, 'automated_signing': False, 'url': 'https://addons.mozilla.org/api/v4/addons/[e-mail]/versions/86.0buildid20210128185743/uploads/ddd1988db88341f0991d4cff260ff44f/', 'files': [], 'passed_review': False, 'pk': 'ddd1988db88341f0991d4cff260ff44f', 'processed': True, 'reviewed': False, 'valid': False, 'validation_results': {'errors': 1, 'warnings': 0, 'notices': 0, 'success': False, 'compatibility_summary': {'notices': 0, 'errors': 0, 'warnings': 0}, 'metadata': {'listed': True, 'is_webextension': True}, 'messages': [{'id': ['validator', 'unexpected_exception'], 'message': "Sorry, we couldn't load your WebExtension.", 'description': ['Validation was unable to complete successfully due to an unexpected error.', 'Check https://developer.mozilla.org/en-US/Add-ons/WebExtensions to ensure your webextension is valid or file a bug at http://bit.ly/1POrYYU'], 'type': 'error', 'fatal': True, 'tier': 1, 'for_appversions': None, 'uid': '35432f419340461897aa8362398339c4'}], 'message_tree': {}, 'ending_tier': 5}, 'validation_url': 'https://addons.mozilla.org/en-US/developers/upload/ddd1988db88341f0991d4cff260ff44f', 'version': '86.0buildid20210128185743'}
2021-01-28 21:50:10,708 - scriptworker.utils - WARNING - retry_async: get_signed_addon_url: too many retries!
2021-01-28 21:50:10,708 - scriptworker.client - ERROR - Failed to run async_main
Traceback (most recent call last):
  File "/app/lib/python3.8/site-packages/scriptworker/client.py", line 205, in _handle_asyncio_loop
    await async_main(context)
  File "/app/lib/python3.8/site-packages/addonscript/script.py", line 88, in async_main
    await asyncio.gather(*tasks)
  File "/app/lib/python3.8/site-packages/addonscript/script.py", line 43, in sign_addon
    signed_addon_url = await retry_async(
  File "/app/lib/python3.8/site-packages/scriptworker/utils.py", line 266, in retry_async
    _check_number_of_attempts(attempt, attempts, func, "retry_async")
  File "/app/lib/python3.8/site-packages/scriptworker/utils.py", line 261, in retry_async
    return await func(*args, **kwargs)
  File "/app/lib/python3.8/site-packages/addonscript/api.py", line 89, in get_signed_addon_url
    raise SignatureError("Expected 1 file. Got ({}) full response: {}".format(len(upload_status["files"]), upload_status))
addonscript.exceptions.SignatureError: Expected 1 file. Got (0) full response: {'guid': '[e-mail]', 'active': False, 'automated_signing': False, 'url': 'https://addons.mozilla.org/api/v4/addons/[e-mail]/versions/86.0buildid20210128185743/uploads/ddd1988db88341f0991d4cff260ff44f/', 'files': [], 'passed_review': False, 'pk': 'ddd1988db88341f0991d4cff260ff44f', 'processed': True, 'reviewed': False, 'valid': False, 'validation_results': {'errors': 1, 'warnings': 0, 'notices': 0, 'success': False, 'compatibility_summary': {'notices': 0, 'errors': 0, 'warnings': 0}, 'metadata': {'listed': True, 'is_webextension': True}, 'messages': [{'id': ['validator', 'unexpected_exception'], 'message': "Sorry, we couldn't load your WebExtension.", 'description': ['Validation was unable to complete successfully due to an unexpected error.', 'Check https://developer.mozilla.org/en-US/Add-ons/WebExtensions to ensure your webextension is valid or file a bug at http://bit.ly/1POrYYU'], 'type': 'error', 'fatal': True, 'tier': 1, 'for_appversions': None, 'uid': '35432f419340461897aa8362398339c4'}], 'message_tree': {}, 'ending_tier': 5}, 'validation_url': 'https://addons.mozilla.org/en-US/developers/upload/ddd1988db88341f0991d4cff260ff44f', 'version': '86.0buildid20210128185743'}
exit code: 7```

Comment 1

[Alexandru Michis [:malexandru]]

Aki, should this or Bugs 1586124, 1578091 be modified to have a more general failure summary, to encompass these kind of failures?

Comment 2

[Aki Sasaki (not active)]

I think they're all probably the same type of bug, but for different langpack locales.
I think we've hit these when the AMO db was set to read-only for an upgrade. We may have also hit AMO-side configuration issues. I think each of these required a fix on the AMO side to resolve.

Would it help if addonscript output a more human-friendly error message in these cases? Or is there some other relatively straightforward way to combine these bugs?

Comment 3

[Alexandru Michis [:malexandru]]

I was thinking of modifying the bug summary to something like:
"Intermittent addonscript.exceptions.SignatureError: Expected 1 file. Got (0) full response: {...}",
Then marking the other bugs as a duplicate to this one.

Comment 4

[Aki Sasaki (not active)]

That works for me!

Comment 5

[Intermittent Failures Robot]

1 failures in 3891 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1

Platform and build breakdown:

• linux64-shippable: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1689486&startday=2021-01-25&endday=2021-01-31&tree=all

Comment 7

[Aki Sasaki (not active)]

Intermittent last seen 2 years ago.