Closed Bug 1376502 Opened 3 years ago Closed 3 years ago

publish to Balrog no longer works in automation for desktop beta

Categories

(Release Engineering :: Release Automation: Other, defect)

defect
Not set

Tracking

(firefox56 fixed)

VERIFIED FIXED
Tracking Status
firefox56 --- fixed

People

(Reporter: mtabara, Assigned: rail)

References

Details

Attachments

(2 files)

Not sure if it's a fallback from enabling signoffs but we fail to update rules in Balrog when we publish. Seems like it was an issue for beta 4 and it failed on beta 5 as well.

Logs are also full of Errors but yet the task reports as green. 
Fixing this by hand for 55.0b5 - manually scheduling the update in Balrog.

10:37:20     INFO - Return code: 0
10:37:20     INFO - Changing directory to /builds/slave/rel-m-beta-fx_publish_balrog-0.
10:37:20     INFO - [mozharness: 2017-06-27 17:37:20.361038Z] Finished pull step (success)
10:37:20     INFO - [mozharness: 2017-06-27 17:37:20.361232Z] Running submit-to-balrog step.
10:37:20     INFO - Running main action method: submit_to_balrog
10:37:20     INFO - retry: Calling <lambda> with args: (), kwargs: {}, attempt #1
10:37:20     INFO - Running command: ['/tools/buildbot/bin/python', '/builds/slave/rel-m-beta-fx_publish_balrog-0/build/tools/scripts/build-promotion/balrog-release-shipper.py', '--api-root', u'https://aus4-admin.mozilla.org/api', '--credentials-file', '/builds/slave/rel-m-beta-fx_publish_balrog-0/oauth.txt', '--username', 'balrog-ffxbld', '--version', u'55.0b5', '--product', u'firefox', '--build-number', '1', '--verbose', '--rules', 32]
10:37:20     INFO - retry: attempt #1 caught TypeError exception: argument of type 'int' is not iterable
10:37:20     INFO - retry: Failed, sleeping 60 seconds before retrying
10:38:20     INFO - retry: Calling <lambda> with args: (), kwargs: {}, attempt #2
10:38:20     INFO - Running command: ['/tools/buildbot/bin/python', '/builds/slave/rel-m-beta-fx_publish_balrog-0/build/tools/scripts/build-promotion/balrog-release-shipper.py', '--api-root', u'https://aus4-admin.mozilla.org/api', '--credentials-file', '/builds/slave/rel-m-beta-fx_publish_balrog-0/oauth.txt', '--username', 'balrog-ffxbld', '--version', u'55.0b5', '--product', u'firefox', '--build-number', '1', '--verbose', '--rules', 32]
10:38:20     INFO - retry: attempt #2 caught TypeError exception: argument of type 'int' is not iterable
10:38:20     INFO - retry: Failed, sleeping 120 seconds before retrying
10:40:20     INFO - retry: Calling <lambda> with args: (), kwargs: {}, attempt #3
10:40:20     INFO - Running command: ['/tools/buildbot/bin/python', '/builds/slave/rel-m-beta-fx_publish_balrog-0/build/tools/scripts/build-promotion/balrog-release-shipper.py', '--api-root', u'https://aus4-admin.mozilla.org/api', '--credentials-file', '/builds/slave/rel-m-beta-fx_publish_balrog-0/oauth.txt', '--username', 'balrog-ffxbld', '--version', u'55.0b5', '--product', u'firefox', '--build-number', '1', '--verbose', '--rules', 32]
10:40:20     INFO - retry: attempt #3 caught TypeError exception: argument of type 'int' is not iterable
10:40:20     INFO - retry: Failed, sleeping 240 seconds before retrying
10:44:20     INFO - retry: Calling <lambda> with args: (), kwargs: {}, attempt #4
10:44:20     INFO - Running command: ['/tools/buildbot/bin/python', '/builds/slave/rel-m-beta-fx_publish_balrog-0/build/tools/scripts/build-promotion/balrog-release-shipper.py', '--api-root', u'https://aus4-admin.mozilla.org/api', '--credentials-file', '/builds/slave/rel-m-beta-fx_publish_balrog-0/oauth.txt', '--username', 'balrog-ffxbld', '--version', u'55.0b5', '--product', u'firefox', '--build-number', '1', '--verbose', '--rules', 32]
10:44:20     INFO - retry: attempt #4 caught TypeError exception: argument of type 'int' is not iterable
10:44:20     INFO - retry: Failed, sleeping 300 seconds before retrying
10:49:20     INFO - retry: Calling <lambda> with args: (), kwargs: {}, attempt #5
10:49:20     INFO - Running command: ['/tools/buildbot/bin/python', '/builds/slave/rel-m-beta-fx_publish_balrog-0/build/tools/scripts/build-promotion/balrog-release-shipper.py', '--api-root', u'https://aus4-admin.mozilla.org/api', '--credentials-file', '/builds/slave/rel-m-beta-fx_publish_balrog-0/oauth.txt', '--username', 'balrog-ffxbld', '--version', u'55.0b5', '--product', u'firefox', '--build-number', '1', '--verbose', '--rules', 32]
10:49:20     INFO - retry: attempt #5 caught TypeError exception: argument of type 'int' is not iterable
10:49:20    ERROR - <function <lambda> at 0x245aa28> failed after 5 tries!
10:49:20    ERROR - Caught exception: argument of type 'int' is not iterable
10:49:20    ERROR - Caught exception: argument of type 'int' is not iterable
10:49:20    ERROR - Caught exception: argument of type 'int' is not iterable
10:49:20    ERROR - Caught exception: argument of type 'int' is not iterable
10:49:20    ERROR - Caught exception: argument of type 'int' is not iterable
10:49:20     INFO - [mozharness: 2017-06-27 17:49:20.728956Z] Finished submit-to-balrog step (success)
10:49:20     INFO - Running post-run listener: copy_logs_to_upload_dir
10:49:20     INFO - Copying logs to upload dir...
10:49:20     INFO - mkdir: /builds/slave/rel-m-beta-fx_publish_balrog-0/build/upload/logs
10:49:20     INFO - Copying logs to upload dir...


More log info here[1]. That task is [2].

[1]: https://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-beta-noarch/release-mozilla-beta-firefox_publish_balrog-bm73-build1-build11.txt.gz
[2]: https://tools.taskcluster.net/task-group-inspector/#/nGJEKlMLTuCsHi7bdW0SjQ/dCkhME3vTROK51hnbF-mSw?_k=khgv24
Assignee: nobody → rail
Comment on attachment 8881499 [details]
Bug 1376502 - publish to Balrog no longer works in automation for desktop beta  a=release DONTBUILD

https://reviewboard.mozilla.org/r/152646/#review157780
Attachment #8881499 - Flags: review?(mtabara) → review+
Pushed by raliiev@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/947edcdbb07d
publish to Balrog no longer works in automation for desktop beta r=mtabara a=release DONTBUILD
Comment on attachment 8881499 [details]
Bug 1376502 - publish to Balrog no longer works in automation for desktop beta  a=release DONTBUILD

https://hg.mozilla.org/releases/mozilla-beta/rev/5772798432083ba5414be04d0070e6220a377f60
Attachment #8881499 - Flags: checked-in+
We should also check the return code. TBD
Keywords: leave-open
Attached patch retry.diffSplinter Review
Attachment #8881582 - Flags: review?(mtabara)
Comment on attachment 8881582 [details] [diff] [review]
retry.diff

Review of attachment 8881582 [details] [diff] [review]:
-----------------------------------------------------------------

lgtm!
Attachment #8881582 - Flags: review?(mtabara) → review+
Pushed by raliiev@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/d362c2910b39
Check command exit code r=mtabara a=release DONTBUILD
Keywords: leave-open
https://hg.mozilla.org/mozilla-central/rev/947edcdbb07d
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
See Also: → 1381882
Duplicate of this bug: 1432868
You need to log in before you can comment on or make changes to this bug.