1369479 - Thunderbird 54.0b3 build1: failed at thunderbird_beta_updates

Status: RESOLVED FIXED

(Release Engineering :: Release Automation, defect, P2)

Description

[Wayne Mery (:wsmwk)]

+++ This bug was initially created as a clone of Bug #1349525 +++

https://archive.mozilla.org/pub/thunderbird/candidates/54.0b3-candidates/build1/logs/release-comm-beta-thunderbird_beta_updates-bm73-build1-build1.txt.gz


    ========= Started 'python /builds/slave/tb-rel-c-beta-tb_beta_upds-000/tools/buildfarm/utils/retry.py ...' failed (results: 2, elapsed: 46 secs) (at 2017-05-31 20:06:23.430948) =========
    python /builds/slave/tb-rel-c-beta-tb_beta_upds-000/tools/buildfarm/utils/retry.py -s 1 -r 5 -t 1260 python /builds/slave/tb-rel-c-beta-tb_beta_upds-000/tools/scripts/updates/balrog-release-pusher.py --build-properties buildprops_balrog.json --api-root https://aus4-admin.mozilla.org/api --buildbot-configs https://hg.mozilla.org/build/buildbot-configs --release-config mozilla/release-thunderbird-comm-beta.py --credentials-file BuildSlaves.py --username tbirdbld --release-channel beta
     in dir /builds/slave/tb-rel-c-beta-tb_beta_upds-000/. (timeout 1320 secs)
     watching logfiles {}
     argv: ['python', '/builds/slave/tb-rel-c-beta-tb_beta_upds-000/tools/buildfarm/utils/retry.py', '-s', '1', '-r', '5', '-t', '1260', 'python', '/builds/slave/tb-rel-c-beta-tb_beta_upds-000/tools/scripts/updates/balrog-release-pusher.py', '--build-properties', 'buildprops_balrog.json', '--api-root', 'https://aus4-admin.mozilla.org/api', '--buildbot-configs', 'https://hg.mozilla.org/build/buildbot-configs', '--release-config', u'mozilla/release-thunderbird-comm-beta.py', '--credentials-file', 'BuildSlaves.py', '--username', 'tbirdbld', '--release-channel', 'beta']
     environment:
    ...
    Starting new HTTPS connection (1): aus4-admin.mozilla.org
    /builds/slave/tb-rel-c-beta-tb_beta_upds-000/tools/scripts/updates/../../lib/python/vendor/requests-2.7.0/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
      InsecurePlatformWarning
    Starting new HTTPS connection (1): aus4-admin.mozilla.org
    /builds/slave/tb-rel-c-beta-tb_beta_upds-000/tools/scripts/updates/../../lib/python/vendor/requests-2.7.0/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
      InsecurePlatformWarning
    Caught HTTPError: {
      "detail": "None is not of type 'object'",
      "status": 400,
      "title": "Bad Request",
      "type": "about:blank"
    }
     
    Traceback (most recent call last):
      File "/builds/slave/tb-rel-c-beta-tb_beta_upds-000/tools/scripts/updates/balrog-release-pusher.py", line 122, in <module>
        release_config['buildNumber'], testChannelRuleIds)
      File "/builds/slave/tb-rel-c-beta-tb_beta_upds-000/tools/lib/python/balrog/submitter/cli.py", line 498, in run
        ).update_rule(**data)
      File "/builds/slave/tb-rel-c-beta-tb_beta_upds-000/tools/lib/python/vendor/balrogclient-0.0.1/balrogclient/api.py", line 238, in update_rule
        return self.request(method='POST', data=rule_data)
      File "/builds/slave/tb-rel-c-beta-tb_beta_upds-000/tools/lib/python/vendor/balrogclient-0.0.1/balrogclient/api.py", line 111, in request
        return self.do_request(url, data, method)
      File "/builds/slave/tb-rel-c-beta-tb_beta_upds-000/tools/lib/python/vendor/balrogclient-0.0.1/balrogclient/api.py", line 129, in do_request
        req.raise_for_status()
      File "/builds/slave/tb-rel-c-beta-tb_beta_upds-000/tools/scripts/updates/../../lib/python/vendor/requests-2.7.0/requests/models.py", line 851, in raise_for_status
        raise HTTPError(http_error_msg, response=self)
    requests.exceptions.HTTPError: 400 Client Error: BAD REQUEST
    retry: Calling <function run_with_timeout at 0x1d1f410> with args: (['python', '/builds/slave/tb-rel-c-beta-tb_beta_upds-000/tools/scripts/updates/balrog-release-pusher.py', '--build-properties', 'buildprops_balrog.json', '--api-root', 'https://aus4-admin.mozilla.org/api', '--buildbot-configs', 'https://hg.mozilla.org/build/buildbot-configs', '--release-config', 'mozilla/release-thunderbird-comm-beta.py',

repeats...

Comment 1

[Aki Sasaki (not active)]

Has something changed in balrog submission in the past week that might cause this?

Comment 2

[Mihai Tabara [:mtabara]⌚️GMT]

Maybe https://hg.mozilla.org/build/tools/rev/8ee47b510392 missing from https://hg.mozilla.org/build/tools/shortlog/old-release-runner which is backing up TB releases? That patch was only pushed to `default` which backs-up our relpro automation.

Comment 3

[Mihai Tabara [:mtabara]⌚️GMT]

(In reply to Mihai Tabara [:mtabara]⌚️GMT from comment #2)
> Maybe https://hg.mozilla.org/build/tools/rev/8ee47b510392 missing from
> https://hg.mozilla.org/build/tools/shortlog/old-release-runner which is
> backing up TB releases? That patch was only pushed to `default` which
> backs-up our relpro automation.

Scratch this. As per IRC conversation:

~mtabara> oh, could be the fact that we didn't "uplift"  the same changes we did to balrog submitter under tools to old-release-runner as well? 
20:10:44 — ~mtabara digs for those changes again to compare
20:17:43 <@rail> that branch is used for release runner only, not for release automation 
20:22:46 <~mtabara> rail: so the lib/python/balrog still points to the `default` branch? (in TB case)
20:22:59 <@rail> yeah
20:23:23 <@aki> could releaserunner fail to set a prop for tb that is resulting in a None var?
20:23:41 <@aki> or is release automation completely separate from releaserunner

Comment 4

[[e-mail] (:bhearsum)]

Turns out this was a backend issue caused by some of the work in https://bugzilla.mozilla.org/show_bug.cgi?id=1336452. We backed it out until we can get it fixed.