1144703 - Some locales are not sent to balrog (l10n + mozharness)

Status: RESOLVED DUPLICATE of bug 1151459

(Release Engineering :: Applications: MozharnessCore, defect)

Description

[Massimo Gervasini [:massimo]]

After moving to the in tree mechanism for creating l10n binaries with mozharness, not all the created binaries are sent to balrog (only the last locale is sent).

Probably we need to move the balrog submission step just after the make upload step and figure out how to pass to balrog the partial file too.

Comment 1

[Massimo Gervasini [:massimo]]

Attachment: [mozharness] Bug 1144703 - Missing some locales in balrog submission.patch

We don't need to move the the balrog submission just after make upload. 
What was missing here was getting the partialInfo from the upload, and setting the buildbot property in submit_to_balrog()

tested in staging, full log here: http://dev-master2.bb.releng.use1.mozilla.com:8914/builders/WINNT%206.1%20x86-64%20ash%20nightly%20l10n%201%2F5/builds/14/steps/run_script/logs/stdio
(requires VPN)

Comment 2

[Massimo Gervasini [:massimo]]

Comment on attachment 8581855 [details] [diff] [review]
[mozharness] Bug 1144703 - Missing some locales in balrog submission.patch

Thanks Kim!

https://hg.mozilla.org/build/mozharness/rev/2ca53a19719b

Comment 3

[Chris Cooper [:coop] (he/him)]

mozharness production tag moved to: https://hg.mozilla.org/build/mozharness/rev/8faa188c6306

Comment 4

[Chris AtLee [:catlee]]

We're hitting some errors sometimes submitting these:

07:10:15     INFO -  Starting new HTTPS connection (1): aus4-admin-dev.allizom.org
07:10:16     INFO -  "HEAD /api/releases/Firefox-mozilla-central-nightly-20150428030209 HTTP/1.1" 200 0
07:10:16     INFO -  Balrog request to https://aus4-admin-dev.allizom.org/api/releases/Firefox-mozilla-central-nightly-20150428030209/builds/Linux_x86-gcc3/nb-NO
07:10:16     INFO -  Data sent: {'product': u'Firefox', 'hashFunction': u'sha512', 'alias': 'null', 'schema_version': 4, 'data_version': '50', 'copyTo': '["Firefox-mozilla-central-nightly-latest"]', 'version': u'40.0a1', 'data': '{"platformVersion": "40.0a1", "partials": [{"fileUrl": "http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2015/04/2015-04-28-03-02-09-mozilla-central-l10n/firefox-40.0a1.nb-NO.linux-i686.partial.20150428030209-20150428093330.mar", "hashValue": "64b0f3fd45b4731b738f3d8f5d34f8fde4ba2ffc0db7b70545b59e98a4e90836f6f3256ee27369bc0070d0167af2f67636410eb2a48ada3b627eac72c3779130", "from": "Firefox-mozilla-central-nightly-20150428030209", "filesize": 360388}], "completes": [{"fileUrl": "http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2015/04/2015-04-28-03-02-09-mozilla-central-l10n/firefox-40.0a1.nb-NO.linux-i686.complete.mar", "hashValue": "a425e5e1f41fdccbf493dac8c32cafbf46b8fdee0b212d4acdab9ea748b553557a35f48aa6b69740a686ee6dcf21f4b0b5c6c980eabd91e0c787a055a610a75b", "from": "*", "filesize": 54556496}], "buildID": "20150428030209", "displayVersion": "40.0a1", "appVersion": "40.0a1"}'}
07:10:16     INFO -  Starting new HTTPS connection (2): aus4-admin-dev.allizom.org
07:10:17     INFO -  "PUT /api/releases/Firefox-mozilla-central-nightly-20150428030209/builds/Linux_x86-gcc3/nb-NO HTTP/1.1" 400 98
07:10:17     INFO -  Caught HTTPError: Couldn't update release: Failed to update row, old_data_version doesn't match current data_version
07:10:17     INFO -  Traceback (most recent call last):
07:10:17     INFO -    File "/builds/slave/m-cen-lx-l10n-ntly-2-000000000/build/tools/scripts/updates/balrog-submitter.py", line 86, in <module>
07:10:17     INFO -      extVersion, isOSUpdate=isOSUpdate, **updateKwargs)
07:10:17     INFO -    File "/builds/slave/m-cen-lx-l10n-ntly-2-000000000/build/tools/scripts/updates/../../lib/python/balrog/submitter/cli.py", line 352, in run
07:10:17     INFO -      return NightlySubmitterBase.run(self, *args, schemaVersion=4, **kwargs)
07:10:17     INFO -    File "/builds/slave/m-cen-lx-l10n-ntly-2-000000000/build/tools/scripts/updates/../../lib/python/balrog/submitter/cli.py", line 304, in run
07:10:17     INFO -      schemaVersion=schemaVersion)
07:10:17     INFO -    File "/builds/slave/m-cen-lx-l10n-ntly-2-000000000/build/tools/scripts/updates/../../lib/python/balrog/submitter/api.py", line 163, in update_build
07:10:17     INFO -      url_template_vars=url_template_vars)
07:10:17     INFO -    File "/builds/slave/m-cen-lx-l10n-ntly-2-000000000/build/tools/scripts/updates/../../lib/python/balrog/submitter/api.py", line 103, in request
07:10:17     INFO -      return self.do_request(url, data, method, url_template_vars)
07:10:17     INFO -    File "/builds/slave/m-cen-lx-l10n-ntly-2-000000000/build/tools/scripts/updates/../../lib/python/balrog/submitter/api.py", line 119, in do_request
07:10:17     INFO -      headers=headers)
07:10:17     INFO -    File "/builds/slave/m-cen-lx-l10n-ntly-2-000000000/build/tools/scripts/updates/../../lib/python/vendor/requests-0.10.8/requests/sessions.py", line 203, in request
07:10:17     INFO -      r.send(prefetch=prefetch)
07:10:17     INFO -    File "/builds/slave/m-cen-lx-l10n-ntly-2-000000000/build/tools/scripts/updates/../../lib/python/vendor/requests-0.10.8/requests/models.py", line 585, in send
07:10:17     INFO -      self.response.raise_for_status()
07:10:17     INFO -    File "/builds/slave/m-cen-lx-l10n-ntly-2-000000000/build/tools/scripts/updates/../../lib/python/vendor/requests-0.10.8/requests/models.py", line 810, in raise_for_status
07:10:17     INFO -      raise http_error
07:10:17     INFO -  requests.exceptions.HTTPError: 400 Client Error
07:10:17    ERROR - Return code: 1

(from http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2015/04/2015-04-28-03-02-09-mozilla-central-l10n/mozilla-central-linux-l10n-nightly-2-unknown-bm77-build1-build7.txt.gz)