Windows 8 L10n Repack often failing on aurora with HTTPError: 400 Client Error: BAD REQUEST



Release Engineering
General Automation
7 months ago
4 months ago


(Reporter: aryx, Unassigned)


Firefox Tracking Flags

(Not tracked)


One failure for Thursday's nightly after bug bug 1344321 landed.

Saturday and Sunday's Nightly saw 5 out of 6 repacks for Win 8 x64 failing:
Flags: needinfo?(bugspam.Callek)
Per one log this is more of a balrog publish failure, which we've seen before...

05:01:40     INFO -  "PUT /api/releases/Firefox-mozilla-aurora-nightly-latest/builds/WINNT_x86-msvc/af HTTP/1.1" 400 87
05:01:40     INFO -  Caught HTTPError: {"data": ["Failed to update row, old_data_version doesn't match current data_version"]}
05:01:40     INFO -  REQUEST STATS: {"url": "", "timestamp": 1489924900.63, "method": "PUT", "elapsed_secs": 4.871000051498413, "status_code": 400}
05:01:40     INFO -  retry: Caught exception:
05:01:40     INFO -  Traceback (most recent call last):
05:01:40     INFO -    File "c:\builds\moz2_slave\m-aurora-w32-l10n-ntly-1-00000\build\tools\lib\python\vendor\redo-1.4.1\redo\", line 152, in retry
05:01:40     INFO -      return action(*args, **kwargs)
05:01:40     INFO -    File "c:\builds\moz2_slave\m-aurora-w32-l10n-ntly-1-00000\build\tools\lib\python\balrog\submitter\", line 338, in update_latest
05:01:40     INFO -      data_version=latest_data_version)
05:01:40     INFO -    File "c:\builds\moz2_slave\m-aurora-w32-l10n-ntly-1-00000\build\tools\lib\python\vendor\balrogclient-0.0.1\balrogclient\", line 223, in update_build
05:01:40     INFO -      return self.request(method='PUT', data=data)
05:01:40     INFO -    File "c:\builds\moz2_slave\m-aurora-w32-l10n-ntly-1-00000\build\tools\lib\python\vendor\balrogclient-0.0.1\balrogclient\", line 111, in request
05:01:40     INFO -      return self.do_request(url, data, method)
05:01:40     INFO -    File "c:\builds\moz2_slave\m-aurora-w32-l10n-ntly-1-00000\build\tools\lib\python\vendor\balrogclient-0.0.1\balrogclient\", line 129, in do_request
05:01:40     INFO -      req.raise_for_status()
05:01:40     INFO -    File "c:\builds\moz2_slave\m-aurora-w32-l10n-ntly-1-00000\build\tools\scripts\updates\../../lib/python/vendor/requests-2.7.0\requests\", line 851, in raise_for_status
05:01:40     INFO -      raise HTTPError(http_error_msg, response=self)
05:01:40     INFO -  HTTPError: 400 Client Error: BAD REQUEST
05:01:40     INFO -  retry: Giving up on <function update_latest at 0x02C80330>

MANY of the errors are INFO only due to RETRIES.

I think there is a bug to dupe to, but I can't find it... Ben do you know said bug?
Flags: needinfo?(bugspam.Callek) → needinfo?(bhearsum)
It's a bit surprising that this would happen all of a sudden - nothing has changed on the backend that would make it more likely to hit this. I wonder if we increased parallelism is funsize recently - that would certainly make it more likely to hit these races.
Flags: needinfo?(bhearsum)


4 months ago
Last Resolved: 4 months ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.