1201249 - Funsize should lookup previous nightlies per platform/locale

Status: RESOLVED WORKSFORME

(Release Engineering :: General, defect, P3)

Description

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Today the funsize tasks were only generating 3 partials instead of 4 on Windows. For Linux and OS X all looked fine.

https://treeherder.allizom.org/#/jobs?repo=mozilla-aurora&revision=40a77893341e

It's better visible via the Firefox UI updates results:

https://treeherder.allizom.org/#/jobs?repo=mozilla-aurora&revision=40a77893341e&filter-job_group_symbol=Ff&filter-job_group_symbol=Fr&filter-job_group_symbol=Fu

Comment 1

[Rail Aliiev [:rail]]

Looks like there was no windows build with build ID 20150829004007. Funsize doesn't handle platform specific balrog history, instead it uses global history (top level blob) to get latest build IDs and then iterates over them to get platform/locale specific date.

 Sep 02 04:59:21 buildbot-master91.bb.releng.usw2.mozilla.com python: funsize_scheduler 2015-09-02 04:59:21,163 - funsize.balrog - INFO - Connecting to https://aus4-admin.mozilla.org/api/releases/Firefox-mozilla-aurora-nightly-20150829004007/builds/WINNT_x86-msvc/en-US
Sep 02 04:59:21 buildbot-master91.bb.releng.usw2.mozilla.com python: funsize_scheduler 2015-09-02 04:59:21,418 - funsize.worker - ERROR - Error getting build, skipping this scenario
Sep 02 04:59:21 buildbot-master91.bb.releng.usw2.mozilla.com python: funsize_scheduler Traceback (most recent call last):
Sep 02 04:59:21 buildbot-master91.bb.releng.usw2.mozilla.com python: funsize_scheduler   File "/builds/funsize/lib/python2.7/site-packages/funsize/worker.py", line 188, in create_partials
Sep 02 04:59:21 buildbot-master91.bb.releng.usw2.mozilla.com python: funsize_scheduler     release_from, platform, locale)
Sep 02 04:59:21 buildbot-master91.bb.releng.usw2.mozilla.com python: funsize_scheduler   File "/builds/funsize/lib/python2.7/site-packages/funsize/balrog.py", line 72, in get_build
Sep 02 04:59:21 buildbot-master91.bb.releng.usw2.mozilla.com python: funsize_scheduler     errors=[500])
Sep 02 04:59:21 buildbot-master91.bb.releng.usw2.mozilla.com python: funsize_scheduler   File "/builds/funsize/lib/python2.7/site-packages/funsize/balrog.py", line 17, in _retry_on_http_errors
Sep 02 04:59:21 buildbot-master91.bb.releng.usw2.mozilla.com python: funsize_scheduler     req.raise_for_status()
Sep 02 04:59:21 buildbot-master91.bb.releng.usw2.mozilla.com python: funsize_scheduler   File "/builds/funsize/lib/python2.7/site-packages/requests/models.py", line 825, in raise_for_status
Sep 02 04:59:21 buildbot-master91.bb.releng.usw2.mozilla.com python: funsize_scheduler     raise HTTPError(http_error_msg, response=self)
Sep 02 04:59:21 buildbot-master91.bb.releng.usw2.mozilla.com python: funsize_scheduler HTTPError: 404 Client Error: NOT FOUND
Sep 02 04:59:21 buildbot-master91.bb.releng.usw2.mozilla.com python: funsize_scheduler 2015-09-02 04:59:21,418 - funsize.worker - WARNING - Nothing to submit

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Wouldn't it be good if we go back even further in date if there is no build for a given day? I mean once we do not build Aurora due to no changes over a day, we would get lesser partials? This could become a problem if that happens for multiple day like a long weekend. Lets say we do not build for 3 days, so we only get a single partial for today - 4?

Comment 3

[Rail Aliiev [:rail]]

Let's morph the title.

The implementation may be a bit tricky...

Comment 4

[Simon Fraser [:sfraser] ⌚️GMT]

I think this is moot now, right?