1199618 - mozregression 0.41 fails to find inbound builds for revisions from this month, works in 0.36

Status: RESOLVED FIXED

(Testing :: mozregression, defect)

Description

[:Gijs (he/him)]

On OS X:

mozregression --inbound-branch mozilla-inbound --good-rev aeb85029c3b3 --bad-rev 2fad09a0540d

works in 0.36, and fails in 0.41 claiming there are no inbound builds.

gkruitbosch-16516:users gkruitbosch$ mozregression --inbound-branch mozilla-inbound --good-rev aeb85029c3b3 --bad-rev 2fad09a0540d
 0:00.62 LOG: MainThread Bisector INFO Getting inbound builds between aeb85029c3b3 and 2fad09a0540d
 3:41.41 LOG: MainThread Bisector INFO No inbound data found.
 3:41.41 LOG: MainThread Bisector INFO There are no build artifacts on inbound for these changesets (they are probably too old).

The same thing happens for smaller sets in that range. Then I went back to 0.36 which I'd seen working for fx-team earlier, and that worked fine. Don't know why it broke.

Comment 1

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Works on Windows with mozgression 0.41, but took a time until the download started. Might this have been an infra issue?

$ mozregression --inbound-branch mozilla-inbound --good-rev aeb85029c3b3 --bad-rev 2fad09a0540d
 0:01.69 LOG: MainThread INFO bits option not specified, using 64-bit builds.
 0:01.69 LOG: MainThread Bisector INFO Getting inbound builds between aeb85029c3b3 and 2fad09a0540d
 1:30.30 LOG: MainThread Bisector INFO Downloading build from: https://queue.taskcluster.net/v1/task/rBJWP6lfQu6DG6jP6j3F0g/artifacts/public/build/firefox-42.0a1.en-US.win64.zip

Comment 2

[Julien Pagès (:parkouss)]

I think it is something related to taskcluster, since we use that for inbounds since 0.37.

My guess is that they changed the way the data is accessible... sigh.

> mozregression --good-rev aeb85029c3b3 --bad-rev 2fad09a0540d --log-mach-level debug
>  0:00.41 LOG: MainThread main INFO Getting inbound builds between aeb85029c3b3 and 2fad09a0540d
>  0:02.04 LOG: MainThread Build Finder INFO Found 271 pushlog entries using `https://hg.mozilla.org/integration/mozilla-inbound/json-pushes?fromchange=aeb85029c3b3&tochange=2fad09a0540d`
>  0:02.04 LOG: MainThread Build Finder DEBUG We need to fetch the lower limit
>  0:02.04 LOG: MainThread Build Finder DEBUG We need to fetch the higher limit
>  0:02.04 LOG: MainThread Build Finder DEBUG We got 271 folders, we need to fetch [0, 1, 269, 270]
>  0:02.04 LOG: Thread-1 mozregression.fetch_build_info DEBUG using taskcluster route 'buildbot.revisions.aeb85029c3b3.mozilla-inbound.linux64'
>  0:02.05 LOG: Thread-2 mozregression.fetch_build_info DEBUG using taskcluster route 'buildbot.revisions.67761c9de541.mozilla-inbound.linux64'
>  0:02.05 LOG: Thread-3 mozregression.fetch_build_info DEBUG using taskcluster route 'buildbot.revisions.29a6e5b45ef9.mozilla-inbound.linux64'
>  0:02.05 LOG: Thread-4 mozregression.fetch_build_info DEBUG using taskcluster route 'buildbot.revisions.c2b5e339ec06.mozilla-inbound.linux64'
>  0:06.90 LOG: MainThread Build Finder DEBUG Now we got 268 folders - 3 were bad
>  0:06.90 LOG: MainThread Build Finder DEBUG We need to fetch the higher limit
>  0:06.91 LOG: MainThread Build Finder DEBUG We got 268 folders, we need to fetch [264, 265, 266, 267]
>  0:06.91 LOG: Thread-5 mozregression.fetch_build_info DEBUG using taskcluster route 'buildbot.revisions.1bf04cb6ea41.mozilla-inbound.linux64'
>  0:06.91 LOG: Thread-6 mozregression.fetch_build_info DEBUG using taskcluster route 'buildbot.revisions.2170e194fe60.mozilla-inbound.linux64'
>  0:06.91 LOG: Thread-7 mozregression.fetch_build_info DEBUG using taskcluster route 'buildbot.revisions.f4610f8fd0be.mozilla-inbound.linux64'
>  0:06.91 LOG: Thread-8 mozregression.fetch_build_info DEBUG using taskcluster route 'buildbot.revisions.0ecdc08f4f0a.mozilla-inbound.linux64'
>  0:10.86 LOG: MainThread Build Finder DEBUG Now we got 264 folders - 4 were bad
> ...

But this is working well if I tweak the code to use full changesets now (instead of only the first 12 characters):

> mozregression --good-rev aeb85029c3b3 --bad-rev 2fad09a0540d --log-mach-level debug
>  0:00.43 LOG: MainThread main INFO Getting inbound builds between aeb85029c3b3 and 2fad09a0540d
>  0:02.18 LOG: MainThread Build Finder INFO Found 271 pushlog entries using `https://hg.mozilla.org/integration/mozilla-inbound/json-pushes?fromchange=aeb85029c3b3&tochange=2fad09a0540d`
>  0:02.18 LOG: MainThread Build Finder DEBUG We need to fetch the lower limit
>  0:02.18 LOG: MainThread Build Finder DEBUG We need to fetch the higher limit
>  0:02.18 LOG: MainThread Build Finder DEBUG We got 271 folders, we need to fetch [0, 1, 269, 270]
>  0:02.18 LOG: Thread-1 mozregression.fetch_build_info DEBUG using taskcluster route 'buildbot.revisions.aeb85029c3b3b594d96a9c72d04b8e971e1bdf5b.mozilla-inbound.linux64'
>  0:02.18 LOG: Thread-2 mozregression.fetch_build_info DEBUG using taskcluster route 'buildbot.revisions.67761c9de5410a874a34c31c638c987f109253d0.mozilla-inbound.linux64'
>  0:02.19 LOG: Thread-3 mozregression.fetch_build_info DEBUG using taskcluster route 'buildbot.revisions.29a6e5b45ef9939d8cc4a57751be53ee33b581c5.mozilla-inbound.linux64'
>  0:02.19 LOG: Thread-4 mozregression.fetch_build_info DEBUG using taskcluster route 'buildbot.revisions.c2b5e339ec06106b6feee56f9d5af26da7276214.mozilla-inbound.linux64'
>  0:07.09 LOG: MainThread Build Finder DEBUG Now we got 271 folders - 0 were bad
>  0:07.09 LOG: MainThread Build Finder DEBUG We need to fetch the mid point 135
>  0:07.09 LOG: MainThread Build Finder DEBUG We got 271 folders, we need to fetch [133, 134, 135, 136]
>  0:07.09 LOG: Thread-5 mozregression.fetch_build_info DEBUG using taskcluster route 'buildbot.revisions.c4dc03f004d602576c19243cb1963130e6e5a9a4.mozilla-inbound.linux64'
>  0:07.09 LOG: Thread-6 mozregression.fetch_build_info DEBUG using taskcluster route 'buildbot.revisions.e90476303f406b58daa6d5eb35edf7e5b85ccb22.mozilla-inbound.linux64'
>  0:07.10 LOG: Thread-7 mozregression.fetch_build_info DEBUG using taskcluster route 'buildbot.revisions.d31e43ead2dcd242e6fba23cd970ae519c7ae6ef.mozilla-inbound.linux64'
>  0:07.10 LOG: Thread-8 mozregression.fetch_build_info DEBUG using taskcluster route 'buildbot.revisions.f763ac7bd0bdb6f643c64c2ddb9275023719f59a.mozilla-inbound.linux64'
>  0:15.26 LOG: MainThread Build Finder DEBUG Now we got 271 folders - 0 were bad
>  0:15.26 LOG: MainThread download INFO Downloading build from: https://queue.taskcluster.net/v1/task/A_8AzyEOTU28PgnU9Ks-EQ/artifacts/public/build/firefox-42.0a1.en-US.linux-x86_64.tar.bz2

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

So since Aug 12th the rev ids are getting stored with the full changeset. I filed bug 1193730 as a regression tracking bug.

Comment 4

[Julien Pagès (:parkouss)]

Attachment: use 40 chars for firefox builds

Comment 5

[William Lachance (:wlach)]

Comment on attachment 8654210 [details] [review]
use 40 chars for firefox builds

One minor suggestion on the comment, but looks good. Thanks!

Comment 6

[Julien Pagès (:parkouss)]

Landed in https://github.com/mozilla/mozregression/commit/e91aed45fe4cecec3d3f0a2e5dcc319692fa477f

I will release a new mozregression version tomorrow with that change.