1247610 - Mozregression gets lost when switching from mozilla-central to mozilla-inbound

Status: RESOLVED FIXED

(Testing :: mozregression, defect)

Description

[Jeff Muizelaar [:jrmuizel]]

In the following log:

https://bug1247585.bmoattachments.org/attachment.cgi?id=8718324

We see something like this:

https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=0babaa3edcf908c393b68a3dc2d1c2a2450c31ed&tochange=0711218a018d912036f7d3be2ae2649e213cfb85

 3:21.65 INFO: Switching bisection method to taskcluster
 3:21.65 INFO: Getting mozilla-central builds between 0babaa3edcf908c393b68a3dc2d1c2a2450c31ed and 0711218a018d912036f7d3be2ae2649e213cfb85
 3:23.55 WARNING: Skipping build 0711218a018d: Unable to find build info using the taskcluster route 'gecko.v2.mozilla-central.revision.0711218a018d912036f7d3be2ae2649e213cfb85.firefox.win64-opt'
 3:28.14 INFO: Pushlog:
https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=0babaa3edcf908c393b68a3dc2d1c2a2450c31ed&tochange=f143af51f6e35932927b8ccac2509facbbe7b539


This WARNING seems to cause us to skip the inbound builds which results in a bad regression window.

Comment 1

[Julien Pagès (:parkouss)]

I understand the issue. For a given range, mozregression will reduce it until it find one start and end builds founds, e.g.:

given [1, 2, 3]
       G     B
if 3 can't be found
we start with [1, 2]
               G  B

And this is broken, since 3 may be the regression. While this is acceptable for the initial given range (though that may be discussed also) it is not at all when we are doing that before going to another branch.

Comment 2

[Julien Pagès (:parkouss)]

Attachment: expand ranges if required

These patches should fix the issue, by expanding the range instead of shrinking it when we move to a "sub bisection" (e.g. switching branches to follow a merge or moving from nightlies to taskcluster back-end).

When a limit of the range is not valid (lower or higher), it will now try to find the nearest appropriate build to use instead (trying 20 builds max for each side, and defaulting to the current behavior if we can't find a valid build, but with a CRITICAL message to explain the situation).

So,
given [1, 2, 3]
       G     B
if 3 can't be found, we will now look into the next range [4, 5, ... 24].

If 4 is valid, we'll get:
given [1, 2, 4]
       G     B

Instead if 4 is broken but 5 valid, we'll get:
given [1, 2, 5]
       G     B

As a side note, this does not change the way the initial ranges given by the user (dates or changesets) are handled: they will still be shrunk in case of missing limit builds. We could change (fix ?) that, but I would prefer another bug to track it since it will be a change of the behavior. So please open a new bug if you'd like that.

Comment 3

[William Lachance (:wlach)]

Comment on attachment 8719267 [details] [review]
expand ranges if required

The logic here makes sense! Thanks

Comment 4

[Julien Pagès (:parkouss)]

Merged in https://github.com/mozilla/mozregression/commit/638ccdee60e85c3354974ffb1c30da0699587a72 - I am going to do a patch release now.