1408696 - mozregression can't verify the revisions it itself produced

Status: NEW

(Testing :: mozregression, defect)

Description

[Gian-Carlo Pascutto [:gcp]]

16:53.75 INFO: No more inbound revisions, bisection finished.
16:53.75 INFO: Last good revision: 1053bf2fbaed195d37a07ae81d5086cac0b8f6b8
16:53.75 INFO: First bad revision: abcd8dfc3a869a1c5292f91b48b7e21f48766f59
16:53.75 INFO: Pushlog:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=1053bf2fbaed195d37a07ae81d5086cac0b8f6b8&tochange=abcd8dfc3a869a1c5292f91b48b7e21f48766f59

So far so good. Now let's double-check this result:

Gian-Carlo Pascutto@Ivy ~
$ mozregression -g 1053bf2fbaed195d37a07ae81d5086cac0b8f6b8 -b  abcd8dfc3a869a1
c5292f91b48b7e21f48766f59
**********
You should use a config file. Please use the --write-config command line flag to help you create one.
**********

 0:00.55 INFO: bits option not specified, using 64-bit builds.
 0:00.55 INFO: abcd8dfc3a869a1c5292f91b48b7e21f48766f59 is not a release, assuming it's a hash...
 0:00.55 INFO: 1053bf2fbaed195d37a07ae81d5086cac0b8f6b8 is not a release, assuming it's a hash...
 0:00.56 INFO: Getting mozilla-inbound builds between 1053bf2fbaed195d37a07ae81d5086cac0b8f6b8 and abcd8dfc3a869a1c5292f91b48b7e21f48766f59
 0:01.83 ERROR: The url 'https://hg.mozilla.org/integration/mozilla-inbound/json-pushes?fromchange=1053bf2fbaed195d37a07ae81d5086cac0b8f6b8&tochange=abcd8dfc3a869a1c5292f91b48b7e21f48766f59' contains no pushlog. Maybe use another range ?

It doesn't seem to know it can look into the autoland repos?

Comment 1

[William Lachance (:wlach)]

(In reply to Gian-Carlo Pascutto [:gcp] from comment #0)
> It doesn't seem to know it can look into the autoland repos?

There should be a --branch option which would let you bisect into autoland. Maybe we should say that we're defaulting to mozilla-inbound when that option isn't specified, in the same way as we do for bits.

Comment 2

[Gian-Carlo Pascutto [:gcp]]

I don't think it should be required to give any extra option besides the ones already given in comment 0. mozregression gave those revisions, surely it can understand them when its own output is fed back?

At worst, it can simply try autoland if there's no hit elsewhere.

Comment 3

[William Lachance (:wlach)]

(In reply to Gian-Carlo Pascutto [:gcp] from comment #2)
> I don't think it should be required to give any extra option besides the
> ones already given in comment 0. mozregression gave those revisions, surely
> it can understand them when its own output is fed back?

I don't see why this would follow. It bisected into autoland on one run, but it's not going to remember that the next time you run it? Not without some additional logic anyway (see below).

> At worst, it can simply try autoland if there's no hit elsewhere.

We could have some kind of logic to auto-try a bunch of branches to see if those revisions map onto them, but I worry about how robust that would be (most attempts to make mozregression "smart" in ways like this have wound up blowing up in our faces -- the reality is we just don't have the resources to understand and test all the edge cases). 

If someone wants to write a patch to do what you suggest and can give me some assurance that they've tested it thoroughly, I would take it. Giving the user a hint to specify --branch seems like a reasonable stopgap in the meantime.