1612627 - mozregression always chooses builds from 2019-11-07 regardless of the date I give it, for gve (geckoview example)

Status: RESOLVED FIXED

(Testing :: mozregression, defect, P1)

Description

[Daniel Holbert [:dholbert]]

STR:

1. Run this command:
   mozregression -n gve --good 2019-10-01 --bad 2020-01-30

ACTUAL RESULTS: This output:

0:00.40 INFO: Getting autoland builds between 2019-10-01 and 2020-01-30
0:02.00 INFO: Using bec112cfa8d2c34fc5ccc1d87a9d14f22884ee7d (pushed on 2019-11-07 16:09:10) for date 2019-10-01
0:02.00 INFO: Using 9d6c5b6e3275e5f28fbe8efc3a8041c849eac2d6 (pushed on 2019-11-07 16:08:12) for date 2020-01-30
0:04.22 INFO: Testing good and bad builds to ensure that they are really good and bad...
0:04.22 INFO: Using local file: /scratch/work/mozregression-cache/bec112cfa8d2--autoland--geckoview_example.apk
0:04.22 INFO: Running autoland build built on 2019-11-07 16:45:15.879000, revision bec112cf

Notice that both of the chosen builds here were "pushed on 2019-11-07", which is not a date I asked for and not a useful bisection range.

EXPECTED RESULTS: The tested builds should be from (or near) my requested dates.

I'm using mozregression --version 3.0.3

setting ni=wlach in the hopes that this interests him, since this seems related to bug 1606902 which he recently fixed.

Comment 1

[Daniel Holbert [:dholbert]]

Note: mozregression -n gve --launch [date] does seem to work correctly. It's only the --good/--bad workflow that's broken, as far as I'm aware.

Some dates (e.g. 2019-10-01) don't have any builds available and just instantly error out, but other days work. e.g. this command works and downloads a build with a datestamp of 10-03 (approximately my requested date):
mozregression -n gve --launch 2019-10-02

Comment 2

[William Lachance (:wlach)]

Sorry about that, I'll definitely take a look when I have a chance. Note that I'm away until February 10th.

Comment 3

[William Lachance (:wlach)]

Finally had a chance to look into this, I think the problem is that we should be looking for the builds on mozilla-central first, then only falling back to the integration branches afterwards. Manually overriding the repo to mozilla-central seems to yield sane results:

mozregression --repo mozilla-central -n gve --good 2019-10-01 --bad 2020-01-30

That said, it's still unclear to me why it can't find a pushlog for that range on mozilla-inbound. I'll investigate this more and hopefully be able to craft a fix soon.

Comment 4

[Daniel Holbert [:dholbert]]

(In reply to William Lachance (:wlach) (use needinfo!) (away until Feb 10 2020) from comment #3)

That said, it's still unclear to me why it can't find a pushlog for that range on mozilla-inbound.

Was "mozilla-inbound" a typo here, or are we actually looking at mozilla-inbound here in a way that I'm not seeing? (Note that the logging in comment 0 doesn't mention mozilla-inbound -- it says Getting autoland builds [...])

(FWIW if we are using mozilla-inbound, then it's notable that mozilla-inbound hasn't had any updates since it became unsupported-as-a-push-target on Nov 19, 2019, and I wouldn't be surprised if the absence-of-newer-builds there might screw things up somehow.)

Comment 5

[William Lachance (:wlach)]

(In reply to Daniel Holbert [:dholbert] from comment #4)

(In reply to William Lachance (:wlach) (use needinfo!) (away until Feb 10 2020) from comment #3)

That said, it's still unclear to me why it can't find a pushlog for that range on mozilla-inbound.

Was "mozilla-inbound" a typo here, or are we actually looking at mozilla-inbound here in a way that I'm not seeing? (Note that the logging in comment 0 doesn't mention mozilla-inbound -- it says Getting autoland builds [...])

Sorry, yeah, was a typo-- I meant "autoland"

(FWIW if we are using mozilla-inbound, then it's notable that mozilla-inbound hasn't had any updates since it became unsupported-as-a-push-target on Nov 19, 2019, and I wouldn't be surprised if the absence-of-newer-builds there might screw things up somehow.)

Yup, that was the motivation for bug 1606902

Comment 6

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/mozregression/pull/566

Comment 7

[William Lachance (:wlach)]

The bad pushlog it was using (that only seems to have dates in November) is this one:

https://hg.mozilla.org/integration/autoland/json-pushes?enddate=2020-01-31&startdate=2019-10-01

I'm not sure exactly why this is, but the one for mozilla-central looks completely sensible:

https://hg.mozilla.org/mozilla-central/json-pushes?enddate=2020-01-31&startdate=2019-10-01

I have a pending PR to make mozregression do the right thing, will land tomorrow.

Comment 8

[BugBot [:suhaib / :marco/ :calixte]]

The priority flag is not set for this bug.
:wlach, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 9

[William Lachance (:wlach)]

Fix merged, doing up new releases