Open Bug 1615667 Opened 6 years ago Updated 3 years ago

mozregression buries its actionable output behind pages of warning-spew, for ranges that are too old for intermediate taskcluster builds

Categories

(Testing :: mozregression, defect, P3)

Product:
Testing
Component:
mozregression
Version:
Version 3
Type:
defect

Tracking

(Not tracked)

People

(Reporter: dholbert, Unassigned)

Details

STR:

  1. Run: mozregression --good 2017-11-29 --bad 2017-11-30

ACTUAL RESULTS:
These four lines (which are the most useful thing it can give me):

0:00.88 INFO: Got as far as we can go bisecting nightlies...
0:00.88 INFO: Last good revision: 91fc3a79606bdc7fb43fef12eff7e65b5b84c00e (2017-11-29)
0:00.88 INFO: First bad revision: ea747bb2ffb77d1fd62b5fa6217cbee15b73d31f (2017-11-30)
0:00.88 INFO: Pushlog:
https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=91fc3a79606bdc7fb43fef12eff7e65b5b84c00e&tochange=ea747bb2ffb77d1fd62b5fa6217cbee15b73d31f

...followed by 55 lines that look like this (abbreviated here) before mozregression quits:

0:00.88 INFO: Switching bisection method to taskcluster
0:00.88 INFO: Getting mozilla-central builds between 91fc3a79606bdc7fb43fef12eff7e65b5b84c00e and ea747bb2ffb77d1fd62b5fa6217cbee15b73d31f
0:03.70 WARNING: Skipping build 91fc3a79606b: Unable to find build info using the taskcluster route 'gecko.v2.mozilla-central.shippable.revision.91fc3a79606bdc7fb43fef12eff7e65b5b84c00e.firefox.linux64-opt'
0:04.06 WARNING: Skipping build ea747bb2ffb7: Unable to find build info using the taskcluster route 'gecko.v2.mozilla-central.shippable.revision.ea747bb2ffb77d1fd62b5fa6217cbee15b73d31f.firefox.linux64-opt'
[...]
0:20.39 WARNING: Skipping build 3e14872b31a7: Unable to find build info using the taskcluster route 'gecko.v2.mozilla-central.shippable.revision.3e14872b31a7b1b207605d09b78fbaaf21f1bba7.firefox.linux64-opt'
0:20.43 CRITICAL: First build 91fc3a79606b is missing, but mozregression can't find a build before - so it is excluded, but it could contain the regression!
0:22.92 WARNING: Skipping build a21f4e2ce518: Unable to find build info using the taskcluster route 'gecko.v2.mozilla-
[...]
0:44.19 WARNING: Skipping build ba283baf4e98: Unable to find build info using the taskcluster route 'gecko.v2.mozilla-central.shippable.revision.ba283baf4e98aa3a5f45a17981077b98697aa73a.firefox.linux64-opt'
0:44.20 INFO: There are no build artifacts on inbound for these changesets (they are probably too old).

EXPECTED RESULTS:

  • The useful/actionable output should not be buried above 55 lines of warning-spam. Either we should condense the warning spam, or we should reiterate the last-actionable-line at the bottom (e.g. "The best range we got was: Last Good... First bad... Pushlog ..." right before we quit)

  • There probably (?) shouldn't be the CRITICAL warning that's quoted in bold above -- I'm not sure what that's about and why it's assessed as CRITICAL. As far as I can tell, this is just a case where we have nightly builds available and we don't have any intermediate builds, and there's nothing "critical" about the absence of intermediate builds.

All of this sounds reasonable, for this case in particular we could probably also avoid bothering searching for taskcluster builds >6 months old, since they are auto-expired.

Priority: -- → P3

Similar to bug 1610427

Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.