1448966 - Can't use release number.

Status: RESOLVED FIXED

(Testing :: mozregression, defect)

Description

[Jean-Yves Avenard [:jya]]

The documentation states:
Bisecting nightlies given firefox releases number
  mozregression --good 33 --bad 34

But that doesn't seem to work with recent release version:

you get:
$   mozregression --good 33 --bad 59
**********
You should use a config file. Please use the --write-config command line flag to help you create one.
**********

 0:00.17 INFO: 59 is not a release, assuming it's a hash...
 0:00.17 INFO: Using date 2014-07-21 for release 33
 0:00.17 INFO: Getting mozilla-inbound builds between 2014-07-21 and 59
 0:00.17 INFO: TaskCluster only keeps builds for one year. Using 2017-03-26 19:59:08.852373 instead of 2014-07-21.
 0:00.90 ERROR: The url 'https://hg.mozilla.org/integration/mozilla-inbound/json-pushes?startdate=2017-03-26&tochange=59' contains no pushlog. Maybe use another range ?

works for 58

59 is a valid revision number.
it should work with 59 (doesn't work with 60 or 61 either)

Comment 1

[William Lachance (:wlach)]

Thanks for the report, unfortunately new releases need to be manually added into mozregression still. I made a change to do exactly this:

https://github.com/mozilla/mozregression/pull/484

Will be releasing a new version shortly.

Comment 2

[William Lachance (:wlach)]

mozregression 2.3.26 released with fix

Comment 3

[Jean-Yves Avenard [:jya]]

Sounds like a painful process...

However, something is still not quite right when you use version 59:
$   mozregression --good 59 --bad 60
**********
You should use a config file. Please use the --write-config command line flag to help you create one.
**********

 0:00.18 INFO: Using date 2018-03-12 for release 60
 0:00.18 INFO: Using date 2018-01-22 for release 59
 0:03.90 INFO: Testing good and bad builds to ensure that they are really good and bad...
 0:03.90 INFO: Downloading build from: https://archive.mozilla.org/pub/firefox/nightly/2018/01/2018-01-22-22-02-31-mozilla-central/firefox-60.0a1.en-US.mac.dmg
===== Downloaded 100% =====
 0:13.72 INFO: Running mozilla-central build for 2018-01-22
 0:26.34 INFO: Launching /private/var/folders/m1/b7gm4k0x7nqdf2jq41m39npm0000gp/T/tmpTZTmty/Firefox Nightly.app/Contents/MacOS/firefox
 0:26.34 INFO: Application command: /private/var/folders/m1/b7gm4k0x7nqdf2jq41m39npm0000gp/T/tmpTZTmty/Firefox Nightly.app/Contents/MacOS/firefox -foreground -profile /var/folders/m1/b7gm4k0x7nqdf2jq41m39npm0000gp/T/tmpwMTiuL.mozrunner
 0:26.35 INFO: application_buildid: 20180122220231
 0:26.35 INFO: application_changeset: 20e194b34185de3009453b87f637daa5f432f74b
 0:26.35 INFO: application_name: Firefox
 0:26.35 INFO: application_repository: https://hg.mozilla.org/mozilla-central
 0:26.35 INFO: application_version: 60.0a1
Was this nightly build good, bad, or broken? (type 'good', 'bad', 'skip', 'retry' or 'exit' and press Enter): bad
 0:42.52 ERROR: Build was expected to be good! The initial good/bad range seems incorrect.


Selecting version 59, downloads version 60.0a1 (which has the problem)

is this intended?

Comment 4

[Jean-Yves Avenard [:jya]]

asking for 58 also gives me 59.0a1 (which in this case is what I want)

Comment 5

[William Lachance (:wlach)]

Yes, this is expected behaviour. Giving you the nightly/integration builds on the branch dates gives you an approximation of what changes went into any given release (minus anything that landed on beta/release afterward). This is imprecise, but going back/forth between release/beta/integration branches would be a whole other layer of complexity.

Comment 6

[William Lachance (:wlach)]

In your case I would just widen the regression range, since it's using binary search it shouldn't add that much time.