mozregression -g X checks X+1 build first
Categories
(Testing :: mozregression, defect, P3)
Tracking
(Not tracked)
People
(Reporter: valentin, Unassigned)
References
Details
~/mozilla-unified$ ./mach mozregression -g 118
-1:60.00 INFO: No 'bad' option specified, using 2024-01-26
1:38.68 INFO: Using date 2023-08-28 for release 118
1:40.86 INFO: Testing good and bad builds to ensure that they are really good and bad...
1:40.86 INFO: Downloading build from: https://archive.mozilla.org/pub/firefox/nightly/2023/08/2023-08-28-21-21-20-mozilla-central/firefox-119.0a1.en-US.linux-x86_64.tar.bz2
===== Downloaded 100% =====
1:47.25 INFO: Running mozilla-central build for 2023-08-28
1:53.09 INFO: Launching /tmp/tmpvxuo0oxx/firefox/firefox
1:53.09 INFO: Application command: /tmp/tmpvxuo0oxx/firefox/firefox -profile /tmp/tmpi00ykiv2.mozrunner
1:53.09 INFO: application_buildid: 20230828212120
1:53.09 INFO: application_changeset: 18ac70a5128d520641cced852fc0059536c52713
1:53.09 INFO: application_name: Firefox
1:53.09 INFO: application_repository: https://hg.mozilla.org/mozilla-central
1:53.09 INFO: application_version: 119.0a1
Was this nightly build good, bad, or broken? (type 'good', 'bad', 'skip', 'retry' or 'exit' and press Enter): b
2:26.42 ERROR: Build was expected to be good! The initial good/bad range seems incorrect.
My expectation was that it would be testing the last nightly build for 118, not the first nightly build for 119.
Is this correct?
|
||
Comment 1•2 years ago
|
||
The severity field is not set for this bug.
:zeid, could you have a look please?
For more information, please visit BugBot documentation.
|
||
Comment 2•1 year ago
|
||
This is happening because the "good" version parameter is converted to a date using the FIREFOX_NIGHTLY_<version>_END tag, which corresponds to the last nightly build date of that version, but also the first day of the next version. It looks like mozregression is picking the latter version instead of the former when fetching from the archive. We could fix this by explicitly picking the earlier version, which would be equivalent to the last nightly build for that version.
|
|
||
Comment 3•1 year ago
|
||
Adding an example to illustrate, 118 would yield August 28, 2023 when converted (i.e., losing the time of day detail) which on archive.mozilla.org has two entries:
- https://archive.mozilla.org/pub/firefox/nightly/2023/08/2023-08-28-09-44-42-mozilla-central/ (118.0a1)
- https://archive.mozilla.org/pub/firefox/nightly/2023/08/2023-08-28-21-21-20-mozilla-central/ (119.0a1)
Most other days would have two builds of the same version.
Description
•