1113964 - Nightly bisection skipping steps [post-0.30-a137f940]

Status: RESOLVED DUPLICATE of bug 1109731

(Testing :: mozregression, defect)

Description

[Elbart]

Build: https://github.com/mozilla/mozregression/tree/a137f9405198ddb4ef96c2fc08bac49399aa7941

>=====
mozregression --bits=32 -g 2014-12-09 -b 2014-12-12 --persist c:\temp

 0:03.29 LOG: MainThread Bisector INFO Running nightly for 2014-12-11
 0:03.29 LOG: MainThread Regression Runner INFO Using local file: c:\temp\2014-12-11--mozilla-central--firefox-37.0a1.en-US.win32.zip
Was this nightly build good, bad, or broken? (type 'good', 'bad', 'skip', 'retry', or 'exit' and press Enter): b
 0:11.19 LOG: MainThread Bisector INFO Narrowed nightly regression window from [2014-12-09, 2014-12-12] (3 days) to [2014-12-09, 2014-12-11] (2 days) (~1 steps left)
 0:13.25 LOG: MainThread Bisector INFO Got as far as we can go bisecting nightlies...
 0:13.25 LOG: MainThread Bisector INFO Ensuring we have enough metadata to get a pushlog...
 0:13.85 LOG: MainThread Bisector INFO Last good revision: f1f48ccb2d4e (2014-12-09)
 0:13.85 LOG: MainThread Bisector INFO First bad revision: 0cf461e62ce5 (2014-12-10)
 0:13.85 LOG: MainThread Bisector INFO Pushlog:
>====
>====
mozregression --bits=32 -g 2014-12-08 -b 2014-12-12
 0:04.08 LOG: MainThread Bisector INFO Running nightly for 2014-12-10
 0:04.08 LOG: MainThread Regression Runner INFO Downloading build from: http://download.cdn.mozilla.net/pub/mozilla.org/firefox/nightly/2014/12/2014-12-10-03-02-07-mozilla-central/firefox-37.0a1.en-US.win32.zip
===== Downloaded 100% =====
Was this nightly build good, bad, or broken? (type 'good', 'bad', 'skip', 'retry', or 'exit' and press Enter): b
Retrying to remove "c:\users\user\appdata\local\temp\tmp2tpljc". Reason: Access is denied (13)
 2:45.68 LOG: MainThread Bisector INFO Narrowed nightly regression window from [2014-12-08, 2014-12-12] (4 days) to [2014-12-08, 2014-12-10] (2 days) (~1 steps left)
 2:47.38 LOG: MainThread Bisector INFO Got as far as we can go bisecting nightlies...
 2:47.38 LOG: MainThread Bisector INFO Ensuring we have enough metadata to get a pushlog...
 2:48.21 LOG: MainThread Bisector INFO Last good revision: 035a951fc24a (2014-12-08)
 2:48.21 LOG: MainThread Bisector INFO First bad revision: 5b01216f97f8 (2014-12-09)
>====
12-10 tested bad, 12-09 is apparently marked as bad, too.
And that "Retrying to remove"-error is occasionally popping up, too.





>====
mozregression --bits=32 -g 2014-12-07 -b 2014-12-12
 0:02.53 LOG: MainThread Bisector INFO Running nightly for 2014-12-10
 0:02.53 LOG: MainThread Regression Runner INFO Downloading build from: http://download.cdn.mozilla.net/pub/mozilla.org/firefox/nightly/2014/12/2014-12-10-03-02-07-mozilla-central/firefox-37.0a1.en-US.win32.zip
===== Downloaded 100% =====
Was this nightly build good, bad, or broken? (type 'good', 'bad', 'skip', 'retry', or 'exit' and press Enter): b
 0:28.45 LOG: MainThread Bisector INFO Narrowed nightly regression window from [2014-12-07, 2014-12-12] (5 days) to [2014-12-07, 2014-12-10] (3 days) (~1 steps left)
 0:29.82 LOG: MainThread Bisector INFO Running nightly for 2014-12-08
 0:29.82 LOG: MainThread Regression Runner INFO Downloading build from: http://download.cdn.mozilla.net/pub/mozilla.org/firefox/nightly/2014/12/2014-12-08-03-02-02-mozilla-central/firefox-37.0a1.en-US.win32.zip
===== Downloaded 100% =====
Was this nightly build good, bad, or broken? (type 'good', 'bad', 'skip', 'retry', or 'exit' and press Enter): g
 0:49.82 LOG: MainThread Bisector INFO Narrowed nightly regression window from [2014-12-07, 2014-12-09] (2 days) to [2014-12-08, 2014-12-09] (1 days) (~0 steps left)
 0:49.82 LOG: MainThread Bisector INFO Got as far as we can go bisecting nightlies...
 0:49.82 LOG: MainThread Bisector INFO Ensuring we have enough metadata to get a pushlog...
 0:49.83 LOG: MainThread Bisector INFO Last good revision: 035a951fc24a (2014-12-08)
 0:49.83 LOG: MainThread Bisector INFO First bad revision: 5b01216f97f8 (2014-12-09)
 0:49.83 LOG: MainThread Bisector INFO Pushlog:
>====
It goes from 12-07/12-10 to 12-08/12-09 after judging 12-08 as good.
The "first bad revision" in the logs are actually of the following days.

Comment 1

[Julien Pagès (:parkouss)]

Yes, thanks Elbart.

I should have said that before, it is probably fixed with my pull request https://github.com/mozilla/mozregression/pull/153 (waiting to be reviewed in Bug 1109731).

Sorry for this mess. This happens for Nightlies now (0.30+) because I refactored the logic of bisection, using the same algorithm for nightly and inbound. Son one bug in this was reflected in the two.

You can use the code here until this is merged in:

https://github.com/parkouss/mozregression/tree/fix_missing_inbound

In the mean time, if you have some time, I would really appreciate if you can test this! Please note one thing, that I noted:

The printed remaining steps are working differently than in inbound. I mean when there is three days:

[2014-12-09, ?, 2014-12-11] it print (2 days). This is wrong I think since we have 3 days in the range. Or should we count something else ?


The "retrying to remove" is not directly a mozregression thing. It probably comes from mozfile.remove calls.

Comment 2

[Elbart]

(In reply to Julien Pagès from comment #1)
> Yes, thanks Elbart.
> 
> I should have said that before, it is probably fixed with my pull request
> https://github.com/mozilla/mozregression/pull/153 (waiting to be reviewed in
> Bug 1109731).

Oh, ok.

> In the mean time, if you have some time, I would really appreciate if you
> can test this! Please note one thing, that I noted:

Yes, that patch also fixes this bug. Going to dupe this to bug 1109731.

> The printed remaining steps are working differently than in inbound. I mean
> when there is three days:
> 
> [2014-12-09, ?, 2014-12-11] it print (2 days). This is wrong I think since
> we have 3 days in the range. Or should we count something else ?

I'd say the difference between the build-times is pretty much two days.
The log also says "regression window", and the window isn't three days long, only three builds.

> The "retrying to remove" is not directly a mozregression thing. It probably
> comes from mozfile.remove calls.

I just saw it in a 0.30-log, too, so it's not new.
Maybe some indexer blocking the folder, no biggie.

Comment 3

[Julien Pagès (:parkouss)]

(In reply to Elbart from comment #2)
> (In reply to Julien Pagès from comment #1)
> > [2014-12-09, ?, 2014-12-11] it print (2 days). This is wrong I think since
> > we have 3 days in the range. Or should we count something else ?
> 
> I'd say the difference between the build-times is pretty much two days.
> The log also says "regression window", and the window isn't three days long,
> only three builds.

Hmm... Three days and three builds are pretty much the same on nightly bisection. That bother me to not have the same progress indicator behaviour on nightlies and inbound. Futhermore, the steps left are impacted, since we don't use the same number to calculate this when the number of remaining builds is the same...

Well, anyway this is another matter. We may think about it with Bug 1084950 for example. I will create links in the bug to not forget about it.

Comment 4

[Julien Pagès (:parkouss)]

Oups, sorry, this is a duplicate;