Closed Bug 1014140 Opened 11 years ago Closed 10 years ago

First inbound-try reuses last downloaded Nightly-build

Categories

(Testing :: mozregression, defect)

Product:
Testing
Component:
mozregression
Platform:
x86_64
Windows 7
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED WORKSFORME

People

(Reporter: elbart, Unassigned)

Details

Attachments

(2 files)

wrong inbound build.png
81.90 KB, image/png
Details
ubuntu_32bit.txt
2.94 KB, text/plain
Details
Attached image wrong inbound build.png —
+++ This bug was initially created as a clone of Bug #996811 +++ Maybe this is related to bug 996811: Affected: mozregression 0.16 and 0.18 The first step of the inbound-bisection is reusing the last downloaded Nightly-build, which of course doesn't have the same revision as the one which should be tested. $ mozregression --bits=32 -g 2014-04-20 -b 2014-04-21 Got as far as we can go bisecting nightlies... Ensuring we have enough metadata to get a pushlog... Downloading build from: http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2 014/04/2014-04-20-03-02-02-mozilla-central/firefox-31.0a1.en-US.win32.zip ===== Downloaded 100% ===== Installing nightly Downloading build from: http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2 014/04/2014-04-21-03-02-02-mozilla-central/firefox-31.0a1.en-US.win32.zip ===== Downloaded 100% ===== Installing nightly Last good revision: 53a6c96cea62 (2014-04-20) First bad revision: 5010b38abf18 (2014-04-21) Pushlog: https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=53a6c96cea62&tocha nge=5010b38abf18 ... attempting to bisect inbound builds (starting from previous day, to make sur e no inbound revision is missed) Downloading build from: http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2 014/04/2014-04-19-03-02-04-mozilla-central/firefox-31.0a1.en-US.win32.zip ===== Downloaded 100% ===== Installing nightly Getting inbound builds between 582b2d81ebe1 and 5010b38abf18 Testing inbound build with timestamp 1397930968, revision 443dcdf8eed26aa7a2134d 970c003d2bd7b85903 Using local file: firefox-31.0a1.en-US.win32.zip Installing nightly Starting nightly (revision: 582b2d81ebe1) Was this inbound build good, bad, or broken? (type 'good', 'bad', 'skip', 'retry ', or 'exit' and press Enter): Testing in 0.15 didn't work, because it crashed with the following error: $ mozregression --bits=32 -g 2014-04-20 -b 2014-04-21 Got as far as we can go bisecting nightlies... Ensuring we have enough metadata to get a pushlog... Downloading build from: http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2 014/04/2014-04-20-03-02-02-mozilla-central/firefox-31.0a1.en-US.win32.zip ===== Downloaded 100% ===== Installing nightly Downloading build from: http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2 014/04/2014-04-21-03-02-02-mozilla-central/firefox-31.0a1.en-US.win32.zip ===== Downloaded 100% ===== Installing nightly Last good revision: 53a6c96cea62 (2014-04-20) First bad revision: 5010b38abf18 (2014-04-21) Pushlog: https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=53a6c96cea62&tocha nge=5010b38abf18 ... attempting to bisect inbound builds (starting from previous day, to make sur e no inbound revision is missed) Downloading build from: http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2 014/04/2014-04-19-03-02-04-mozilla-central/firefox-31.0a1.en-US.win32.zip ===== Downloaded 100% ===== Installing nightly Getting inbound builds between 582b2d81ebe1 and 5010b38abf18 bits: 32 Traceback (most recent call last): File "c:\mozilla-build\python\Scripts\mozregression-script.py", line 8, in <mo dule> load_entry_point('mozregression==0.15', 'console_scripts', 'mozregression')( ) File "build\bdist.win32\egg\mozregression\regression.py", line 277, in cli File "build\bdist.win32\egg\mozregression\regression.py", line 160, in bisect_ nightlies File "build\bdist.win32\egg\mozregression\regression.py", line 99, in bisect_i nbound File "build\bdist.win32\egg\mozregression\inboundfinder.py", line 52, in getIn boundRevisions File "build\bdist.win32\egg\mozregression\inboundfinder.py", line 51, in <lamb da> ValueError: invalid literal for int() with base 10: 'latest' "latest"-folder again? Workaround: Retrying once downloads the proper build, but you'd first have to notice that you're running the wrong build...
Can you confirm this?
At least on linux64, I'm not getting this behaviour with mozregression 0.22. Quite a bit of the logic around this has been changed. Can you confirm that you're still seeing this? If so I'll investigate.
Flags: needinfo?(elbart)
Yes, still happening with 0.22 on both of my Win7-machines. It's working with --persist, though. >============================================================================== User@USER ~ $ easy_install -U mozregression Searching for mozregression Reading http://pypi.python.org/simple/mozregression/ Best match: mozregression 0.22 Processing mozregression-0.22-py2.7.egg mozregression 0.22 is already the active version in easy-install.pth Installing mozregression-script.py script to c:\mozilla-build\python\Scripts Installing mozregression.exe script to c:\mozilla-build\python\Scripts Installing mozregression.exe.manifest script to c:\mozilla-build\python\Scripts Installing moznightly-script.py script to c:\mozilla-build\python\Scripts Installing moznightly.exe script to c:\mozilla-build\python\Scripts Installing moznightly.exe.manifest script to c:\mozilla-build\python\Scripts Using c:\mozilla-build\python\lib\site-packages\mozregression-0.22-py2.7.egg Processing dependencies for mozregression Finished processing dependencies for mozregression User@USER ~ $ mozregression --bits=32 -g 2014-07-17 -b 2014-07-19 Running nightly for 2014-07-18 Downloading build from: http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2 014/07/2014-07-18-03-02-02-mozilla-central/firefox-33.0a1.en-US.win32.zip ===== Downloaded 100% ===== Installing nightly Starting nightly (revision: 99f694d1b50c) Was this nightly build good, bad, or broken? (type 'good', 'bad', 'skip', 'retry ', or 'exit' and press Enter): g Got as far as we can go bisecting nightlies... Ensuring we have enough metadata to get a pushlog... Getting http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2014/07/2014-07-1 9-03-02-03-mozilla-central/firefox-33.0a1.en-US.win32.txt Last good revision: 99f694d1b50c (2014-07-18) First bad revision: 35f3fa435d2c (2014-07-19) Pushlog: https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=99f694d1b50c&tocha nge=35f3fa435d2c Ensuring we have enough metadata to get a pushlog... Last good revision: 99f694d1b50c (2014-07-18) First bad revision: 35f3fa435d2c (2014-07-19) Pushlog: https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=99f694d1b50c&tocha nge=35f3fa435d2c ... attempting to bisect inbound builds (starting from previous day, to make sur e no inbound revision is missed) Getting http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2014/07/2014-07-1 7-03-03-39-mozilla-central/firefox-33.0a1.en-US.win32.txt Getting inbound builds between a74600665875 and 35f3fa435d2c Testing inbound build with timestamp 1405677813, revision c1ddceb19abc60b28624e6 efe7099e6715456de9 Using local file: firefox-33.0a1.en-US.win32.zip Installing nightly Starting nightly (revision: 99f694d1b50c) Was this inbound build good, bad, or broken? (type 'good', 'bad', 'skip', 'retry ', or 'exit' and press Enter): exit Newest known good inbound revision: a74600665875 Oldest known bad inbound revision: 35f3fa435d2c To resume, run: mozregression --good-rev=a74600665875 --bad-rev=35f3fa435d2c --app=firefox --bit s=32 User@USER ~ $ mozregression --bits=32 -g 2014-07-17 -b 2014-07-19 --persist /c/temp Running nightly for 2014-07-18 Downloading build from: http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2 014/07/2014-07-18-03-02-02-mozilla-central/firefox-33.0a1.en-US.win32.zip ===== Downloaded 100% ===== Installing nightly Starting nightly (revision: 99f694d1b50c) Was this nightly build good, bad, or broken? (type 'good', 'bad', 'skip', 'retry ', or 'exit' and press Enter): g Got as far as we can go bisecting nightlies... Ensuring we have enough metadata to get a pushlog... Getting http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2014/07/2014-07-1 9-03-02-03-mozilla-central/firefox-33.0a1.en-US.win32.txt Last good revision: 99f694d1b50c (2014-07-18) First bad revision: 35f3fa435d2c (2014-07-19) Pushlog: https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=99f694d1b50c&tocha nge=35f3fa435d2c Ensuring we have enough metadata to get a pushlog... Last good revision: 99f694d1b50c (2014-07-18) First bad revision: 35f3fa435d2c (2014-07-19) Pushlog: https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=99f694d1b50c&tocha nge=35f3fa435d2c ... attempting to bisect inbound builds (starting from previous day, to make sur e no inbound revision is missed) Getting http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2014/07/2014-07-1 7-03-03-39-mozilla-central/firefox-33.0a1.en-US.win32.txt Getting inbound builds between a74600665875 and 35f3fa435d2c Testing inbound build with timestamp 1405677813, revision c1ddceb19abc60b28624e6 efe7099e6715456de9 Downloading build from: http://inbound-archive.pub.build.mozilla.org/pub/mozilla .org/firefox/tinderbox-builds/mozilla-inbound-win32/1405677813/firefox-33.0a1.en -US.win32.zip ===== Downloaded 100% ===== Installing nightly Starting nightly (revision: c1ddceb19abc) Was this inbound build good, bad, or broken? (type 'good', 'bad', 'skip', 'retry ', or 'exit' and press Enter): exit Newest known good inbound revision: a74600665875 Oldest known bad inbound revision: 35f3fa435d2c To resume, run: mozregression --good-rev=a74600665875 --bad-rev=35f3fa435d2c --app=firefox --bit s=32 --persist=c:/temp
Flags: needinfo?(elbart)
This is with 0.24: $ mozregression --bits=32 -g 2014-07-22 -b 2014-07-26 Running nightly for 2014-07-24 Downloading build from: http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2 014/07/2014-07-24-03-02-01-mozilla-central/firefox-34.0a1.en-US.win32.zip ===== Downloaded 100% ===== Installing nightly Starting nightly (revision: a91ec42d6a9c) Was this nightly build good, bad, or broken? (type 'good', 'bad', 'skip', 'retry ', or 'exit' and press Enter): b Running nightly for 2014-07-23 Downloading build from: http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2 014/07/2014-07-23-03-02-02-mozilla-central/firefox-34.0a1.en-US.win32.zip ===== Downloaded 100% ===== Installing nightly Starting nightly (revision: 82df3654cd80) Was this nightly build good, bad, or broken? (type 'good', 'bad', 'skip', 'retry ', or 'exit' and press Enter): g Got as far as we can go bisecting nightlies... Ensuring we have enough metadata to get a pushlog... Last good revision: 82df3654cd80 (2014-07-23) First bad revision: a91ec42d6a9c (2014-07-24) Pushlog: https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=82df3654cd80&tocha nge=a91ec42d6a9c ... attempting to bisect inbound builds (starting from previous week, to make su re no inbound revision is missed) Getting http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2014/07/2014-07-1 6-03-02-02-mozilla-central/firefox-33.0a1.en-US.win32.txt Getting inbound builds between 869971ad9fd6 and a91ec42d6a9c Testing inbound build with timestamp 1405776260, revision 9350909a34017d71dc947d 4ac118ad3527203227 Downloading build from: http://inbound-archive.pub.build.mozilla.org/pub/mozilla .org/firefox/tinderbox-builds/mozilla-inbound-win32/1405776260/firefox-33.0a1.en -US.win32.zip ===== Downloaded 100% ===== Installing nightly Starting nightly (revision: 9350909a3401) Was this inbound build good, bad, or broken? (type 'good', 'bad', 'skip', 'retry ', or 'exit' and press Enter): g Testing inbound build with timestamp 1406036849, revision 3bdb961fd6d1ef332b06e6 ff9268c0ab56347003 >Using local file: firefox-34.0a1.en-US.win32.zip Installing nightly Starting nightly (revision: 82df3654cd80) Was this inbound build good, bad, or broken? (type 'good', 'bad', 'skip', 'retry ', or 'exit' and press Enter): g Testing inbound build with timestamp 1406092007, revision 601963f474d8e98e13b7cb 1778710590eaa567fb Downloading build from: http://inbound-archive.pub.build.mozilla.org/pub/mozilla .org/firefox/tinderbox-builds/mozilla-inbound-win32/1406092007/firefox-34.0a1.en -US.win32.zip This time it was the second step, probably because of the longer inbound-range (7 days instead of 2), which resulted in a different filename for the first inbound-step.
Flags: needinfo?(wlachance)
Ok, I'll try and take a look at this today!
Hey, first of all, really sorry it took me so long to look into this. At least on linux64, the downloaded build revision matches that of the one that we're supposedly testing for the test case you mention. I'll test later on win32.
Flags: needinfo?(wlachance)
Reminder to self to look into Win32 on mozregression 0.24.
Flags: needinfo?(wlachance)
Ok, I *can* actually reproduce this. Only on Win32 strangely enough. tl;dr: in the above dump we should be testing revision 3bdb961fd6d1ef33 but instead we're testing 82df3654cd80. Most odd, will look into it.
Attached file ubuntu_32bit.txt —
I can reproduce it on Ubuntu 32bit too, though.
When exiting at this stage, the following error is shown: Was this inbound build good, bad, or broken? (type 'good', 'bad', 'skip', 'retry', or 'exit' and press Enter): exit 13:25.12 LOG: MainThread Bisector INFO Newest known good inbound revision: 818f353b7aa6 13:25.12 LOG: MainThread Bisector INFO Oldest known bad inbound revision: 229801d17f7e 13:25.12 LOG: MainThread Bisector INFO To resume, run: 13:25.12 LOG: MainThread Regression Runner INFO mozregression --good-rev=818f353b7aa6 --bad-rev=229801d17f7e --app=firefox --inbound-branch=mozilla-inbound --bits=32 Exception OSError: (2, 'No such file or directory', 'firefox-35.0a1.en-US.linux-i686.tar.bz2') in <bound method FirefoxInbound.cleanup of <mozregression.runinbound.FirefoxInbound object at 0xb6940cac>> ignored blah@ubuntu:~/Desktop/mozreg$
I'm not sure this is still a bug on master. We have done a lot of work on mozregression that may have resolved this I think. Well we'll see this once William will be able to look at it in details.
Ok, I tried to reproduce this bug with the following command line (on win 7): mozregression --bits=32 -g 2015-01-20 -b 2015-01-22 And I can't reproduce it. In fact, in the log we can see in the previous commands, persistent files were named like "firefox-34.0a1.en-US.win32.zip". Now we are prefixing these filenames like this: 2015-01-21--mozilla-central--firefox-38.0a1.en-US.win32.zip (nightly) 1421752059--mozilla-inbound--firefox-38.0a1.en-US.win32.zip (inbound) so we use: {YYYY-MM-DD}--{repo}--{filename} (nightly) {timestamp}--{inbound-branch}--{filename} This can be seen in the code here: https://github.com/mozilla/mozregression/blob/7a32b36ceca758ba95eced2a5898bcf9dec8afe5/mozregression/test_runner.py#L42 https://github.com/mozilla/mozregression/blob/7a32b36ceca758ba95eced2a5898bcf9dec8afe5/mozregression/test_runner.py#L45 So even if repo and filename are equivalent, there is no chance that {YYYY-MM-DD} could be equal to {timestamp}. There is no possible collision between nightlies and inbound persistent files, so I think the bug here is resolved. I can't remember since when we use this prefix format. I will look at the github history and try to find that.
So, I discussed with wlach, we agree to close this bug. I think it is not valid anymore and I can't reproduce it.
Status: UNCONFIRMED → RESOLVED
Closed: 10 years ago
Flags: needinfo?(wlachance)
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: