Closed Bug 510219 Opened 15 years ago Closed 14 years ago

random orange in xpcshell/tests/test_browser_places/unit/test_browserGlue_smartBookmarks.js : 2 == 3

Categories

(Firefox :: Bookmarks & History, defect)

Product:
Firefox
Component:
Bookmarks & History
Platform:
x86
All
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED FIXED
Milestone:
Firefox 4.0b9

People

(Reporter: dbaron, Unassigned)

References

Details

(Keywords: intermittent-failure)

Attachments

(1 file)

listen to console
4.83 KB, patch
sdwilsh
: review+
Details | Diff | Splinter Review 
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1250156191.1250157740.22617.gz
Linux mozilla-central test everythingelse on 2009/08/13 02:36:31  

had the following random orange:

TEST-UNEXPECTED-FAIL | /builds/moz2_slave/mozilla-central-linux-unittest-everythingelse/build/xpcshell/tests/test_browser_places/unit/test_browserGlue_smartBookmarks.js | test failed (with xpcshell return code: 0), see following log:

...

TEST-UNEXPECTED-FAIL | /builds/moz2_slave/mozilla-central-linux-unittest-everythingelse/build/xpcshell/tests/test_browser_places/unit/test_browserGlue_smartBookmarks.js | 2 == 3 - See following stack:
JS frame :: /builds/moz2_slave/mozilla-central-linux-unittest-everythingelse/build/xpcshell/head.js :: do_throw :: line 197
JS frame :: /builds/moz2_slave/mozilla-central-linux-unittest-everythingelse/build/xpcshell/head.js :: do_check_eq :: line 227
JS frame :: /builds/moz2_slave/mozilla-central-linux-unittest-everythingelse/build/xpcshell/tests/test_browser_places/unit/test_browserGlue_smartBookmarks.js :: anonymous :: line 87
JS frame :: /builds/moz2_slave/mozilla-central-linux-unittest-everythingelse/build/xpcshell/tests/test_browser_places/unit/test_browserGlue_smartBookmarks.js :: next_test :: line 213
JS frame :: /builds/moz2_slave/mozilla-central-linux-unittest-everythingelse/build/xpcshell/tests/test_browser_places/unit/test_browserGlue_smartBookmarks.js :: run_test :: line 222
JS frame :: /builds/moz2_slave/mozilla-central-linux-unittest-everythingelse/build/xpcshell/head.js :: _execute_test :: line 125
JS frame :: -e :: <TOP_LEVEL> :: line 1
TEST-INFO | (xpcshell/head.js) | exiting test
Blocks: 438871
Whiteboard: [oange]
Whiteboard: [oange] → [orange]
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1255627046.1255629337.22160.gz
WINNT 5.2 mozilla-central test everythingelse on 2009/10/15 10:17:26

Note: the second failure in this log has "false == true" instead of "2 == 3".  The first failure matches the one in comment 0, though.
OS: Linux → All
Summary: random orange in xpcshell/tests/test_browser_places/unit/test_browserGlue_smartBookmarks.js : 2 == 3 → random orange in xpcshell/tests/test_browser_places/unit/test_browserGlue_smartBookmarks.js : "2 == 3" or "false == true"
Daniel, i don't understand your report, the failure you reported is in test_browserGlue_prefs.js while this bug is about test_browserGlue_smartBookmarks.js

you should have filed a separate report since the issue looks different.
Oops! Good catch -- sorry about that.  My brain must've only been doing prefix-matching on the test names, or something. I've now filed bug Bug 522735 for the log in comment 4.
OS: All → Linux
Summary: random orange in xpcshell/tests/test_browser_places/unit/test_browserGlue_smartBookmarks.js : "2 == 3" or "false == true" → random orange in xpcshell/tests/test_browser_places/unit/test_browserGlue_smartBookmarks.js : 2 == 3
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1256540664.1256542693.2589.gz
OS X 10.5.2 mozilla-central test opt everythingelse on 2009/10/26 00:04:24
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1256578348.1256580845.27853.gz
WINNT 5.2 mozilla-central test everythingelse on 2009/10/26 10:32:28
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1257157307.1257160093.24216.gz
WINNT 5.2 mozilla-central test everythingelse on 2009/11/02 02:21:47
I don't suppose this test could be made any more verbose about where it's failing, in the error message itself, or could maybe be broken into multiple separate tests?

I was just about to claim that http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1258065623.1258068454.9885.gz was this when I noticed the 4 == 3, and eventually realized that this bug is about a failure in "All smart bookmarks are created if smart bookmarks version is 0" and that at least once I've wrongly claimed that a failure in "An explicitly removed smart bookmark should not be recreated" was this, and was about to claim that a failure in "An existing smart bookmark is replaced when version changes" was as well.
And just to prove it's not just me, while I was typing that it got starred as being this bug :)
it's interesting that this still fails, it should not.
but probably all failures have a common cause and it's ok to annotate all failures for this bug here.
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1258744850.1258747416.30723.gz
OS X 10.5.2 mozilla-central opt test everythingelse on 2009/11/20 11:20:50
Bug 451915 - move Firefox/Places bugs to Firefox/Bookmarks and History. Remove all bugspam from this move by filtering for the string "places-to-b-and-h".

In Thunderbird 3.0b, you do that as follows:
Tools | Message Filters
Make sure the correct account is selected. Click "New"
Conditions: Body   contains   places-to-b-and-h
Change the action to "Delete Message".
Select "Manually Run" from the dropdown at the top.
Click OK.

Select the filter in the list, make sure "Inbox" is selected at the bottom, and click "Run Now". This should delete all the bugspam. You can then delete the filter.

Gerv
Component: Places → Bookmarks & History
QA Contact: places → bookmarks
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1259691077.1259693834.22065.gz
OS X 10.5.2 mozilla-central opt test everythingelse on 2009/12/01 10:11:17
OS: Linux → All
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1260597068.1260599811.23220.gz
OS X 10.5.2 mozilla-central test everythingelse on 2009/12/11 21:51:08
s: moz2-darwin9-slave02
TEST-UNEXPECTED-FAIL | /builds/slave/mozilla-central-macosx-unittest-everythingelse/build/xpcshell/tests/test_browser_places/unit/test_browserGlue_smartBookmarks.js | test failed (with xpcshell return code: 0), see following log:
TEST-UNEXPECTED-FAIL | /builds/slave/mozilla-central-macosx-unittest-everythingelse/build/xpcshell/tests/test_browser_places/unit/test_browserGlue_smartBookmarks.js | 4 == 3 - See following stack:
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1260626796.1260629156.29436.gz#err1
OS X 10.5.2 mozilla-central opt test everythingelse on 2009/12/12 06:06:36
s: moz2-darwin9-slave09
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1260733047.1260737205.14036.gz
OS X 10.5.2 mozilla-central test everythingelse on 2009/12/13 11:37:27
s: moz2-darwin9-slave13
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1261519407.1261522760.23688.gz
OS X 10.5.2 mozilla-central test everythingelse on 2009/12/22 14:03:27
s: moz2-darwin9-slave15

(the run before that push)
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1261543890.1261546484.29628.gz
OS X 10.5.2 mozilla-central opt test everythingelse on 2009/12/22 20:51:30
s: moz2-darwin9-slave19
these failures are crazy nonsense. btw this was failing even if workarounded for sync, so there must be another reason.
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1261977159.1261979711.19180.gz#err1
OS X 10.5.2 mozilla-central opt test everythingelse on 2009/12/27 21:12:39  
s: moz2-darwin9-slave19
OS X 10.5.2 mozilla-central test everythingelse [testfailed] Started 05:16, finished 06:03
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1262092588.1262095322.27854.gz&fulltext=1
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1262150853.1262153210.23339.gz
OS X 10.5.2 mozilla-central opt test everythingelse on 2009/12/29 21:27:33
s: moz2-darwin9-slave11
OS X 10.5.2 mozilla-central opt test everythingelse [testfailed] Started 04:21, finished 05:06
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1262175699.1262178325.13083.gz&fulltext=1
WINNT 5.2 mozilla-central debug test everythingelse [testfailed] Started 07:30, finished 09:32
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1262187051.1262194284.960.gz&fulltext=1
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1262194053.1262199620.27484.gz
WINNT 5.2 mozilla-central debug test everythingelse on 2009/12/30 09:27:33
s: win32-slave37
TEST-UNEXPECTED-FAIL | e:\builds\moz2_slave\mozilla-central-win32-debug-unittest-everythingelse\build\xpcshell\tests\test_browser_places\unit\test_browserGlue_smartBookmarks.js | test failed (with xpcshell return code: 0), see following log:
TEST-UNEXPECTED-FAIL | e:/builds/moz2_slave/mozilla-central-win32-debug-unittest-everythingelse/build/xpcshell/tests/test_browser_places/unit/test_browserGlue_smartBookmarks.js | 4 == 6 - See following stack:

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1262196034.1262201357.13878.gz
WINNT 5.2 mozilla-central debug test everythingelse on 2009/12/30 10:00:34
s: win32-slave41
TEST-UNEXPECTED-FAIL | e:\builds\moz2_slave\mozilla-central-win32-debug-unittest-everythingelse\build\xpcshell\tests\test_browser_places\unit\test_browserGlue_smartBookmarks.js | test failed (with xpcshell return code: 0), see following log:
TEST-UNEXPECTED-FAIL | e:/builds/moz2_slave/mozilla-central-win32-debug-unittest-everythingelse/build/xpcshell/tests/test_browser_places/unit/test_browserGlue_smartBookmarks.js | 4 == 6 - See following stack:
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1262211015.1262218500.11505.gz
WINNT 5.2 mozilla-central debug test everythingelse on 2009/12/30 14:10:15
s: win32-slave01
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1262222901.1262229949.6450.gz
WINNT 5.2 mozilla-central debug test everythingelse on 2009/12/30 17:28:21
s: win32-slave23
Looking at these test failures, the only thing that makes sense to me is a sync issue, although I'm not even sure of that.  I've pushed a change that disables sync for this test:
http://hg.mozilla.org/mozilla-central/rev/03d6ae8962ce
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1262267084.1262274271.17359.gz
WINNT 5.2 mozilla-central debug test everythingelse on 2009/12/31 05:44:44  

(before commit in previous comment)
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1262271847.1262278819.3155.gz
WINNT 5.2 mozilla-central debug test everythingelse on 2009/12/31 07:04:07
s: win32-slave17
TEST-UNEXPECTED-FAIL | e:\builds\moz2_slave\mozilla-central-win32-debug-unittest-everythingelse\build\xpcshell\tests\test_browser_places\unit\test_browserGlue_smartBookmarks.js | test failed (with xpcshell return code: 0), see following log:
TEST-UNEXPECTED-FAIL | e:/builds/moz2_slave/mozilla-central-win32-debug-unittest-everythingelse/build/xpcshell/tests/test_browser_places/unit/test_browserGlue_smartBookmarks.js | 3 == 2 - See following stack:



(this is with the commit two comments back, so it looks like it didn't help)
Well, now it's failing at line 147 instead of line 149.  I'm going to disable this test because it's been failing frequently.  Marco and I will take a hard look at this test again this week.

http://hg.mozilla.org/mozilla-central/rev/10bca9b1ccef
i did already know this was not fixed by disabling sync component, we already did that, this test does not have anything strange that calls for a fix, and i was never able to reproduce the issue locally, it fails only on tinderboxes, exactly like the random failure in test_browserGlue_distribution and test_browserGlue_prefs.
this test exists from March but it started failing only on August, without apparent reasons, or maybe something landed during that month that we did not notice.
I think there could be something bad in importExportService that is used by all tests, looks like sometimes bookmarks service fails in it, i've suggested a first cleanup (with more error logging) in bug 420729.
it could be unrelated, btw we upgraded to sqlite 3.6.16 just some day before first failures, i can't find other interesting changes in the first days of August atm
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1262958311.1262963890.31824.gz

i've enabled the test to get more logging out of it, i'll disable it again once i get enough data.
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1262972292.1262977869.31609.gz

WINNT 5.2 mozilla-central debug test everythingelse [testfailed] Started 09:38, finished 11:12
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1262982910.1262990102.5800.gz
WINNT 5.2 mozilla-central debug test everythingelse on 2010/01/08 12:35:10
s: win32-slave08
TEST-UNEXPECTED-FAIL | e:\builds\moz2_slave\mozilla-central-win32-debug-unittest-everythingelse\build\xpcshell\tests\test_browser_places\unit\test_browserGlue_smartBookmarks.js | test failed (with xpcshell return code: 0), see following log:
TEST-UNEXPECTED-FAIL | e:/builds/moz2_slave/mozilla-central-win32-debug-unittest-everythingelse/build/xpcshell/tests/test_browser_places/unit/test_browserGlue_smartBookmarks.js | 3 == 2 - See following stack:
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1262980756.1262987037.17050.gz
WINNT 5.2 mozilla-central debug test everythingelse on 2010/01/08 11:59:16
s: win32-slave30
WINNT 5.2 mozilla-central debug test everythingelse [testfailed] Started 13:21, finished 15:45

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1262985689.1262994280.17902.gz
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1262991673.1262998513.26390.gz
WINNT 5.2 mozilla-central debug test everythingelse on 2010/01/08 15:01:13
s: win32-slave15
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1262992139.1262998964.1538.gz
WINNT 5.2 mozilla-central debug test everythingelse on 2010/01/08 15:08:59
s: win32-slave27
Possibly related?

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1262999214.1263002092.28434.gz
OS X 10.5.2 mozilla-central opt test everythingelse on 2010/01/08 17:06:54
s: moz2-darwin9-slave19
TEST-UNEXPECTED-FAIL | /builds/slave/mozilla-central-macosx-opt-unittest-everythingelse/build/xpcshell/tests/test_browser_places/unit/test_browserGlue_smartBookmarks.js | test failed (with xpcshell return code: 0), see following log:
TEST-UNEXPECTED-FAIL | /builds/slave/mozilla-central-macosx-opt-unittest-everythingelse/build/xpcshell/tests/test_browser_places/unit/test_browserGlue_smartBookmarks.js | new title != new title - See following stack:

[There is no stack]
WINNT 5.2 mozilla-central debug
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1263019552.1263026560.31850.gz

TEST-UNEXPECTED-FAIL | e:/builds/moz2_slave/mozilla-central-win32-debug-unittest-everythingelse/build/xpcshell/tests/test_browser_places/unit/test_browserGlue_smartBookmarks.js | 4 == 6 - See following stack:
more cleanup and dumps for browserGlue_smartBookmarks and browserGlue_prefs:
http://hg.mozilla.org/mozilla-central/rev/b9d15f414e28

For now i've been able to identify a failure reason for test_browserGlue_distribution.js, that will be fixed in next days in bug 523936.
These are still a bit obscure even if i've identified some dangerous usage of notifications.
WINNT 5.2 mozilla-central debug test everythingelse on 2010/01/11 08:45:56
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1263228356.1263234142.15096.gz

test_browserGlue_smartBookmarks.js | test failed (with xpcshell return code: 0), see following log:
test_browserGlue_smartBookmarks.js | 4 == 6 - See following stack:
it is possible that something throws in ensurePlacesDefaultQueriesInitialized, in such a case we reportError but continue, that won't be detected in an xpcshell test.
since there is nothing crazy in tables dumps, let's see if we catch any exception in the console.
Attachment #421097 - Flags: review?(sdwilsh)
Comment on attachment 421097 [details] [diff] [review]
listen to console

>+    QueryInterface: function(iid) {
>+      if (iid.equals(Ci.nsIConsoleListener) ||
>+          iid.equals(Ci.nsISupports)) {
>+        return this;
>+      }
>+      throw Cr.NS_NOINTERFACE;
>+    }
nit: use XPCOMUtils, and add trailing comma please

r=sdwilsh
Attachment #421097 - Flags: review?(sdwilsh) → review+
WINNT 5.2 mozilla-central debug test everythingelse [testfailed] Started 11:59, finished 13:51
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1263239949.1263246650.27088.gz&fulltext=1
WINNT 5.2 mozilla-central debug test everythingelse [testfailed] Started 12:20, finished 14:07
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1263241221.1263247583.4912.gz&fulltext=1
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1263252702.1263258335.26928.gz
WINNT 5.2 mozilla-central debug test everythingelse on 2010/01/11 15:31:42
s: win32-slave41
TEST-UNEXPECTED-FAIL | e:\builds\moz2_slave\mozilla-central-win32-debug-unittest-everythingelse\build\xpcshell\tests\test_browser_places\unit\test_browserGlue_smartBookmarks.js | test failed (with xpcshell return code: 0), see following log:
TEST-UNEXPECTED-FAIL | e:/builds/moz2_slave/mozilla-central-win32-debug-unittest-everythingelse/build/xpcshell/tests/test_browser_places/unit/test_browserGlue_smartBookmarks.js | 4 == 6 - See following stack:
i disabled the test on Windows for now, other platforms will be enough to ensure the functionality.
The console logging (comment 59) has not brought any interesting result.
I'll try to setup a record and replay env, hoping to be able to reproduce the failure.

http://hg.mozilla.org/mozilla-central/rev/27c48ab51c3a
Whiteboard: [orange] → [orange][disabled on Windows in central]
But note that (from comment #51) this bug may also happen on other platforms than Windows.
(In reply to comment #64)
> But note that (from comment #51) this bug may also happen on other platforms
> than Windows.

it is possible (the test changed a bit from then and also we fixed some query that could have been related to that specific failure). The shouting case was still Windows (practically perma orange), i would survive a failure a week to avoid completely losing testing a code path. So let's see how this performs right now.
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1267656514.1267657946.7621.gz
OS X 10.5.2 mozilla-central opt test xpcshell on 2010/03/03 14:48:34
s: moz2-darwin9-slave14

TEST-UNEXPECTED-FAIL | /builds/slave/mozilla-central-macosx-opt-unittest-xpcshell/build/xpcshell/tests/test_browser_places/unit/test_browserGlue_smartBookmarks.js | test failed (with xpcshell return code: 0), see following log:
TEST-UNEXPECTED-FAIL | /builds/slave/mozilla-central-macosx-opt-unittest-xpcshell/build/xpcshell/tests/test_browser_places/unit/test_browserGlue_smartBookmarks.js | 3 == 2 - See following stack:
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1268588709.1268590688.19829.gz
OS X 10.5.2 mozilla-central debug test xpcshell on 2010/03/14 10:45:09
s: moz2-darwin9-slave16

TEST-UNEXPECTED-FAIL | /builds/slave/mozilla-central-macosx-debug-unittest-xpcshell/build/xpcshell/tests/test_browser_places/unit/test_browserGlue_smartBookmarks.js | test failed (with xpcshell return code: 0), see following log:
TEST-UNEXPECTED-FAIL | /builds/slave/mozilla-central-macosx-debug-unittest-xpcshell/build/xpcshell/tests/test_browser_places/unit/test_browserGlue_smartBookmarks.js | 4 == 3 - See following stack:
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1268658320.1268660094.11279.gz
OS X 10.5.2 mozilla-central debug test xpcshell on 2010/03/15 06:05:20
meant to paste this ^^: OS X 10.5.2 mozilla-central debug test xpcshell [testfailed] Started 10:19, finished 10:53
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1270491944.1270494222.5080.gz
OS X 10.5.2 mozilla-central debug test xpcshell on 2010/04/05 11:25:44
s: moz2-darwin9-slave39

TEST-UNEXPECTED-FAIL | /builds/slave/mozilla-central-macosx-debug-unittest-xpcshell/build/xpcshell/tests/test_browser_places/unit/test_browserGlue_distribution.js | test failed (with xpcshell return code: -6), see following log:
TEST-UNEXPECTED-FAIL | /builds/slave/mozilla-central-macosx-debug-unittest-xpcshell/build/xpcshell/tests/test_browser_places/unit/test_browserGlue_smartBookmarks.js | test failed (with xpcshell return code: 0), see following log:
TEST-UNEXPECTED-FAIL | /builds/slave/mozilla-central-macosx-debug-unittest-xpcshell/build/xpcshell/tests/test_browser_places/unit/test_browserGlue_smartBookmarks.js | 5 == 3 - See following stack:
Blocks: 559354
the latest failures here are due to est_browserGlue_distribution.js | test failed (with xpcshell return code: -6)

since the test exits badly, sounds like we don't run registered cleanup functions...
distribution test has been disabled for now, till we figure it out.
are you saying that this is solely caused by distribution.ini being imported? if so, should this be resolved?
no, it has 2 causes, one is distribution.ini, the other one is unknown, even if actually we could try re-enable it and see what happens. Fixing tests without being able to reproduce failures is hard :(
should be fixed by Places branch merge.
Status: NEW → RESOLVED
Closed: 14 years ago
Resolution: --- → FIXED
Whiteboard: [orange][disabled on Windows in central] → [orange]
Target Milestone: --- → Firefox 4.0b9
Whiteboard: [orange]
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: