intermittent failure in url-classifier/tests/unit/test_hashcompleter.js, failure after "Now on completion set index 4"

RESOLVED FIXED in Firefox 7

Status

()

Toolkit
Safe Browsing
RESOLVED FIXED
6 years ago
3 years ago

People

(Reporter: mak, Assigned: mmm)

Tracking

({intermittent-failure})

unspecified
Firefox 7
x86
Linux
intermittent-failure
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment, 3 obsolete attachments)

(Reporter)

Description

6 years ago
http://tinderbox.mozilla.org/showlog.cgi?log=Places/1303519746.1303520765.14912.gz
Rev3 Fedora 12 places opt test xpcshell on 2011/04/22 17:49:06

Now on completion set index 4
TEST-PASS | /home/cltbld/talos-slave/test/build/xpcshell/tests/toolkit/components/url-classifier/tests/unit/test_hashcompleter.js | [completionFinished : 296] false == false
TEST-PASS | /home/cltbld/talos-slave/test/build/xpcshell/tests/toolkit/components/url-classifier/tests/unit/test_hashcompleter.js | [completionFinished : 296] false == false
TEST-PASS | /home/cltbld/talos-slave/test/build/xpcshell/tests/toolkit/components/url-classifier/tests/unit/test_hashcompleter.js | [completionFinished : 296] false == false
TEST-PASS | /home/cltbld/talos-slave/test/build/xpcshell/tests/toolkit/components/url-classifier/tests/unit/test_hashcompleter.js | [completionFinished : 296] false == false
TEST-UNEXPECTED-FAIL | /home/cltbld/talos-slave/test/build/xpcshell/tests/toolkit/components/url-classifier/tests/unit/test_hashcompleter.js | false == true - See following stack:
JS frame :: /home/cltbld/talos-slave/test/build/xpcshell/head.js :: do_throw :: line 444
JS frame :: /home/cltbld/talos-slave/test/build/xpcshell/head.js :: _do_check_eq :: line 538
JS frame :: /home/cltbld/talos-slave/test/build/xpcshell/head.js :: do_check_eq :: line 559
JS frame :: /home/cltbld/talos-slave/test/build/xpcshell/head.js :: do_check_true :: line 573
JS frame :: /home/cltbld/talos-slave/test/build/xpcshell/tests/toolkit/components/url-classifier/tests/unit/test_hashcompleter.js :: completion :: line 267
(Reporter)

Comment 1

6 years ago
Also, philor pointed out this happened on 20 Apr in central

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1303304729.1303305530.18043.gz
Created attachment 527903 [details] [diff] [review]
A couple dumps to provide more information in this case. (checked in)

It seems we are running into this intermittent issue in the randomly generated portion of the test. So far, the only case I have thought of is that two generated hashes are the same and one expects a completion while the other doesn't.

It does seem odd that these are popping up recently.

Pushed this patch to m-c:
http://hg.mozilla.org/mozilla-central/rev/05825e4dc77a
Created attachment 527906 [details] [diff] [review]
Patch to fix typo. (combined check-in)

Previous attachment had a typo. If the intermittent issue isn't hit, it shouldn't cause a problem.

This patch is a combination of the past and current.

http://hg.mozilla.org/mozilla-central/rev/4f40b01a2f47
Attachment #527903 - Attachment is obsolete: true
Comment hidden (Treeherder Robot)
Don't have internet right now so I can't actively track/work on this but my plan of action is to change the random number generator and output a seed on start to help make this more reproducible.

So far, I still have the same theory: we create data which creates contradictions in what we expect from the completion (e.g. a data set with two same prefixes that have different tables and different |expectCompletion| values)
Assignee: nobody → mars.martian+bugmail
Summary: intermittent failure in url-classifier/tests/unit/test_hashcompleter.js | false == true → intermittent failure in url-classifier/tests/unit/test_hashcompleter.js, failure after "Now on completion set index 4"
Comment hidden (Treeherder Robot)
(In reply to comment #6)
> m_kato%ga2.so-net.ne.jp
> http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1305099298.1305100163.
> 17366.gz
> Rev3 MacOSX Leopard 10.5.8 mozilla-central opt test xpcshell on 2011/05/11
> 00:34:58
> 
> s: talos-r3-leopard-038
> TEST-UNEXPECTED-FAIL |
> /Users/cltbld/talos-slave/test/build/xpcshell/tests/toolkit/components/url-
> classifier/tests/unit/test_hashcompleter.js | test failed (with xpcshell
> return code: 0), see following log:
> TEST-UNEXPECTED-FAIL |
> /Users/cltbld/talos-slave/test/build/xpcshell/tests/toolkit/components/url-
> classifier/tests/unit/test_hashcompleter.js | false == true - See following
> stack:

Yaus. As expected, it looks like we created duplicate entries with the same prefix. Relevant debug info:

Did not expect a completion for this result. Provided values:
hash: "<\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000"
table: e8hmjb
chunkId: 29884
Actual values:
hash: "<\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000"
table: undefined
chunkId: undefined

my plan of action is to check for duplicate prefixes and handle them explicitly. (i.e. create a new prefix)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Created attachment 537414 [details] [diff] [review]
Add random number generator, switch to that and check for duplicates.

I added in a tiny random number generator so that the random numbers are created by a seed that is trackable. (This should make debugging easier.)

I also added a hook to ensure that all prefixes are unique which handles this intermittent and lastly, I removed the debug lines I added in.

Dave, I flagged you as you reviewed the original test though I can move it to someone else.
Attachment #527906 - Attachment is obsolete: true
Attachment #537414 - Flags: review?(dcamp)

Updated

6 years ago
Attachment #537414 - Flags: review?(dcamp) → review+
Comment hidden (Treeherder Robot)
http://hg.mozilla.org/integration/mozilla-inbound/rev/96fac50b57ed
Status: NEW → ASSIGNED
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
I backed out the attempt to fix this, since it caused Windows opt unit tests to be orange on every single run.
Comment hidden (Treeherder Robot)
Created attachment 539017 [details] [diff] [review]
Take two: check for duplicate prefixes, not hashes.

In the first patch I ensured that we generated unique hashes and thought this implied a unique |completion| call per hash. However, if two hashes share the same prefix (first 32 bits) then it is possible to get multiple completions per hash. I changed the logic to check for duplicate prefixes.

This ensures that each hash only goes through one code path while limiting the number of possible hashes generated by the random set to 256^4. I reran the failed tests (by using the same seed) and they now pass. And lastly, it looks like this failure was hit on WinXP opt 2/3 times and by chance (that is, it does not appear to be a problem with WinXP opt test runs).
Attachment #537414 - Attachment is obsolete: true
Attachment #539017 - Flags: review?(dcamp)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)

Updated

6 years ago
Attachment #539017 - Flags: review?(dcamp) → review+
Second try, pushed to mozilla-inbound.
http://hg.mozilla.org/integration/mozilla-inbound/rev/303ddaeb736f
http://hg.mozilla.org/mozilla-central/rev/303ddaeb736f
Status: ASSIGNED → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 7
Keywords: intermittent-failure
Whiteboard: [orange]
Component: Phishing Protection → Phishing Protection
Product: Firefox → Toolkit
You need to log in before you can comment on or make changes to this bug.