Closed Bug 652294 Opened 13 years ago Closed 13 years ago

intermittent failure in url-classifier/tests/unit/test_hashcompleter.js, failure after "Now on completion set index 4"

Categories

(Toolkit :: Safe Browsing, defect)

Product:
Toolkit
Component:
Safe Browsing
Platform:
x86
Linux
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED FIXED
Milestone:
Firefox 7

People

(Reporter: mak, Assigned: mmm)

References

Details

(Keywords: intermittent-failure)

Attachments

(1 file, 3 obsolete files)

Take two: check for duplicate prefixes, not hashes.
4.69 KB, patch
dcamp
: review+
Details | Diff | Splinter Review 
http://tinderbox.mozilla.org/showlog.cgi?log=Places/1303519746.1303520765.14912.gz
Rev3 Fedora 12 places opt test xpcshell on 2011/04/22 17:49:06

Now on completion set index 4
TEST-PASS | /home/cltbld/talos-slave/test/build/xpcshell/tests/toolkit/components/url-classifier/tests/unit/test_hashcompleter.js | [completionFinished : 296] false == false
TEST-PASS | /home/cltbld/talos-slave/test/build/xpcshell/tests/toolkit/components/url-classifier/tests/unit/test_hashcompleter.js | [completionFinished : 296] false == false
TEST-PASS | /home/cltbld/talos-slave/test/build/xpcshell/tests/toolkit/components/url-classifier/tests/unit/test_hashcompleter.js | [completionFinished : 296] false == false
TEST-PASS | /home/cltbld/talos-slave/test/build/xpcshell/tests/toolkit/components/url-classifier/tests/unit/test_hashcompleter.js | [completionFinished : 296] false == false
TEST-UNEXPECTED-FAIL | /home/cltbld/talos-slave/test/build/xpcshell/tests/toolkit/components/url-classifier/tests/unit/test_hashcompleter.js | false == true - See following stack:
JS frame :: /home/cltbld/talos-slave/test/build/xpcshell/head.js :: do_throw :: line 444
JS frame :: /home/cltbld/talos-slave/test/build/xpcshell/head.js :: _do_check_eq :: line 538
JS frame :: /home/cltbld/talos-slave/test/build/xpcshell/head.js :: do_check_eq :: line 559
JS frame :: /home/cltbld/talos-slave/test/build/xpcshell/head.js :: do_check_true :: line 573
JS frame :: /home/cltbld/talos-slave/test/build/xpcshell/tests/toolkit/components/url-classifier/tests/unit/test_hashcompleter.js :: completion :: line 267
Also, philor pointed out this happened on 20 Apr in central

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1303304729.1303305530.18043.gz
It seems we are running into this intermittent issue in the randomly generated portion of the test. So far, the only case I have thought of is that two generated hashes are the same and one expects a completion while the other doesn't.

It does seem odd that these are popping up recently.

Pushed this patch to m-c:
http://hg.mozilla.org/mozilla-central/rev/05825e4dc77a
Previous attachment had a typo. If the intermittent issue isn't hit, it shouldn't cause a problem.

This patch is a combination of the past and current.

http://hg.mozilla.org/mozilla-central/rev/4f40b01a2f47
Attachment #527903 - Attachment is obsolete: true
Don't have internet right now so I can't actively track/work on this but my plan of action is to change the random number generator and output a seed on start to help make this more reproducible.

So far, I still have the same theory: we create data which creates contradictions in what we expect from the completion (e.g. a data set with two same prefixes that have different tables and different |expectCompletion| values)
Assignee: nobody → mars.martian+bugmail
Summary: intermittent failure in url-classifier/tests/unit/test_hashcompleter.js | false == true → intermittent failure in url-classifier/tests/unit/test_hashcompleter.js, failure after "Now on completion set index 4"
(In reply to comment #6)
> m_kato%ga2.so-net.ne.jp
> http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1305099298.1305100163.
> 17366.gz
> Rev3 MacOSX Leopard 10.5.8 mozilla-central opt test xpcshell on 2011/05/11
> 00:34:58
> 
> s: talos-r3-leopard-038
> TEST-UNEXPECTED-FAIL |
> /Users/cltbld/talos-slave/test/build/xpcshell/tests/toolkit/components/url-
> classifier/tests/unit/test_hashcompleter.js | test failed (with xpcshell
> return code: 0), see following log:
> TEST-UNEXPECTED-FAIL |
> /Users/cltbld/talos-slave/test/build/xpcshell/tests/toolkit/components/url-
> classifier/tests/unit/test_hashcompleter.js | false == true - See following
> stack:

Yaus. As expected, it looks like we created duplicate entries with the same prefix. Relevant debug info:

Did not expect a completion for this result. Provided values:
hash: "<\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000"
table: e8hmjb
chunkId: 29884
Actual values:
hash: "<\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000"
table: undefined
chunkId: undefined

my plan of action is to check for duplicate prefixes and handle them explicitly. (i.e. create a new prefix)
I added in a tiny random number generator so that the random numbers are created by a seed that is trackable. (This should make debugging easier.)

I also added a hook to ensure that all prefixes are unique which handles this intermittent and lastly, I removed the debug lines I added in.

Dave, I flagged you as you reviewed the original test though I can move it to someone else.
Attachment #527906 - Attachment is obsolete: true
Attachment #537414 - Flags: review?(dcamp)
Attachment #537414 - Flags: review?(dcamp) → review+
I backed out the attempt to fix this, since it caused Windows opt unit tests to be orange on every single run.
In the first patch I ensured that we generated unique hashes and thought this implied a unique |completion| call per hash. However, if two hashes share the same prefix (first 32 bits) then it is possible to get multiple completions per hash. I changed the logic to check for duplicate prefixes.

This ensures that each hash only goes through one code path while limiting the number of possible hashes generated by the random set to 256^4. I reran the failed tests (by using the same seed) and they now pass. And lastly, it looks like this failure was hit on WinXP opt 2/3 times and by chance (that is, it does not appear to be a problem with WinXP opt test runs).
Attachment #537414 - Attachment is obsolete: true
Attachment #539017 - Flags: review?(dcamp)
Attachment #539017 - Flags: review?(dcamp) → review+
http://hg.mozilla.org/mozilla-central/rev/303ddaeb736f
Status: ASSIGNED → RESOLVED
Closed: 13 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 7
Whiteboard: [orange]
Product: Firefox → Toolkit
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: