Closed Bug 739927 Opened 11 years ago Closed 11 years ago

"TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/dom-level1-core/test_characterdatadeletedatanomodificationallowederr.html | entElementContentNotNull" and more, reported though not counted

Categories

(Core :: DOM: Core & HTML, defect, P3)

Product:
Core
Component:
DOM: Core & HTML
Type:
defect

Tracking

()

Status:
VERIFIED FIXED
Milestone:
mozilla14

People

(Reporter: sgautherie, Assigned: sgautherie)

References

(Blocks 1 open bug)

Details

(Keywords: regression, Whiteboard: [mistagged when bug 696306 or bug 739557 happen])

Attachments

(3 files, 1 obsolete file)

(Log) with test_bug500328.html (and more) case
8.59 KB, text/plain
Details
(Log) with test_dom_input_event_on_htmleditor.html case
7.36 KB, text/plain
Details
(Av2) dom-level*-*/DOMTestCase.js: Fix SimpleTest._logResult() override + testFails() + markTodos(), Fake a missing warn() function in checkInitialization(), ... [Checked in: Comment 24]
16.63 KB, patch
jmaher
: review+
Details | Diff | Splinter Review 
Bug 483992 patch Av1
"dom-level*-*/DOMTestCase.js: Remove sayrer's override of SimpleTest._logResult()"
"revealed" this random behavior (on Windows).

NB: Although this is kind of a temporary regression, it is the goal to sort out how these tests actually behave (rather than just "ignoring" their results).

***

These (reproducible) failures happen in dom-level1-core and dom-level2-core.
http://mxr.mozilla.org/mozilla-central/find?text=&string=dom%2Ftests%2Fmochitest%2Fdom-level1-core%2F
http://mxr.mozilla.org/mozilla-central/find?text=&string=dom%2Ftests%2Fmochitest%2Fdom-level2-core%2F

Hopefully, it's just the first test causing failures in the following ones...

***

This also causes failures in 1+ (very) later test(s).

Need to investigate that too/first...
The additional failures were
{
10064 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/whatwg/test_bug500328.html | an unexpected uncaught JS exception reported through window.onerror - uncaught exception: [Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIDOMHistory.pushState]"  nsresult: "0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame :: http://mochi.test:8888/tests/dom/tests/mochitest/whatwg/test_bug500328.html :: runTest :: line 469"  data: no] at :0

11039 ERROR TEST-UNEXPECTED-FAIL | /tests/editor/libeditor/base/tests/test_bug586662.html | Test timed out.

15913 ERROR TEST-UNEXPECTED-FAIL | /tests/editor/libeditor/html/tests/test_bug332636.html | The backspace key should delete the UTF-16 surrogate pair correctly - got axb, expected ab

[...]

15943 ERROR TEST-UNEXPECTED-FAIL | (SimpleTest/TestRunner.js) | 4 test timeouts, giving up.
15944 ERROR TEST-UNEXPECTED-FAIL | (SimpleTest/TestRunner.js) | Skipping 45 remaining tests.
}
The additional failures are
{
21179 ERROR TEST-UNEXPECTED-FAIL | /tests/editor/libeditor/html/tests/test_dom_input_event_on_htmleditor.html | Editor4, html and div has contenteditable attribute: wrong element was edited - got , expected a
21180 ERROR TEST-UNEXPECTED-FAIL | /tests/editor/libeditor/html/tests/test_dom_input_event_on_htmleditor.html | Editor4, html and div has contenteditable attribute: input event wasn't fired by 'a' key
21181 ERROR TEST-UNEXPECTED-FAIL | /tests/editor/libeditor/html/tests/test_dom_input_event_on_htmleditor.html | an unexpected uncaught JS exception reported through window.onerror - inputEvent is null at http://mochi.test:8888/tests/editor/libeditor/html/tests/test_dom_input_event_on_htmleditor.html:72
}
https://tbpl.mozilla.org/php/getParsedLog.php?id=10432623&tree=Firefox
Rev3 WINNT 5.1 mozilla-central debug test mochitests-3/5 on 2012-03-28 02:12:50 PDT for push 4f8b9ff09a8a

Same as comment 3.


(In reply to Tim Taubert [:ttaubert] from comment #1)
> https://tbpl.mozilla.org/php/getParsedLog.php?id=10433198&tree=Fx-Team

Rev3 WINNT 5.1 fx-team debug test mochitests-3/5 on 2012-03-28 02:17:00 PDT for push 9f166123a74f

Same as comment 2, but on "WINNT 5.1 fx-team debug test".


It looks like both cases can happen randomly...
(In reply to Tim Taubert [:ttaubert] from comment #5)
> https://tbpl.mozilla.org/php/getParsedLog.php?id=10435936&tree=Fx-Team

Rev3 WINNT 5.1 fx-team opt test mochitests-3/5 on 2012-03-28 04:41:49 PDT for push 014c39d29e30

Same as comment 3, but on "WINNT 5.1 fx-team opt test".

(In reply to Serge Gautherie (:sgautherie) from comment #4)
> It looks like both cases can happen randomly...

Confirmed.
Attachment #610067 - Attachment description: (Log) 2nd occurence (WINNT 5.1 mozilla-central debug test) → (Log) with test_dom_input_event_on_htmleditor.html case
Attachment #610066 - Attachment description: (Log) 1st occurence (WINNT 6.1 try opt test) → (Log) with test_bug500328.html (and more) case
(In reply to Serge Gautherie (:sgautherie) from comment #0)

> Hopefully, it's just the first test causing failures in the following ones...

This is a log regression that happens in all runs,
though the tests are behaving (and are managed) as (currently) expected :-<

Sorry I didn't notice it in the first place, because these "failures" are actually morphed into "todos" wrt result counts :-|

> This also causes failures in 1+ (very) later test(s).
> 
> Need to investigate that too/first...

These 2 random failing cases are actually bug 696306 and bug 739557 !


(In reply to TinderboxPushlog Robot from comment #13)
> https://tbpl.mozilla.org/php/getParsedLog.php?id=10452825&tree=Mozilla-
> Inbound
> Android XUL Tegra 250 mozilla-inbound opt test mochitest-3 on 2012-03-28
> 15:59:16

This occurrence is this log regression alone :->
Blocks: 696306, 739557
No longer blocks: 438871
OS: Windows XP → All
Hardware: x86 → All
Summary: Intermittent "TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/dom-level1-core/test_characterdatadeletedatanomodificationallowederr.html | entElementContentNotNull" and more → "TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/dom-level1-core/test_characterdatadeletedatanomodificationallowederr.html | entElementContentNotNull" and more, reported though not counted
Whiteboard: [orange] → [mistagged when bug 696306 or bug 739557 happen]
Blocks: 483555
Try run for a78fe53feb56 is complete.
Detailed breakdown of the results available here:
    https://tbpl.mozilla.org/?tree=Try&rev=a78fe53feb56
Results (out of 31 total builds):
    success: 17
    warnings: 14
Builds (or logs if builds failed) available at:
http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/sgautherie.bz@free.fr-a78fe53feb56
(In reply to Mozilla RelEng Bot from comment #15)
>     https://tbpl.mozilla.org/?tree=Try&rev=a78fe53feb56

Bah, stupid typo :-<
Try run for ac524fae3fb6 is complete.
Detailed breakdown of the results available here:
    https://tbpl.mozilla.org/?tree=Try&rev=ac524fae3fb6
Results (out of 31 total builds):
    success: 30
    warnings: 1
Builds (or logs if builds failed) available at:
http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/sgautherie.bz@free.fr-ac524fae3fb6
Try run for 49b05527e882 is complete.
Detailed breakdown of the results available here:
    https://tbpl.mozilla.org/?tree=Try&rev=49b05527e882
Results (out of 31 total builds):
    success: 30
    warnings: 1
Builds (or logs if builds failed) available at:
http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/sgautherie.bz@free.fr-49b05527e882
Comment on attachment 610732 [details] [diff] [review]
(Av1) dom-level*-*/DOMTestCase.js: Fix SimpleTest._logResult() override + testFails() + markTodos(), Fake a missing warn() function in checkInitialization(), Add missing early returns to ...

I don't have time for these right now. Can we get one of the other testing peers to take a look at this? Thanks.
Attachment #610732 - Flags: review?(rcampbell) → review?(jmaher)
Comment on attachment 610732 [details] [diff] [review]
(Av1) dom-level*-*/DOMTestCase.js: Fix SimpleTest._logResult() override + testFails() + markTodos(), Fake a missing warn() function in checkInitialization(), Add missing early returns to ...

Review of attachment 610732 [details] [diff] [review]:
-----------------------------------------------------------------

I don't understand why we are removing the logging function from dom-level2-html.  I also have a lot of questions below.

::: dom/tests/mochitest/dom-level1-core/DOMTestCase.js
@@ +35,4 @@
>      ok(actual !== null, descr);
> +    // Work around too strict checks.
> +    if (!actual) {
> +      ok(actual, "[assertSize()] 'actual' has a value");

not sure I understand the point of this ok() check.

@@ +188,5 @@
>    function assertURIEquals(assertID, scheme, path, host, file, name, query, fragment, isAbsolute, actual) {
>      //
>      //  URI must be non-null
> +    ok(assertID, "[assertURIEquals()] 'assertID' has no value");
> +    ok(actual, "[assertURIEquals()] 'actual' has no value");

how does this ok(actual...) call differ from the one in assertSize()?  The error messages are different indicating this one has no value and the one in assertSize has a value.

@@ +193,5 @@
> +/*
> +    // Add missing early return.
> +    if (!actual)
> +      return;
> +*/

why is this commented out?

::: dom/tests/mochitest/dom-level2-core/DOMTestCase.js
@@ +38,5 @@
> +    if (!actual) {
> +      ok(actual, "[assertSize()] 'actual' has a value");
> +      return;
> +    }
> +*/

why are we adding code inside a comment here?  Shouldn't we do what we do for dom-level1-core?

@@ +197,5 @@
> +/*
> +    // Add missing early return.
> +    if (!actual)
> +      return;
> +*/

same question here?  why the comment?

::: dom/tests/mochitest/dom-level2-html/DOMTestCase.js
@@ +38,5 @@
> +    if (!actual) {
> +      ok(actual, "[assertSize()] 'actual' has a value");
> +      return;
> +    }
> +*/

why the comments?

@@ +197,5 @@
> +/*
> +    // Add missing early return.
> +    if (!actual)
> +      return;
> +*/

why the comments?

@@ +683,2 @@
>  }
> +*/

why is this entire block commented out?
Attachment #610732 - Flags: review?(jmaher) → review-
Av1, with comment 21 suggestion(s).


(In reply to Joel Maher (:jmaher) from comment #21)

> I don't understand why we are removing the logging function from
> dom-level2-html.

I'm not: its feature was commented out.

> >      ok(actual !== null, descr);
> > +    // Work around too strict checks.
> > +    if (!actual) {
> > +      ok(actual, "[assertSize()] 'actual' has a value");
> 
> not sure I understand the point of this ok() check.

To log why the early return, despite the previous/existing ok().

> how does this ok(actual...) call differ from the one in assertSize()?  The
> error messages are different indicating this one has no value and the one in
> assertSize has a value.

Right, messages fixed.

> @@ +193,5 @@
> > +/*
> > +    // Add missing early return.
> > +    if (!actual)
> > +      return;
> > +*/
> 
> why is this commented out?

Because, in the few logs I checked, we're not hitting this case,
yet I'm "documenting" this code flaw, for future use if need be.

> why are we adding code inside a comment here?  Shouldn't we do what we do
> for dom-level1-core?

Because, in the few logs I checked, we're not hitting this case,
yet I'm maintaining some code parity between the 3 files, for future use if need be.

> @@ +197,5 @@
> > +/*
> > +    // Add missing early return.
> > +    if (!actual)
> > +      return;
> > +*/
> 
> same question here?  why the comment?

Same answer. To give a hint that it's not existing code that was commented out, but new code to use if need be.

> @@ +683,2 @@
> >  }
> > +*/
> 
> why is this entire block commented out?

Because there is no ToDo tests in dom-level2-html atm.
Attachment #610732 - Attachment is obsolete: true
Attachment #611029 - Flags: review?(jmaher)
Comment on attachment 611029 [details] [diff] [review]
(Av2) dom-level*-*/DOMTestCase.js: Fix SimpleTest._logResult() override + testFails() + markTodos(), Fake a missing warn() function in checkInitialization(), ...
[Checked in: Comment 24]

Review of attachment 611029 [details] [diff] [review]:
-----------------------------------------------------------------

thanks.
Attachment #611029 - Flags: review?(jmaher) → review+
Comment on attachment 611029 [details] [diff] [review]
(Av2) dom-level*-*/DOMTestCase.js: Fix SimpleTest._logResult() override + testFails() + markTodos(), Fake a missing warn() function in checkInitialization(), ...
[Checked in: Comment 24]

https://hg.mozilla.org/mozilla-central/rev/85a31c1e4963
Attachment #611029 - Attachment description: (Av2) dom-level*-*/DOMTestCase.js: Fix SimpleTest._logResult() override + testFails() + markTodos(), Fake a missing warn() function in checkInitialization(), Add missing early returns to ... → (Av2) dom-level*-*/DOMTestCase.js: Fix SimpleTest._logResult() override + testFails() + markTodos(), Fake a missing warn() function in checkInitialization(), ... [Checked in: Comment 24]
Status: ASSIGNED → RESOLVED
Closed: 11 years ago
Flags: in-testsuite+
Resolution: --- → FIXED
https://tbpl.mozilla.org/php/getParsedLog.php?id=10566168&tree=Firefox
Rev3 Fedora 12 mozilla-central opt test mochitests-3/5 on 2012-04-02 08:23:49 PDT for push 85a31c1e4963

V.Fixed
Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.