Last Comment Bug 739927 - "TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/dom-level1-core/test_characterdatadeletedatanomodificationallowederr.html | entElementContentNotNull" and more, reported though not counted
: "TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/dom-level1-core/test_chara...
Status: VERIFIED FIXED
[mistagged when bug 696306 or bug 739...
: regression
Product: Core
Classification: Components
Component: DOM: Core & HTML (show other bugs)
: Trunk
: All All
: P3 normal (vote)
: mozilla14
Assigned To: Serge Gautherie (:sgautherie)
:
Mentors:
Depends on:
Blocks: 696306 483555 483992 739557
  Show dependency treegraph
 
Reported: 2012-03-28 03:40 PDT by Serge Gautherie (:sgautherie)
Modified: 2012-04-02 11:26 PDT (History)
2 users (show)
bugzillamozillaorg_serge_20140323: in‑testsuite+
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments
(Log) with test_bug500328.html (and more) case (8.59 KB, text/plain)
2012-03-28 03:53 PDT, Serge Gautherie (:sgautherie)
no flags Details
(Log) with test_dom_input_event_on_htmleditor.html case (7.36 KB, text/plain)
2012-03-28 03:58 PDT, Serge Gautherie (:sgautherie)
no flags Details
(Av1) dom-level*-*/DOMTestCase.js: Fix SimpleTest._logResult() override + testFails() + markTodos(), Fake a missing warn() function in checkInitialization(), Add missing early returns to ... (16.64 KB, patch)
2012-03-29 15:53 PDT, Serge Gautherie (:sgautherie)
jmaher: review-
Details | Diff | Review
(Av2) dom-level*-*/DOMTestCase.js: Fix SimpleTest._logResult() override + testFails() + markTodos(), Fake a missing warn() function in checkInitialization(), ... [Checked in: Comment 24] (16.63 KB, patch)
2012-03-30 14:22 PDT, Serge Gautherie (:sgautherie)
jmaher: review+
Details | Diff | Review

Description Serge Gautherie (:sgautherie) 2012-03-28 03:40:49 PDT
Bug 483992 patch Av1
"dom-level*-*/DOMTestCase.js: Remove sayrer's override of SimpleTest._logResult()"
"revealed" this random behavior (on Windows).

NB: Although this is kind of a temporary regression, it is the goal to sort out how these tests actually behave (rather than just "ignoring" their results).

***

These (reproducible) failures happen in dom-level1-core and dom-level2-core.
http://mxr.mozilla.org/mozilla-central/find?text=&string=dom%2Ftests%2Fmochitest%2Fdom-level1-core%2F
http://mxr.mozilla.org/mozilla-central/find?text=&string=dom%2Ftests%2Fmochitest%2Fdom-level2-core%2F

Hopefully, it's just the first test causing failures in the following ones...

***

This also causes failures in 1+ (very) later test(s).

Need to investigate that too/first...
Comment 2 Serge Gautherie (:sgautherie) 2012-03-28 03:53:00 PDT
Created attachment 610066 [details]
(Log) with test_bug500328.html (and more) case

The additional failures were
{
10064 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/whatwg/test_bug500328.html | an unexpected uncaught JS exception reported through window.onerror - uncaught exception: [Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIDOMHistory.pushState]"  nsresult: "0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame :: http://mochi.test:8888/tests/dom/tests/mochitest/whatwg/test_bug500328.html :: runTest :: line 469"  data: no] at :0

11039 ERROR TEST-UNEXPECTED-FAIL | /tests/editor/libeditor/base/tests/test_bug586662.html | Test timed out.

15913 ERROR TEST-UNEXPECTED-FAIL | /tests/editor/libeditor/html/tests/test_bug332636.html | The backspace key should delete the UTF-16 surrogate pair correctly - got axb, expected ab

[...]

15943 ERROR TEST-UNEXPECTED-FAIL | (SimpleTest/TestRunner.js) | 4 test timeouts, giving up.
15944 ERROR TEST-UNEXPECTED-FAIL | (SimpleTest/TestRunner.js) | Skipping 45 remaining tests.
}
Comment 3 Serge Gautherie (:sgautherie) 2012-03-28 03:58:21 PDT
Created attachment 610067 [details]
(Log) with test_dom_input_event_on_htmleditor.html case

The additional failures are
{
21179 ERROR TEST-UNEXPECTED-FAIL | /tests/editor/libeditor/html/tests/test_dom_input_event_on_htmleditor.html | Editor4, html and div has contenteditable attribute: wrong element was edited - got , expected a
21180 ERROR TEST-UNEXPECTED-FAIL | /tests/editor/libeditor/html/tests/test_dom_input_event_on_htmleditor.html | Editor4, html and div has contenteditable attribute: input event wasn't fired by 'a' key
21181 ERROR TEST-UNEXPECTED-FAIL | /tests/editor/libeditor/html/tests/test_dom_input_event_on_htmleditor.html | an unexpected uncaught JS exception reported through window.onerror - inputEvent is null at http://mochi.test:8888/tests/editor/libeditor/html/tests/test_dom_input_event_on_htmleditor.html:72
}
Comment 4 Serge Gautherie (:sgautherie) 2012-03-28 04:07:32 PDT
https://tbpl.mozilla.org/php/getParsedLog.php?id=10432623&tree=Firefox
Rev3 WINNT 5.1 mozilla-central debug test mochitests-3/5 on 2012-03-28 02:12:50 PDT for push 4f8b9ff09a8a

Same as comment 3.


(In reply to Tim Taubert [:ttaubert] from comment #1)
> https://tbpl.mozilla.org/php/getParsedLog.php?id=10433198&tree=Fx-Team

Rev3 WINNT 5.1 fx-team debug test mochitests-3/5 on 2012-03-28 02:17:00 PDT for push 9f166123a74f

Same as comment 2, but on "WINNT 5.1 fx-team debug test".


It looks like both cases can happen randomly...
Comment 6 Serge Gautherie (:sgautherie) 2012-03-28 06:07:03 PDT
(In reply to Tim Taubert [:ttaubert] from comment #5)
> https://tbpl.mozilla.org/php/getParsedLog.php?id=10435936&tree=Fx-Team

Rev3 WINNT 5.1 fx-team opt test mochitests-3/5 on 2012-03-28 04:41:49 PDT for push 014c39d29e30

Same as comment 3, but on "WINNT 5.1 fx-team opt test".

(In reply to Serge Gautherie (:sgautherie) from comment #4)
> It looks like both cases can happen randomly...

Confirmed.
Comment 13 Treeherder Robot 2012-03-28 16:32:03 PDT
edmorley
https://tbpl.mozilla.org/php/getParsedLog.php?id=10452825&tree=Mozilla-Inbound
Android XUL Tegra 250 mozilla-inbound opt test mochitest-3 on 2012-03-28 15:59:16

114 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/dom-level1-core/test_characterdatadeletedatanomodificationallowederr.html | entElementContentNotNull
115 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/dom-level1-core/test_characterdatadeletedatanomodificationallowederr.html | throw_NO_MODIFICATION_ALLOWED_ERR
129 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/dom-level1-core/test_characterdataindexsizeerrdeletedatacountnegative.html | throws_INDEX_SIZE_ERR
145 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/dom-level1-core/test_characterdataindexsizeerrreplacedatacountnegative.html | throws_INDEX_SIZE_ERR
155 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/dom-level1-core/test_characterdataindexsizeerrsubstringcountnegative.html | throws_INDEX_SIZE_ERR
315 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/dom-level1-core/test_documentinvalidcharacterexceptioncreateentref.html | throw_NOT_SUPPORTED_ERR
319 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/dom-level1-core/test_documentinvalidcharacterexceptioncreateentref1.html | throw_NOT_SUPPORTED_ERR
323 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/dom-level1-core/test_documentinvalidcharacterexceptioncreatepi.html | throw_NOT_SUPPORTED_ERR
327 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/dom-level1-core/test_documentinvalidcharacterexceptioncreatepi1.html | throw_NOT_SUPPORTED_ERR
500 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/dom-level1-core/test_elementwrongdocumenterr.html | throw_WRONG_DOCUMENT_ERR
536 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/dom-level1-core/test_hc_attrappendchild5.html | throw_WRONG_DOCUMENT_ERR
581 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/dom-level1-core/test_hc_attrgetvalue2.html | throw_NOT_SUPPORTED_ERR
607 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/dom-level1-core/test_hc_attrinsertbefore6.html | throw_WRONG_DOCUMENT_ERR
702 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/dom-level1-core/test_hc_characterdataindexsizeerrdeletedatacountnegative.html | throws_INDEX_SIZE_ERR
718 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/dom-level1-core/test_hc_characterdataindexsizeerrreplacedatacountnegative.html | throws_INDEX_SIZE_ERR
728 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/dom-level1-core/test_hc_characterdataindexsizeerrsubstringcountnegative.html | throws_INDEX_SIZE_ERR
958 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/dom-level1-core/test_hc_elementnotfounderr.html | throw_NOT_FOUND_ERR
962 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/dom-level1-core/test_hc_elementremoveattribute.html | attrValue - got , expected null
996 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/dom-level1-core/test_hc_elementwrongdocumenterr.html | throw_WRONG_DOCUMENT_ERR
1054 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/dom-level1-core/test_hc_namednodemapwrongdocumenterr.html | throw_WRONG_DOCUMENT_ERR
1094 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/dom-level1-core/test_hc_nodeappendchildnewchilddiffdocument.html | throw_WRONG_DOCUMENT_ERR
1271 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/dom-level1-core/test_hc_nodeinsertbeforenewchilddiffdocument.html | throw_WRONG_DOCUMENT_ERR
1382 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/dom-level1-core/test_hc_nodereplacechildnewchilddiffdocument.html | throw_WRONG_DOCUMENT_ERR
1424 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/dom-level1-core/test_hc_nodevalue03.html | throw_NOT_SUPPORTED_ERR
1530 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/dom-level1-core/test_namednodemapwrongdocumenterr.html | throw_WRONG_DOCUMENT_ERR
1550 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/dom-level1-core/test_nodeappendchildnewchilddiffdocument.html | throw_WRONG_DOCUMENT_ERR
1770 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/dom-level1-core/test_nodeinsertbeforenewchilddiffdocument.html | throw_WRONG_DOCUMENT_ERR
1895 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/tests/mochitest/dom-level1-core/test_nodereplacechildnewchilddiffdocument.html | throw_WRONG_DOCUMENT_ERR
Comment 14 Serge Gautherie (:sgautherie) 2012-03-29 00:20:51 PDT
(In reply to Serge Gautherie (:sgautherie) from comment #0)

> Hopefully, it's just the first test causing failures in the following ones...

This is a log regression that happens in all runs,
though the tests are behaving (and are managed) as (currently) expected :-<

Sorry I didn't notice it in the first place, because these "failures" are actually morphed into "todos" wrt result counts :-|

> This also causes failures in 1+ (very) later test(s).
> 
> Need to investigate that too/first...

These 2 random failing cases are actually bug 696306 and bug 739557 !


(In reply to TinderboxPushlog Robot from comment #13)
> https://tbpl.mozilla.org/php/getParsedLog.php?id=10452825&tree=Mozilla-
> Inbound
> Android XUL Tegra 250 mozilla-inbound opt test mochitest-3 on 2012-03-28
> 15:59:16

This occurrence is this log regression alone :->
Comment 15 Mozilla RelEng Bot 2012-03-29 09:05:34 PDT
Try run for a78fe53feb56 is complete.
Detailed breakdown of the results available here:
    https://tbpl.mozilla.org/?tree=Try&rev=a78fe53feb56
Results (out of 31 total builds):
    success: 17
    warnings: 14
Builds (or logs if builds failed) available at:
http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/sgautherie.bz@free.fr-a78fe53feb56
Comment 16 Serge Gautherie (:sgautherie) 2012-03-29 09:11:48 PDT
(In reply to Mozilla RelEng Bot from comment #15)
>     https://tbpl.mozilla.org/?tree=Try&rev=a78fe53feb56

Bah, stupid typo :-<
Comment 17 Mozilla RelEng Bot 2012-03-29 11:32:46 PDT
Try run for ac524fae3fb6 is complete.
Detailed breakdown of the results available here:
    https://tbpl.mozilla.org/?tree=Try&rev=ac524fae3fb6
Results (out of 31 total builds):
    success: 30
    warnings: 1
Builds (or logs if builds failed) available at:
http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/sgautherie.bz@free.fr-ac524fae3fb6
Comment 18 Mozilla RelEng Bot 2012-03-29 15:17:39 PDT
Try run for 49b05527e882 is complete.
Detailed breakdown of the results available here:
    https://tbpl.mozilla.org/?tree=Try&rev=49b05527e882
Results (out of 31 total builds):
    success: 30
    warnings: 1
Builds (or logs if builds failed) available at:
http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/sgautherie.bz@free.fr-49b05527e882
Comment 19 Serge Gautherie (:sgautherie) 2012-03-29 15:53:51 PDT
Created attachment 610732 [details] [diff] [review]
(Av1) dom-level*-*/DOMTestCase.js: Fix SimpleTest._logResult() override + testFails() + markTodos(), Fake a missing warn() function in checkInitialization(), Add missing early returns to ...
Comment 20 Rob Campbell [:rc] (:robcee) 2012-03-30 09:29:38 PDT
Comment on attachment 610732 [details] [diff] [review]
(Av1) dom-level*-*/DOMTestCase.js: Fix SimpleTest._logResult() override + testFails() + markTodos(), Fake a missing warn() function in checkInitialization(), Add missing early returns to ...

I don't have time for these right now. Can we get one of the other testing peers to take a look at this? Thanks.
Comment 21 Joel Maher (:jmaher) 2012-03-30 10:12:03 PDT
Comment on attachment 610732 [details] [diff] [review]
(Av1) dom-level*-*/DOMTestCase.js: Fix SimpleTest._logResult() override + testFails() + markTodos(), Fake a missing warn() function in checkInitialization(), Add missing early returns to ...

Review of attachment 610732 [details] [diff] [review]:
-----------------------------------------------------------------

I don't understand why we are removing the logging function from dom-level2-html.  I also have a lot of questions below.

::: dom/tests/mochitest/dom-level1-core/DOMTestCase.js
@@ +35,4 @@
>      ok(actual !== null, descr);
> +    // Work around too strict checks.
> +    if (!actual) {
> +      ok(actual, "[assertSize()] 'actual' has a value");

not sure I understand the point of this ok() check.

@@ +188,5 @@
>    function assertURIEquals(assertID, scheme, path, host, file, name, query, fragment, isAbsolute, actual) {
>      //
>      //  URI must be non-null
> +    ok(assertID, "[assertURIEquals()] 'assertID' has no value");
> +    ok(actual, "[assertURIEquals()] 'actual' has no value");

how does this ok(actual...) call differ from the one in assertSize()?  The error messages are different indicating this one has no value and the one in assertSize has a value.

@@ +193,5 @@
> +/*
> +    // Add missing early return.
> +    if (!actual)
> +      return;
> +*/

why is this commented out?

::: dom/tests/mochitest/dom-level2-core/DOMTestCase.js
@@ +38,5 @@
> +    if (!actual) {
> +      ok(actual, "[assertSize()] 'actual' has a value");
> +      return;
> +    }
> +*/

why are we adding code inside a comment here?  Shouldn't we do what we do for dom-level1-core?

@@ +197,5 @@
> +/*
> +    // Add missing early return.
> +    if (!actual)
> +      return;
> +*/

same question here?  why the comment?

::: dom/tests/mochitest/dom-level2-html/DOMTestCase.js
@@ +38,5 @@
> +    if (!actual) {
> +      ok(actual, "[assertSize()] 'actual' has a value");
> +      return;
> +    }
> +*/

why the comments?

@@ +197,5 @@
> +/*
> +    // Add missing early return.
> +    if (!actual)
> +      return;
> +*/

why the comments?

@@ +683,2 @@
>  }
> +*/

why is this entire block commented out?
Comment 22 Serge Gautherie (:sgautherie) 2012-03-30 14:22:41 PDT
Created attachment 611029 [details] [diff] [review]
(Av2) dom-level*-*/DOMTestCase.js: Fix SimpleTest._logResult() override + testFails() + markTodos(), Fake a missing warn() function in checkInitialization(), ...
[Checked in: Comment 24]

Av1, with comment 21 suggestion(s).


(In reply to Joel Maher (:jmaher) from comment #21)

> I don't understand why we are removing the logging function from
> dom-level2-html.

I'm not: its feature was commented out.

> >      ok(actual !== null, descr);
> > +    // Work around too strict checks.
> > +    if (!actual) {
> > +      ok(actual, "[assertSize()] 'actual' has a value");
> 
> not sure I understand the point of this ok() check.

To log why the early return, despite the previous/existing ok().

> how does this ok(actual...) call differ from the one in assertSize()?  The
> error messages are different indicating this one has no value and the one in
> assertSize has a value.

Right, messages fixed.

> @@ +193,5 @@
> > +/*
> > +    // Add missing early return.
> > +    if (!actual)
> > +      return;
> > +*/
> 
> why is this commented out?

Because, in the few logs I checked, we're not hitting this case,
yet I'm "documenting" this code flaw, for future use if need be.

> why are we adding code inside a comment here?  Shouldn't we do what we do
> for dom-level1-core?

Because, in the few logs I checked, we're not hitting this case,
yet I'm maintaining some code parity between the 3 files, for future use if need be.

> @@ +197,5 @@
> > +/*
> > +    // Add missing early return.
> > +    if (!actual)
> > +      return;
> > +*/
> 
> same question here?  why the comment?

Same answer. To give a hint that it's not existing code that was commented out, but new code to use if need be.

> @@ +683,2 @@
> >  }
> > +*/
> 
> why is this entire block commented out?

Because there is no ToDo tests in dom-level2-html atm.
Comment 23 Joel Maher (:jmaher) 2012-04-02 07:53:04 PDT
Comment on attachment 611029 [details] [diff] [review]
(Av2) dom-level*-*/DOMTestCase.js: Fix SimpleTest._logResult() override + testFails() + markTodos(), Fake a missing warn() function in checkInitialization(), ...
[Checked in: Comment 24]

Review of attachment 611029 [details] [diff] [review]:
-----------------------------------------------------------------

thanks.
Comment 24 Serge Gautherie (:sgautherie) 2012-04-02 08:17:00 PDT
Comment on attachment 611029 [details] [diff] [review]
(Av2) dom-level*-*/DOMTestCase.js: Fix SimpleTest._logResult() override + testFails() + markTodos(), Fake a missing warn() function in checkInitialization(), ...
[Checked in: Comment 24]

https://hg.mozilla.org/mozilla-central/rev/85a31c1e4963
Comment 25 Serge Gautherie (:sgautherie) 2012-04-02 11:26:56 PDT
https://tbpl.mozilla.org/php/getParsedLog.php?id=10566168&tree=Firefox
Rev3 Fedora 12 mozilla-central opt test mochitests-3/5 on 2012-04-02 08:23:49 PDT for push 85a31c1e4963

V.Fixed

Note You need to log in before you can comment on or make changes to this bug.