run w3c DOM suites for automated test coverage

VERIFIED FIXED in mozilla1.9alpha3

Status

()

Core
DOM
VERIFIED FIXED
11 years ago
7 years ago

People

(Reporter: Robert Sayre, Assigned: Robert Sayre)

Tracking

Trunk
mozilla1.9alpha3
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments, 3 obsolete attachments)

(Assignee)

Description

11 years ago
They store their tests in an XML format, and then transform it. We'll need to adapt their XSLT to produce tests that we can run in mochikit, instead of whatever JS lib they use.
(Assignee)

Comment 1

11 years ago
Created attachment 248628 [details]
test-to-mochikit.xslt

quick and dirty version that gets some Level 1 Core stuff working.
Assignee: general → sayrer
Status: NEW → ASSIGNED
(Assignee)

Updated

11 years ago
Attachment #248628 - Attachment is patch: false
(Assignee)

Comment 2

11 years ago
Created attachment 248680 [details] [diff] [review]
new and improved
Attachment #248628 - Attachment is obsolete: true
I'm all for dropping the use of JsUnit in the DOM tests and transforming the raw XML test descriptions into something else, but if you are keeping JsUnit etc what benefit do you get from mochikit?
(Assignee)

Comment 4

11 years ago
I need to attach some more files. I deleted half of DOMTestCase.js and wrote some shims that map assertEquals and friends to the MochiKit stuff. This means all the JSUnit-esque assertions go through the MochiKit logger (and thus out to files, stdio, etc).
(Assignee)

Comment 5

11 years ago
Created attachment 253212 [details] [diff] [review]
generate MochiKit from W3C dom-level1-core tests
Attachment #248680 - Attachment is obsolete: true
(Assignee)

Comment 6

11 years ago
Created attachment 253214 [details] [diff] [review]
generate MochiKit from W3C dom-level1-core tests 

this patch doesn't include all of the generated test files, but there are enough there to get the idea. 

The exclusions.js file converts failing tests to todo() calls in the named files. Most of the ones we fail are DTD-related, but there are others.
Attachment #253212 - Attachment is obsolete: true
Attachment #253214 - Flags: review?(jst)
Comment on attachment 253214 [details] [diff] [review]
generate MochiKit from W3C dom-level1-core tests 

Seems like Bob would be a more appropriate reviewer for this. I'll happily sr if there's a need for that here.
Attachment #253214 - Flags: review?(jst) → review?(bclary)
(Assignee)

Comment 8

11 years ago
Created attachment 253779 [details]
bz2 of patch
following up comment 3:

The patch takes the approach of taking the output of the W3C DOM Test Suite build process, modifying the resulting html files to use a modified version of the DOMTestCase.js with mochikit replacing JsUnit. The result is a static set of pages forked from the W3C DOM Test Suite that contains code that is copyright W3C and licensed under the W3C license.

I'm not particularly in love with the W3C DOM Test Suite or JsUnit, but I don't see the major benefit in going down this path. I do have the means to run the output of the W3C DOM Test Suite build process automatically and to either generate the test results as text files or by submitting the individual test results to a web service. As soon as we can get more of the buildbot infrastructure deployed, it will be relatively easy to execute the suite and collect the results.

If mochikit is desired, I would think that we could standardize on using it for our new mozilla.org dom tests and leave the W3C DOM Test Suite as is. 

If we must replace JsUnit, then we should take the approach that the W3C DOM Test Suite build process uses which is taking the raw xml format description of the testcases and transform them into mochikit based tests. This would alleviate any issue of forking or checking in W3C copyrighted materials into CVS.

Just my opinion.
(Assignee)

Comment 10

11 years ago
Requirements:

* no dependencies other than what is already required by our build system 
* must be runnable by developers with that setup
* must report to the console
* must run automated

I am not going to get involved with JsUnit, so anyone who wants to use that should assign this bug to themselves.
Summary: convert w3c DOM suite to mochikit → run w3c DOM suites for automated test coverage
(Assignee)

Updated

11 years ago
Assignee: sayrer → general
Status: ASSIGNED → NEW
(Assignee)

Comment 11

11 years ago
oops, forgot one requirement:

* we don't pass all of the DOM suite, and it's unlikely we ever will, so the
  harness must have a facility for "todo" or "expected fail".
ignore me, do whatever you want.
Attachment #253214 - Flags: review?(bclary) → review?(rcampbell)
Comment on attachment 253214 [details] [diff] [review]
generate MochiKit from W3C dom-level1-core tests 

let's hook it up and try it.
Attachment #253214 - Flags: review?(rcampbell) → review+
(Assignee)

Updated

11 years ago
Status: NEW → RESOLVED
Last Resolved: 11 years ago
OS: Mac OS X → All
Hardware: PC → All
Resolution: --- → FIXED
Target Milestone: --- → mozilla1.9alpha3
Depends on: 483992

Updated

7 years ago
Assignee: general → sayrer
QA Contact: ian → general

Updated

7 years ago
Status: RESOLVED → VERIFIED
Depends on: 628974
You need to log in before you can comment on or make changes to this bug.