1089034 - Invalid mochitests skipped in dom/imptests/failures/*/mochitest.ini

Status: RESOLVED DUPLICATE of bug 1194860

(Core :: DOM: Core & HTML, defect, P5)

Description

[Kelsey Gilbert [:jgilbert]]

> 16:07:15     INFO -  Warning: test_event.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/editing/conformancetest/mochitest.ini is not a valid test
> 16:07:15     INFO -  Warning: test_runtest.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/editing/conformancetest/mochitest.ini is not a valid test
> 16:07:15     INFO -  Warning: test_Document-open.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/editing/selecttest/mochitest.ini is not a valid test
> 16:07:15     INFO -  Warning: test_addRange.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/editing/selecttest/mochitest.ini is not a valid test
> 16:07:15     INFO -  Warning: test_collapse.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/editing/selecttest/mochitest.ini is not a valid test
> 16:07:15     INFO -  Warning: test_collapseToStartEnd.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/editing/selecttest/mochitest.ini is not a valid test
> 16:07:15     INFO -  Warning: test_extend.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/editing/selecttest/mochitest.ini is not a valid test
> 16:07:15     INFO -  Warning: test_getSelection.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/editing/selecttest/mochitest.ini is not a valid test
> 16:07:15     INFO -  Warning: test_removeAllRanges.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/editing/selecttest/mochitest.ini is not a valid test
> 16:07:15     INFO -  Warning: test_selectAllChildren.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/editing/selecttest/mochitest.ini is not a valid test
> 16:07:15     INFO -  Warning: test_exceptions.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/dom/errors/mochitest.ini is not a valid test
> 16:07:15     INFO -  Warning: test_DOMTokenList-stringifier.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/dom/lists/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_historical.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/dom/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_interfaces.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/dom/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_Document-createElement-namespace.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/dom/nodes/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_Document-createElementNS.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/dom/nodes/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_Document-getElementsByTagName.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/dom/nodes/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_Node-isEqualNode.xhtml.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/dom/nodes/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_Node-properties.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/dom/nodes/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_attributes.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/dom/nodes/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_case.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/dom/nodes/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_getElementsByClassName-10.xml.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/dom/nodes/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_getElementsByClassName-11.xml.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/dom/nodes/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_window-indexed-properties-strict.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/html/browsers/the-window-object/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_window-named-properties.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/html/browsers/the-window-object/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_window-properties.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/html/browsers/the-window-object/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_window-null-names.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/html/browsers/the-window-object/named-access-on-the-window-object/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_document.getElementsByName-newelements.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/html/dom/documents/dta/doc.gEBN/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_document.title-03.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/html/dom/documents/dta/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_document.title-04.xhtml.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/html/dom/documents/dta/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_document.title-06.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/html/dom/documents/dta/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_document.title-07.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/html/dom/documents/dta/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_nameditem-02.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/html/dom/documents/dta/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_nameditem-03.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/html/dom/documents/dta/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_nameditem-04.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/html/dom/documents/dta/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_nameditem-05.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/html/dom/documents/dta/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_nameditem-06.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/html/dom/documents/dta/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_script-IDL-event-htmlfor.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/html/obsolete/implreq/oeaaa/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_option-text-spaces.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/html/semantics/forms/the-option-element/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_select-named-getter.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/html/semantics/forms/the-select-element/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_select-remove.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/html/semantics/forms/the-select-element/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_script-for-event.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/html/semantics/scripting-1/the-script-element/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_script-language-type.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/html/semantics/scripting-1/the-script-element/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_script-languages-01.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/html/semantics/scripting-1/the-script-element/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_script-languages-02.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/html/semantics/scripting-1/the-script-element/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_table-insertRow.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/html/semantics/tabular-data/the-table-element/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_table-rows.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/html/semantics/tabular-data/the-table-element/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_base64.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/html/webappapis/atob/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_WeakMap.prototype-properties.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/js/builtins/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_001.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/microdata/microdata-dom-api/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_constructors.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/html/typedarrays/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_storage_local_security.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/webapps/WebStorage/tests/submissions/Infraware/mochitest.ini is not a valid test
> 16:07:16     INFO -  Warning: test_interfaces.html.json from manifest /builds/slave/test/build/tests/mochitest/tests/dom/imptests/failures/webapps/XMLHttpRequest/tests/submissions/Ms2ger/mochitest.ini is not a valid test

Comment 1

[Vaibhav (:vaibhav1994)]

I was going through the code and all of the above tests are present in mochitest.ini files that are auto-generated in http://dxr.mozilla.org/mozilla-central/source/dom/imptests/parseFailures.py
All these tests files are present in the path: http://dxr.mozilla.org/mozilla-central/source/dom/imptests/failures

Since all the tests present in the 'dom/imptests/failures' path are skipped, one straight forward solution is to remove the entries in moz.build (http://dxr.mozilla.org/mozilla-central/source/dom/imptests/moz.build#14), but I think there might be a reason why these tests are there.

:Ms2ger, could you let me know if it will be fine to just remove the entries or are the tests used for some other purpose and should remain there.

Comment 2

[:Ms2ger (he/him; ⌚ UTC+1/+2)]

They are not tests, they are test expectations. Unless they're breaking something, leave them alone.

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

Ms2ger, can we make the support files or something else?  Defining them as tests, then spitting out warnings during manifest parsing clutters the log files, and makes it hard to create new rules to help people not waste time doing something wrong (i.e. defining a test incorrectly)

Comment 4

[Vaibhav (:vaibhav1994)]

:Ms2ger, it would be great if you can suggest some way in which we can remove the warnings and still have them as test expectations. The logs will be cleaner if we do so. May be remove just the entries from moz.build or something?

Comment 5

[:Ms2ger (he/him; ⌚ UTC+1/+2)]

If we support support-only manifests now, you can use those, and update the script that generates them.

Comment 6

[Andrew Halberstadt [:ahal]]

So my understanding is that these just need to end up in the test package. If that's the case, using TEST_HARNESS_FILES.mochitest in moz.build is probably the best way forward.

Adding:

TEST_HARNESS_FILES.testing.mochitest.tests.dom.imptests.failures += ['foo.py']

to any moz.build should package foo.py in the proper directory. It may be possible to set this up in just the root moz.build in dom.imptests, but not sure.

Comment 7

[Andrew Halberstadt [:ahal]]

I had to hack around this in bug 1131098 because these "fake tests" were throwing off chunking numbers and causing test order related oranges.

Ms2ger, I'm not sure how these work.. is comment 6 a possibility?

Comment 8

[:Ms2ger (he/him; ⌚ UTC+1/+2)]

Dunno, but I believe we support manifest files with only support-files now?

Comment 9

[Firefox Bug Husbandry Bot]

https://bugzilla.mozilla.org/show_bug.cgi?id=1472046

Move all DOM bugs that haven’t been updated in more than 3 years and has no one currently assigned to P5.

If you have questions, please contact :mdaly.