1338068 - Something is seriously broken in our handling of .any.js tests

Status: RESOLVED FIXED

(Testing :: web-platform-tests, defect)

Description

[Boris Zbarsky [:bzbarsky]]

Bug 1317236 claims to have fixed things, but they look pretty broken to me.

A concrete example: console/console-is-a-namespace.any.js.  As far as I can tell, we think we generate two tests from this: console/console-is-a-namespace.any.worker.html and console/console-is-a-namespace.any.html.  At least that's what ends up in MANIFEST.json.

The second one of these works fine.  It loads a document, that document loads a script at /console/console-is-a-namespace.any.js and everything is dandy.

The first one, however, is busted.  Loading http://web-platform.test:8000/console/console-is-a-namespace.any.worker.html gives an error 500 in the harness.  Presumably this is why that test is annotated as "expected: TIMEOUT".

Loading http://web-platform.test:8000/console/console-is-a-namespace.worker.any.html _does_ produce and HTML file.  But it tries to load /console/console-is-a-namespace.worker.any.js which of course doesn't exist.  This is probably expected; this URL doesn't correspond to anything sane, effectively...

Anyway, the upshot is that as far as I can tell all our .any.js tests are NOT running in workers!

Comment 1

[Boris Zbarsky [:bzbarsky]]

I should note that http://w3c-test.org/console/console-is-a-namespace.any.worker.html is working correctly, so this is something about our local harness...

Comment 2

[Boris Zbarsky [:bzbarsky]]

> gives an error 500 in the harness

To be precise, it's complaining that:

  No such file or directory: u'testing/web-platform/tests/html/dom/self-origin.any.worker.js'

(but with an absolute file path).  But that's not what it should be doing.  It should be spitting out HTML that does:

  fetch_tests_from_worker(new Worker("/html/dom/self-origin.any.worker.js"));

and I should note that loading http://web-platform.test:8000/html/dom/self-origin.any.worker.js does in fact give a file that looks right.

Comment 3

[:Ms2ger (he/him; ⌚ UTC+1/+2)]

Attachment: Correct the metadata handling for .any.js tests in workers

The previous code would try to read a virtual file from the file system, which
would result in an exception in the handler and a 500 Internal Server Error
sent to the client.

This change is part of the code that landed upstream in
<https://github.com/w3c/wpt-tools/pull/156>.

Comment 4

[:Ms2ger (he/him; ⌚ UTC+1/+2)]

This will need expectation updates, but I'm not really setup for generating them. James, could you handle that?

Comment 5

[Boris Zbarsky [:bzbarsky]]

I can confirm that this fixes the "test just doesn't run" problem.  

There are only two existing .any.js tests console/console-is-a-namespace.any.js and XMLHttpRequest/XMLHttpRequest-withCredentials.any.js.  I just tried running both, and they pass completely.  So I think you can just "hg rm testing/web-platform/meta/XMLHttpRequest/XMLHttpRequest-withCredentials.any.js.ini testing/web-platform/meta/console/console-is-a-namespace.any.js.ini" and that's all the expectation update you need.  ;)

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/e647056f169d
Correct the metadata handling for .any.js tests in workers; r=jgraham

Comment 7

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/e647056f169d