1522055 - Permanent failure message shown (even for passing jobs) "JavascriptException: SecurityError: Permission denied to access property "testdriver_callback" on cross-origin object" DON'T USE FOR CLASSIFICATION

Status: NEW

(Testing :: web-platform-tests, defect, P5)

Description

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=9474df714baf1145fe9cdb2ba8e659b4ac7a7404&searchStr=wpt&group_state=expanded&selectedJob=223459830 has permanent failure messages also seen for other pushes, e.g. for this test:

[task 2019-01-12T22:17:50.543Z] 22:17:50 INFO - TEST-START | /trusted-types/block-string-assignment-to-Location-assign.tentative.html
[task 2019-01-12T22:17:50.546Z] 22:17:50 INFO - Closing window 4294967417
[task 2019-01-12T22:17:50.927Z] 22:17:50 INFO - PID 19463 | JavaScript error: http://web-platform.test:8000/trusted-types/block-string-assignment-to-Location-assign.tentative.html, line 39: TypeError: window.TrustedTypes is undefined
[task 2019-01-12T22:17:51.126Z] 22:17:51 WARNING - Traceback (most recent call last):
[task 2019-01-12T22:17:51.127Z] 22:17:51 WARNING - File "/builds/worker/workspace/build/tests/web-platform/tests/tools/wptrunner/wptrunner/executors/executormarionette.py", line 588, in _run
[task 2019-01-12T22:17:51.127Z] 22:17:51 WARNING - self.result = True, self.func(self.protocol, self.url, self.timeout)
[task 2019-01-12T22:17:51.128Z] 22:17:51 WARNING - File "/builds/worker/workspace/build/tests/web-platform/tests/tools/wptrunner/wptrunner/executors/executormarionette.py", line 706, in do_testharness
[task 2019-01-12T22:17:51.131Z] 22:17:51 WARNING - self.script_resume % format_map, async=True)
[task 2019-01-12T22:17:51.132Z] 22:17:51 WARNING - File "/builds/worker/workspace/build/tests/web-platform/tests/tools/wptrunner/wptrunner/executors/executormarionette.py", line 60, in execute_script
[task 2019-01-12T22:17:51.135Z] 22:17:51 WARNING - return method(script, new_sandbox=False, sandbox=None)
[task 2019-01-12T22:17:51.136Z] 22:17:51 WARNING - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/marionette.py", line 1819, in execute_async_script
[task 2019-01-12T22:17:51.136Z] 22:17:51 WARNING - rv = self._send_message("WebDriver:ExecuteAsyncScript", body, key="value")
[task 2019-01-12T22:17:51.137Z] 22:17:51 WARNING - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/decorators.py", line 26, in _
[task 2019-01-12T22:17:51.137Z] 22:17:51 WARNING - return func(*args, **kwargs)
[task 2019-01-12T22:17:51.137Z] 22:17:51 WARNING - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/marionette.py", line 764, in _send_message
[task 2019-01-12T22:17:51.137Z] 22:17:51 WARNING - self._handle_error(err)
[task 2019-01-12T22:17:51.137Z] 22:17:51 WARNING - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/marionette.py", line 784, in _handle_error
[task 2019-01-12T22:17:51.138Z] 22:17:51 WARNING - raise errors.lookup(error)(message, stacktrace=stacktrace)
[task 2019-01-12T22:17:51.138Z] 22:17:51 WARNING - JavascriptException: SecurityError: Permission denied to access property "testdriver_callback" on cross-origin object
[task 2019-01-12T22:17:51.138Z] 22:17:51 WARNING - stacktrace:
[task 2019-01-12T22:17:51.138Z] 22:17:51 WARNING - @tests/web-platform/tests/tools/wptrunner/wptrunner/executors/executormarionette.py:62:1
[task 2019-01-12T22:17:51.138Z] 22:17:51 WARNING - @tests/web-platform/tests/tools/wptrunner/wptrunner/executors/executormarionette.py:60:59
[task 2019-01-12T22:17:51.138Z] 22:17:51 WARNING -
[task 2019-01-12T22:17:51.139Z] 22:17:51 WARNING -
[task 2019-01-12T22:17:51.140Z] 22:17:51 INFO - TEST-ERROR | /trusted-types/block-string-assignment-to-Location-assign.tentative.html | took 603ms

James, can the situation be changed that these are disappear from the failure summary, either by fixing or disabling the tests which trigger that, please?

Comment 1

[James Graham [:jgraham]]

The cross_origin_object thing should be gone since I just landed a wpt sync that changes how the harness works there.

In general removing this kind of logging is hard because there's an impedance mismatch between what the log parser does and what people want it to do. What it does is pulls out lines that match certain regexp. What people want is for it to show all the lines that led to the job turning orange. In the wpt case these are not aligned because some kinds of error we simply have to ignore for the purposes of turning the job orange (e.g. we have "expected" crashes). So I don't really have a general solution for you here.

Comment 2

[James Graham [:jgraham]]

Setting to P5 since the general issue is mostly long slow cleanup of a lot of tests.