1559494 - Perma [tier2] /html/cross-origin-opener/new_window_null.tentative.html | null document opening popup to http://web-platform.test:8000 with COOP: "same-site" - assert_equals: expected "" but got "null_to_SAME_ORIGIN_same-site"

Status: RESOLVED WORKSFORME

(Core :: DOM: Networking, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: dvarga [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=251936853&repo=mozilla-central
Full log: https://queue.taskcluster.net/v1/task/RXbJEKLlQLyzaIi_Qv0NNw/runs/0/artifacts/public/logs/live_backing.log

---

[task 2019-06-14T19:48:49.521Z] 19:48:49     INFO - TEST-START | /fetch/corb/script-resource-with-json-parser-breaker.tentative.sub.html
[task 2019-06-14T19:48:49.522Z] 19:48:49     INFO - Closing window 2147483673
[task 2019-06-14T19:48:49.719Z] 19:48:49  WARNING - Traceback (most recent call last):
[task 2019-06-14T19:48:49.719Z] 19:48:49  WARNING -   File "/builds/worker/workspace/build/tests/web-platform/tests/tools/wptrunner/wptrunner/executors/executormarionette.py", line 624, in _run
[task 2019-06-14T19:48:49.719Z] 19:48:49  WARNING -     self.result = True, self.func(self.protocol, self.url, self.timeout)
[task 2019-06-14T19:48:49.719Z] 19:48:49  WARNING -   File "/builds/worker/workspace/build/tests/web-platform/tests/tools/wptrunner/wptrunner/executors/executormarionette.py", line 738, in do_testharness
[task 2019-06-14T19:48:49.719Z] 19:48:49  WARNING -     self.script_resume % format_map, async=True)
[task 2019-06-14T19:48:49.719Z] 19:48:49  WARNING -   File "/builds/worker/workspace/build/tests/web-platform/tests/tools/wptrunner/wptrunner/executors/executormarionette.py", line 61, in execute_script
[task 2019-06-14T19:48:49.719Z] 19:48:49  WARNING -     return method(script, new_sandbox=False, sandbox=None)
[task 2019-06-14T19:48:49.719Z] 19:48:49  WARNING -   File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/marionette.py", line 1656, in execute_async_script
[task 2019-06-14T19:48:49.719Z] 19:48:49  WARNING -     rv = self._send_message("WebDriver:ExecuteAsyncScript", body, key="value")
[task 2019-06-14T19:48:49.719Z] 19:48:49  WARNING -   File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/decorators.py", line 26, in _
[task 2019-06-14T19:48:49.719Z] 19:48:49  WARNING -     return func(*args, **kwargs)
[task 2019-06-14T19:48:49.719Z] 19:48:49  WARNING -   File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/marionette.py", line 598, in _send_message
[task 2019-06-14T19:48:49.719Z] 19:48:49  WARNING -     self._handle_error(err)
[task 2019-06-14T19:48:49.719Z] 19:48:49  WARNING -   File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_driver/marionette.py", line 618, in _handle_error
[task 2019-06-14T19:48:49.719Z] 19:48:49  WARNING -     raise errors.lookup(error)(message, stacktrace=stacktrace)
[task 2019-06-14T19:48:49.719Z] 19:48:49  WARNING - JavascriptException: TypeError: window.__wptrunner_process_next_event is not a function
[task 2019-06-14T19:48:49.719Z] 19:48:49  WARNING - stacktrace:
[task 2019-06-14T19:48:49.719Z] 19:48:49  WARNING - 	@tests/web-platform/tests/tools/wptrunner/wptrunner/executors/executormarionette.py:66:8
[task 2019-06-14T19:48:49.719Z] 19:48:49  WARNING - 	@tests/web-platform/tests/tools/wptrunner/wptrunner/executors/executormarionette.py:67:8
[task 2019-06-14T19:48:49.719Z] 19:48:49  WARNING - 
[task 2019-06-14T19:48:49.719Z] 19:48:49  WARNING - 
[task 2019-06-14T19:48:49.936Z] 19:48:49     INFO - TEST-ERROR | /fetch/corb/script-resource-with-json-parser-breaker.tentative.sub.html | took 412ms
[task 2019-06-14T19:48:50.563Z] 19:48:50     INFO - Closing logging queue
[task 2019-06-14T19:48:50.563Z] 19:48:50     INFO - queue closed
[task 2019-06-14T19:48:50.579Z] 19:48:50     INFO - Setting up ssl
[task 2019-06-14T19:48:50.587Z] 19:48:50     INFO - certutil | 
[task 2019-06-14T19:48:50.607Z] 19:48:50     INFO - certutil | 
[task 2019-06-14T19:48:50.614Z] 19:48:50     INFO - certutil | 
[task 2019-06-14T19:48:50.614Z] 19:48:50     INFO - Certificate Nickname                                         Trust Attributes
[task 2019-06-14T19:48:50.614Z] 19:48:50     INFO -                                                              SSL,S/MIME,JAR/XPI
[task 2019-06-14T19:48:50.614Z] 19:48:50     INFO - 
[task 2019-06-14T19:48:50.614Z] 19:48:50     INFO - web-platform-tests                                           CT,, 
[task 2019-06-14T19:48:50.614Z] 19:48:50     INFO - 
[task 2019-06-14T19:48:52.320Z] 19:48:52     INFO - adb Granting important runtime permissions to org.mozilla.geckoview.test
[task 2019-06-14T19:48:53.562Z] 19:48:53     INFO - adb launch_application: am start -W -n org.mozilla.geckoview.test/org.mozilla.geckoview.test.TestRunnerActivity -a android.intent.action.MAIN --es env9 MOZ_PROCESS_LOG=/tmp/tmpeOJV08pidlog --es env8 MOZ_CRASHREPORTER_NO_REPORT=1 --es args "-no-remote -profile /sdcard/tests/profile --marionette about:blank" --es env3 STYLO_THREADS=4 --es env2 MOZ_HIDE_RESULTS_TABLE=1 --es env1 R_LOG_VERBOSE=1 --es env0 MOZ_CRASHREPORTER=1 --es env7 MOZ_DISABLE_NONLOCAL_CONNECTIONS=1 --es env6 R_LOG_DESTINATION=stderr --es env5 MOZ_CRASHREPORTER_SHUTDOWN=1 --es env4 MOZ_LOG=signaling:3,mtransport:4,DataChannel:4,jsep:4 --ez use_multiprocess True --es env10 R_LOG_LEVEL=6
[task 2019-06-14T19:48:54.986Z] 19:48:54     INFO - Starting runner

Comment 1

[Daniel Varga [:dvarga]]

:valentin, can you please take a look?

Comment 2

[Valentin Gosu [:valentin] (he/him)]

James, this looks like a WPT testharness error. Any idea why it's happening?

Comment 3

[Valentin Gosu [:valentin] (he/him)]

Nevermind, I just saw that cross-origin-opener failures.
I think the __wptrunner_process_next_event may be a fluke

I think we need to disable cross-origin-opener on non-e10s android.

Comment 4

[Andreea Pavel [:apavel]]

Hi Valentin, are you working on disabling this?

There are 29 failures in the last 7 days on android-em-7-0-x86 opt and android-em-7-0-x86_64 debug and opt

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=252121659&repo=mozilla-central&lineNumber=4411

[task 2019-06-16T21:15:16.643Z] 21:15:16 INFO - TEST-START | /fetch/corb/style-html-correctly-labeled.sub.html
[task 2019-06-16T21:15:16.646Z] 21:15:16 INFO - Closing window 2147483659
[task 2019-06-16T21:15:17.316Z] 21:15:17 INFO - .
[task 2019-06-16T21:15:17.316Z] 21:15:17 INFO - TEST-OK | /fetch/corb/style-html-correctly-labeled.sub.html | took 670ms
[task 2019-06-16T21:15:17.955Z] 21:15:17 INFO - Closing logging queue
[task 2019-06-16T21:15:17.956Z] 21:15:17 INFO - queue closed
[task 2019-06-16T21:15:17.971Z] 21:15:17 INFO - Setting up ssl
[task 2019-06-16T21:15:17.987Z] 21:15:17 INFO - certutil |
[task 2019-06-16T21:15:18.009Z] 21:15:18 INFO - certutil |
[task 2019-06-16T21:15:18.024Z] 21:15:18 INFO - certutil |
[task 2019-06-16T21:15:18.024Z] 21:15:18 INFO - Certificate Nickname Trust Attributes
[task 2019-06-16T21:15:18.024Z] 21:15:18 INFO - SSL,S/MIME,JAR/XPI
[task 2019-06-16T21:15:18.025Z] 21:15:18 INFO -
[task 2019-06-16T21:15:18.025Z] 21:15:18 INFO - web-platform-tests CT,,
[task 2019-06-16T21:15:18.025Z] 21:15:18 INFO -
[task 2019-06-16T21:15:19.749Z] 21:15:19 INFO - adb Granting important runtime permissions to org.mozilla.geckoview.test
[task 2019-06-16T21:15:20.990Z] 21:15:20 INFO - adb launch_application: am start -W -n org.mozilla.geckoview.test/org.mozilla.geckoview.test.TestRunnerActivity -a android.intent.action.MAIN --es env9 MOZ_PROCESS_LOG=/tmp/tmpcd6ycHpidlog --es env8 MOZ_CRASHREPORTER_NO_REPORT=1 --es args "-no-remote -profile /sdcard/tests/profile --marionette about:blank" --es env3 STYLO_THREADS=4 --es env2 MOZ_HIDE_RESULTS_TABLE=1 --es env1 R_LOG_VERBOSE=1 --es env0 MOZ_CRASHREPORTER=1 --es env7 MOZ_DISABLE_NONLOCAL_CONNECTIONS=1 --es env6 R_LOG_DESTINATION=stderr --es env5 MOZ_CRASHREPORTER_SHUTDOWN=1 --es env4 MOZ_LOG=signaling:3,mtransport:4,DataChannel:4,jsep:4 --ez use_multiprocess True --es env10 R_LOG_LEVEL=6
[task 2019-06-16T21:15:22.726Z] 21:15:22 INFO - Starting runner
[task 2019-06-16T21:15:23.099Z] 21:15:23 INFO - TEST-START | /html/cross-origin-opener/new_window_null.tentative.html
[task 2019-06-16T21:15:23.120Z] 21:15:23 INFO - Setting pref browser.tabs.remote.useCrossOriginOpenerPolicy (true)
[task 2019-06-16T21:15:32.115Z] 21:15:32 INFO -
[task 2019-06-16T21:15:32.115Z] 21:15:32 INFO - TEST-PASS | /html/cross-origin-opener/new_window_null.tentative.html | null document opening popup to http://web-platform.test:8000 with COOP: ""
[task 2019-06-16T21:15:32.115Z] 21:15:32 INFO - TEST-PASS | /html/cross-origin-opener/new_window_null.tentative.html | null document opening popup to http://web-platform.test:8000 with COOP: "jibberish"
[task 2019-06-16T21:15:32.115Z] 21:15:32 INFO - TEST-UNEXPECTED-FAIL | /html/cross-origin-opener/new_window_null.tentative.html | null document opening popup to http://web-platform.test:8000 with COOP: "same-site" - assert_equals: expected "" but got "null_to_SAME_ORIGIN_same-site"
[task 2019-06-16T21:15:32.115Z] 21:15:32 INFO - coop_test/bc.onmessage<@http://web-platform.test:8000/html/cross-origin-opener/common.sub.js:9:5
[task 2019-06-16T21:15:32.115Z] 21:15:32 INFO - Test.prototype.step@http://web-platform.test:8000/resources/testharness.js:1594:25
[task 2019-06-16T21:15:32.115Z] 21:15:32 INFO - Test.prototype.step_func_done/<@http://web-platform.test:8000/resources/testharness.js:1634:32
[task 2019-06-16T21:15:32.115Z] 21:15:32 INFO - coop_test@http://web-platform.test:8000/html/cross-origin-opener/common.sub.js:13:18
[task 2019-06-16T21:15:32.115Z] 21:15:32 INFO - run_coop_tests/<@http://web-platform.test:8000/html/cross-origin-opener/common.sub.js:23:5
[task 2019-06-16T21:15:32.115Z] 21:15:32 INFO - Test.prototype.step@http://web-platform.test:8000/resources/testharness.js:1594:25
[task 2019-06-16T21:15:32.115Z] 21:15:32 INFO - async_test@http://web-platform.test:8000/resources/testharness.js:576:22
[task 2019-06-16T21:15:32.115Z] 21:15:32 INFO - run_coop_tests@http://web-platform.test:8000/html/cross-origin-opener/common.sub.js:22:4
[task 2019-06-16T21:15:32.115Z] 21:15:32 INFO - @http://web-platform.test:8000/html/cross-origin-opener/new_window_null.tentative.html:36:1

Comment 5

[Intermittent Failures Robot]

36 failures in 4753 pushes (0.008 failures/push) were associated with this bug in the last 7 days.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 35
• try: 1

Platform breakdown:

• android-em-7-0-x86_64: 26
• linux64-qr: 1
• android-em-7-0-x86: 9

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1559494&startday=2019-06-10&endday=2019-06-16&tree=all

Comment 6

[Intermittent Failures Robot]

39 failures in 614 pushes (0.064 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 21
• try: 3
• autoland: 15

Platform breakdown:

• android-em-7-0-x86: 13
• android-em-7-0-x86_64: 26

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1559494&startday=2019-06-19&endday=2019-06-19&tree=all

Comment 7

[Andreea Pavel [:apavel]]

Attachment: Bug 1559494 - disabled new_window_null.tentative.html on android r=jmaher

Comment 8

[Intermittent Failures Robot]

58 failures in 562 pushes (0.103 failures/push) were associated with this bug yesterday.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• autoland: 46
• mozilla-central: 6
• try: 6

Platform breakdown:

• android-em-7-0-x86: 19
• android-em-7-0-x86_64: 39

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1559494&startday=2019-06-20&endday=2019-06-20&tree=all

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/440efb54ea20
disabled new_window_null.tentative.html on android r=jmaher

Comment 10

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/mozilla-central/rev/440efb54ea20

Comment 11

[Intermittent Failures Robot]

119 failures in 2581 pushes (0.046 failures/push) were associated with this bug in the last 7 days.

This is the #8 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 48
• autoland: 62
• try: 9

Platform breakdown:

• android-em-7-0-x86: 39
• android-em-7-0-x86_64: 79
• linux64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1559494&startday=2019-06-17&endday=2019-06-23&tree=all

Comment 12

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/17476 for changes under testing/web-platform/tests

Comment 13

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR merged

Comment 14

[Josh Matthews [:jdm]]

The .ini file was added to the wrong location and so the test is not actually disabled. It belongs under testing/web-platform/meta/html/cross-origin-opener/ instead of testing/web-platform/tests/html/cross-origin-opener.

Comment 15

[Andreea Pavel [:apavel]]

Attachment: Bug 1559494 - disable new_window_null.tentative.html on android under the correct path r=jdm

Comment 16

[Andreea Pavel [:apavel]]

Thanks Josh, please take a look.

Comment 17

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ef772f4c6eec
disable new_window_null.tentative.html on android under the correct path r=jdm

Comment 18

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/17498 for changes under testing/web-platform/tests

Comment 19

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream web-platform-tests status checks passed, PR will merge once commit reaches central.

Comment 20

[Arthur Iakab [arthur_iakab]]

https://hg.mozilla.org/mozilla-central/rev/ef772f4c6eec

Comment 21

[Valentin Gosu [:valentin] (he/him)]

I suspect the test fails on Android because we only have one content process, and a cross-process redirect ends up in the same process as before.

Comment 22

[Intermittent Failures Robot]

2 failures in 3501 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 2

Platform breakdown:

• android-em-7-0-x86_64: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1559494&startday=2019-07-01&endday=2019-07-07&tree=all

Comment 23

[Kershaw Chang [:kershaw]]

(In reply to PTO until September 9th from comment #21)

I suspect the test fails on Android because we only have one content process, and a cross-process redirect ends up in the same process as before.

Junior, could you take a look? Thanks.

Comment 24

[Junior [inactive]]

Yes. Set P5 with coop in android. Might be WORKSFORME given comment 21

Comment 25

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/a56991306ec8
[wpt PR 17476] - [Gecko Bug 1559494] disabled new_window_null.tentative.html on android, a=testonly
https://hg.mozilla.org/integration/mozilla-inbound/rev/53bb11307763
[wpt PR 17498] - [Gecko Bug 1559494] disable new_window_null.tentative.html on android under the correct path, a=testonly

Comment 26

[Razvan Maries]

https://hg.mozilla.org/mozilla-central/rev/a56991306ec8
https://hg.mozilla.org/mozilla-central/rev/53bb11307763

Comment 27

[Intermittent Failures Robot]

5 failures in 4666 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1
• autoland: 4

Platform breakdown:

• linux64-qr: 1
• linux64: 2
• macosx1014-64: 1
• windows10-64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1559494&startday=2019-08-05&endday=2019-08-11&tree=all

Comment 28

[Intermittent Failures Robot]

16 failures in 1102 pushes (0.015 failures/push) were associated with this bug yesterday.

Repository breakdown:

• try: 16

Platform breakdown:

• macosx1014-64: 2
• linux64-qr: 11
• linux64-shippable: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1559494&startday=2019-08-13&endday=2019-08-13&tree=all

Comment 29

[Intermittent Failures Robot]

16 failures in 4839 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 16

Platform breakdown:

• macosx1014-64: 2
• linux64-qr: 11
• linux64-shippable: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1559494&startday=2019-08-12&endday=2019-08-18&tree=all

Comment 30

[Junior [inactive]]

IMO it's fine to close now