1699758 - Perma TEST-UNEXPECTED-PASS | /html/browsers/the-windowproxy-exotic-object/windowproxy-define-own-property-unforgeable-same-origin.html | [[DefineOwnProperty]] success: "window" when Gecko 88 merges to Beta on 2021-03-22

Status: VERIFIED FIXED

(Core :: DOM: Core & HTML, defect)

Description

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

• central-as-beta simulation
• How to run these simulations
• Failure log

[task 2021-03-19T15:33:16.970Z] 15:33:16 INFO - TEST-UNEXPECTED-PASS | /html/browsers/the-windowproxy-exotic-object/windowproxy-define-own-property-unforgeable-same-origin.html | [[DefineOwnProperty]] success: "window" - expected FAIL
[task 2021-03-19T15:33:16.970Z] 15:33:16 INFO - TEST-INFO | expected FAIL
[task 2021-03-19T15:33:16.971Z] 15:33:16 INFO -
[task 2021-03-19T15:33:16.971Z] 15:33:16 INFO - TEST-FAIL | /html/browsers/the-windowproxy-exotic-object/windowproxy-define-own-property-unforgeable-same-origin.html | [[DefineOwnProperty]] failure: "window" - assert_throws_js: [[Configurable]]: true function "() => {
[task 2021-03-19T15:33:16.971Z] 15:33:16 INFO - Object.defineProperty(window, key, { configurable: true });
[task 2021-03-19T15:33:16.971Z] 15:33:16 INFO - }" did not throw
[task 2021-03-19T15:33:16.971Z] 15:33:16 INFO - @http://web-platform.test:8000/html/browsers/the-windowproxy-exotic-object/windowproxy-define-own-property-unforgeable-same-origin.html:28:25
[task 2021-03-19T15:33:16.971Z] 15:33:16 INFO - Test.prototype.step@http://web-platform.test:8000/resources/testharness.js:2095:25
[task 2021-03-19T15:33:16.972Z] 15:33:16 INFO - test@http://web-platform.test:8000/resources/testharness.js:566:30
[task 2021-03-19T15:33:16.972Z] 15:33:16 INFO - @http://web-platform.test:8000/html/browsers/the-windowproxy-exotic-object/windowproxy-define-own-property-unforgeable-same-origin.html:27:9

TEST-UNEXPECTED-PASS | /html/browsers/the-windowproxy-exotic-object/windowproxy-define-own-property-unforgeable-same-origin.html | [[DefineOwnProperty]] success: "document" - expected FAIL
TEST-UNEXPECTED-PASS | /html/browsers/the-windowproxy-exotic-object/windowproxy-define-own-property-unforgeable-same-origin.html | [[DefineOwnProperty]] success: "location" - expected FAIL
TEST-UNEXPECTED-PASS | /html/browsers/the-windowproxy-exotic-object/windowproxy-define-own-property-unforgeable-same-origin.html | [[DefineOwnProperty]] success: "top" - expected FAIL

Comment 1

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Andrew, do you know why this behavior is different between Nightly and Beta/DevEdition?

Comment 2

[Andrew McCreight [:mccr8]]

What is that based on? I don't see that file in mozilla-central.

https://hg.mozilla.org/mozilla-central/file/tip/testing/web-platform/tests/html/browsers/the-windowproxy-exotic-object

Comment 3

[Andrew McCreight [:mccr8]]

Looks like that is on autoland, but not m-c: https://bugzilla.mozilla.org/show_bug.cgi?id=1698724

Comment 4

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Sorry, it's based on autoland because I wanted to get the wpt sync with upstream into the simulation with better chances to detect fixes and issues than have them tomorrow for the first time in a simulation.

Comment 5

[Andrew McCreight [:mccr8]]

Here's a link to the test: https://hg.mozilla.org/try/file/66056b95ab22d584e3cd93f8db580a1e19d04c08/testing/web-platform/tests/html/browsers/the-windowproxy-exotic-object/windowproxy-define-own-property-unforgeable-same-origin.html

I did confirm that we are running the test on autoland.

Comment 6

[Andrew McCreight [:mccr8]]

These are unexpected passes, which means that the meta file says that the tests are expected to fail.

This is the meta file:
https://hg.mozilla.org/integration/autoland/rev/c0ae2ca6b635

The entries in there look like this:

[[[DefineOwnProperty\]\] success: "window"]
    expected: FAIL

[[[DefineOwnProperty\]\] failure: "window"]
    expected: FAIL

So I guess failure is expected to fail, and it does, so we succeed? But success is supposed to also fail, but instead it succeeds, so we fail? Honestly I'm not sure what the test is even doing or what is going wrong.

It seems like this particular part of the test is failing, but I thought it was supposed to fail, so is that a success?

TEST-FAIL | /html/browsers/the-windowproxy-exotic-object/windowproxy-define-own-property-unforgeable-same-origin.html | [[DefineOwnProperty]] failure: "window" - assert_throws_js: [[Configurable]]: true function "() => {
[task 2021-03-19T15:33:16.971Z] 15:33:16 INFO - Object.defineProperty(window, key, { configurable: true });
[task 2021-03-19T15:33:16.971Z] 15:33:16 INFO - }" did not throw

This looks like it is testing some pretty fundamental stuff, so I'd be surprised if something is changing here, but maybe something like SHIP is having some weird effect? Peter, do you have any idea?

Comment 7

[Andrew McCreight [:mccr8]]

If you want to see what the failures were that caused these tests to be marked failing, I think the logs are here: https://treeherder.mozilla.org/jobs?repo=try&revision=3ff2284a8f2c8108df4ca8ba16dad4a850cbb9fd&selectedTaskRun=clCT_xOpQUWCwjW7WvH5wA.0

Comment 8

[Andrew McCreight [:mccr8]]

The easiest workaround would be to change the meta file so that these tests are marked as either passing or failing. I think that's a thing.

Comment 9

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Attachment: Bug 1699758 - adjust expectation for new test windowproxy-define-own-property-unforgeable-same-origin.html as having unexpected passes on release and beta. r=mccr8 DONTBUILD

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a65de19aa72b
adjust expectation for new test windowproxy-define-own-property-unforgeable-same-origin.html as having unexpected passes on release and beta. r=mccr8 DONTBUILD

Comment 11

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/a65de19aa72b

Comment 12

[Cosmin Sabou [:CosminS]]

Patch works in latest beta-sim.

Comment 13

[Peter Van der Beken [:peterv]]

(In reply to Sebastian Hengst [:aryx] (needinfo on intermittent or backout) from comment #1)

Andrew, do you know why this behavior is different between Nightly and Beta/DevEdition?

The bug to make the behaviour the same between nightly and beta/release is bug 1496510.