1578750 - Perma /html/dom/idlharness.https.html?include=HTML.* | HTMLIFrameElement interface: attribute allow - assert_true: The prototype object must have a property "allow" expected true got false when Gecko 71 merges to Beta on 2019-10-14

Status: VERIFIED FIXED

(Core :: DOM: Core & HTML, defect, P1)

Description

[Treeherder Bug Filer]

Filed by: shindli [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=264937506&repo=try
Full log: https://queue.taskcluster.net/v1/task/NY0_MUqPRDS-ko1uVVyziw/runs/0/artifacts/public/logs/live_backing.log

---

[task 2019-09-04T13:01:25.027Z] 13:01:25 INFO - Certificate Nickname Trust Attributes
[task 2019-09-04T13:01:25.027Z] 13:01:25 INFO - SSL,S/MIME,JAR/XPI
[task 2019-09-04T13:01:25.027Z] 13:01:25 INFO -
[task 2019-09-04T13:01:25.027Z] 13:01:25 INFO - web-platform-tests CT,,
[task 2019-09-04T13:01:25.027Z] 13:01:25 INFO -
[task 2019-09-04T13:01:27.808Z] 13:01:27 INFO - adb Granting important runtime permissions to org.mozilla.geckoview.test
[task 2019-09-04T13:01:29.075Z] 13:01:29 INFO - adb launch_application: am start -W -n org.mozilla.geckoview.test/org.mozilla.geckoview.test.TestRunnerActivity -a android.intent.action.MAIN --es env9 MOZ_DISABLE_NONLOCAL_CONNECTIONS=1 --es env8 R_LOG_DESTINATION=stderr --es args "-no-remote -profile /sdcard/tests/profile --marionette about:blank" --es env3 MOZ_HIDE_RESULTS_TABLE=1 --es env2 R_LOG_VERBOSE=1 --es env1 MOZ_WEBRENDER=0 --es env0 MOZ_CRASHREPORTER=1 --es env7 MOZ_CRASHREPORTER_SHUTDOWN=1 --es env6 MOZ_IN_AUTOMATION=1 --es env5 MOZ_LOG=signaling:3,mtransport:4,DataChannel:4,jsep:4 --es env4 STYLO_THREADS=4 --ez use_multiprocess True --es env12 R_LOG_LEVEL=6 --es env11 MOZ_PROCESS_LOG=/tmp/tmpSysbrepidlog --es env10 MOZ_CRASHREPORTER_NO_REPORT=1
[task 2019-09-04T13:01:30.473Z] 13:01:30 INFO - Starting runner
[task 2019-09-04T13:01:30.584Z] 13:01:30 INFO - TEST-START | /html/dom/idlharness.https.html?include=(Document|Window)
[task 2019-09-04T13:01:31.545Z] 13:01:31 INFO -
[task 2019-09-04T13:01:31.545Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: attribute sessionStorage
[task 2019-09-04T13:01:31.545Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: attribute localStorage
[task 2019-09-04T13:01:31.545Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: internal [[SetPrototypeOf]] method of global platform object - setting to a new value via Object.setPrototypeOf should throw a TypeError
[task 2019-09-04T13:01:31.545Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: internal [[SetPrototypeOf]] method of global platform object - setting to a new value via proto should throw a TypeError
[task 2019-09-04T13:01:31.545Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: internal [[SetPrototypeOf]] method of global platform object - setting to a new value via Reflect.setPrototypeOf should return false
[task 2019-09-04T13:01:31.545Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: internal [[SetPrototypeOf]] method of global platform object - setting to its original value via Object.setPrototypeOf should not throw
[task 2019-09-04T13:01:31.545Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: internal [[SetPrototypeOf]] method of global platform object - setting to its original value via proto should not throw
[task 2019-09-04T13:01:31.545Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: internal [[SetPrototypeOf]] method of global platform object - setting to its original value via Reflect.setPrototypeOf should return true
[task 2019-09-04T13:01:31.545Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window must be primary interface of window
[task 2019-09-04T13:01:31.545Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Stringification of window
[task 2019-09-04T13:01:31.545Z] 13:01:31 INFO - TEST-UNEXPECTED-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: window must have own property "window" - expected FAIL
[task 2019-09-04T13:01:31.546Z] 13:01:31 INFO - TEST-INFO | expected FAIL
[task 2019-09-04T13:01:31.546Z] 13:01:31 INFO -
[task 2019-09-04T13:01:31.546Z] 13:01:31 INFO - TEST-FAIL | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: window must inherit property "self" with the proper type - Unrecognized type WindowProxy
[task 2019-09-04T13:01:31.546Z] 13:01:31 INFO - TEST-UNEXPECTED-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: window must have own property "document" - expected FAIL
[task 2019-09-04T13:01:31.546Z] 13:01:31 INFO - TEST-INFO | expected FAIL
[task 2019-09-04T13:01:31.546Z] 13:01:31 INFO -
[task 2019-09-04T13:01:31.546Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: window must inherit property "name" with the proper type
[task 2019-09-04T13:01:31.546Z] 13:01:31 INFO - TEST-UNEXPECTED-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: window must have own property "location" - expected FAIL
[task 2019-09-04T13:01:31.546Z] 13:01:31 INFO - TEST-INFO | expected FAIL
[task 2019-09-04T13:01:31.549Z] 13:01:31 INFO -
[task 2019-09-04T13:01:31.549Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: window must inherit property "statusbar" with the proper type
[task 2019-09-04T13:01:31.549Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: window must inherit property "toolbar" with the proper type
[task 2019-09-04T13:01:31.549Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: window must inherit property "status" with the proper type
[task 2019-09-04T13:01:31.549Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: window must inherit property "close()" with the proper type
[task 2019-09-04T13:01:31.549Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: window must inherit property "closed" with the proper type
[task 2019-09-04T13:01:31.549Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: window must inherit property "stop()" with the proper type
[task 2019-09-04T13:01:31.549Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: window must inherit property "focus()" with the proper type
[task 2019-09-04T13:01:31.549Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: window must inherit property "blur()" with the proper type
[task 2019-09-04T13:01:31.550Z] 13:01:31 INFO - TEST-FAIL | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: window must inherit property "frames" with the proper type - Unrecognized type WindowProxy
[task 2019-09-04T13:01:31.550Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: window must inherit property "length" with the proper type
[task 2019-09-04T13:01:31.550Z] 13:01:31 INFO - TEST-UNEXPECTED-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: window must have own property "top" - expected FAIL
[task 2019-09-04T13:01:31.550Z] 13:01:31 INFO - TEST-INFO | expected FAIL
[task 2019-09-04T13:01:32.061Z] 13:01:32 INFO - .....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
[task 2019-09-04T13:01:32.061Z] 13:01:32 INFO - TEST-OK | /html/dom/idlharness.https.html?include=(Document|Window) | took 1477ms
[task 2019-09-04T13:01:32.721Z] 13:01:32 INFO - Closing logging queue
[task 2019-09-04T13:01:32.721Z] 13:01:32 INFO - queue closed
[task 2019-09-04T13:01:32.729Z] 13:01:32 INFO - Setting up ssl

Comment 1

[Stefan Hindli [:stefan_hindli]]

[Tracking Requested - why for this release]:

Comment 2

[Intermittent Failures Robot]

26 failures in 874 pushes (0.03 failures/push) were associated with this bug yesterday.

Repository breakdown:

• try: 26

Platform breakdown:

• android-em-7-0-x86_64: 6
• macosx1014-64: 1
• linux64: 2
• linux64-qr: 1
• macosx1014-64-shippable: 1
• windows10-64-qr: 1
• windows10-64: 1
• macosx1014-64-devedition: 1
• windows7-32: 3
• linux64-devedition: 1
• linux32-shippable: 1
• windows7-32-shippable: 1
• linux64-shippable: 1
• linux64-shippable-qr: 1
• windows7-32-devedition: 1
• windows10-64-devedition: 1
• windows10-64-shippable: 1
• windows10-64-shippable-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1578750&startday=2019-09-04&endday=2019-09-04&tree=all

Comment 3

[Olli Pettay [:smaug][[e-mail]]]

This is really weird since the relevant .webidl files haven't changed and pref should be disabled on non-nightly.
https://searchfox.org/mozilla-central/source/dom/webidl/HTMLIFrameElement.webidl#78-79
https://searchfox.org/mozilla-central/source/modules/libpref/init/StaticPrefList.yaml#2061-2066

Did bug 1567063 break this?

Comment 4

[Olli Pettay [:smaug][[e-mail]]]

Er, no, since that was already 70, and this is 71

Comment 5

[Olli Pettay [:smaug][[e-mail]]]

If looking at dom.security.featurePolicy.enabled in about:config in 71-as-beta build, is it enabled?

Comment 6

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Downloaded the Windows build which is also affected and dom.security.featurePolicy.enabled is set to false. That's from https://searchfox.org/mozilla-central/rev/e5327b05c822cdac24e233afa37d72c0552dbbaf/modules/libpref/init/StaticPrefList.yaml#2065

Some idlharness changes landed 2 days ago: https://hg.mozilla.org/mozilla-central/log?rev=idlharness

Comment 7

[Intermittent Failures Robot]

28 failures in 4408 pushes (0.006 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 28

Platform breakdown:

• android-em-7-0-x86_64: 6
• macosx1014-64: 1
• linux64: 2
• linux64-qr: 1
• macosx1014-64-shippable: 1
• windows10-64-qr: 2
• windows10-64: 2
• macosx1014-64-devedition: 1
• windows7-32: 3
• linux64-devedition: 1
• linux32-shippable: 1
• windows7-32-shippable: 1
• linux64-shippable: 1
• linux64-shippable-qr: 1
• windows7-32-devedition: 1
• windows10-64-devedition: 1
• windows10-64-shippable: 1
• windows10-64-shippable-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1578750&startday=2019-09-02&endday=2019-09-08&tree=all

Comment 8

[Intermittent Failures Robot]

14 failures in 4637 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 14

Platform breakdown:

• windows10-64: 1
• linux64: 2
• linux64-qr: 1
• macosx1014-64: 1
• linux64-devedition: 1
• linux64-shippable-qr: 1
• linux64-shippable: 2
• linux32-shippable: 2
• android-em-7-0-x86_64: 2
• windows7-32-devedition: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1578750&startday=2019-09-09&endday=2019-09-15&tree=all

Comment 9

[Hsin-Yi Tsai (she/her) [:hsinyi]]

Hi Thomas, my understanding is that you're currently working on Feature Policy 'allow' work. Looks we will need to update the harness to mark the test failed as expected. Could you please help with this?

Comment 10

[Thomas Nguyen (:tnguyen)]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=433aae87ad5ae43013713a03339c6afefbaf4a82
https://treeherder.mozilla.org/#/jobs?repo=try&revision=4fc934c359b9565721d4c25d82377b1a460528f7

Comment 11

[Thomas Nguyen (:tnguyen)]

I think we should enable pref in test instead of mark the test failed. (because the test will pass in nightly and fail in beta)

Comment 13

[Intermittent Failures Robot]

6 failures in 4350 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 6

Platform breakdown:

• linux64-qr: 2
• linux64: 3
• macosx1014-64-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1578750&startday=2019-09-16&endday=2019-09-22&tree=all

Comment 14

[Thomas Nguyen (:tnguyen)]

Attachment: Bug 1578750 - Mark correct expected result idlharness tests on different builds

Comment 15

[Hsin-Yi Tsai (she/her) [:hsinyi]]

I was told that :jgraham may know if there's some ini syntax that we can set pass expectation on nightly but fail on other channels, as I learned we should not have tests force pref on for all channels - is that right, Peter?

Comment 16

[Thomas Nguyen (:tnguyen)]

(In reply to Hsin-Yi Tsai [:hsinyi] from comment #15)

I was told that :jgraham may know if there's some ini syntax that we can set pass expectation on nightly but fail on other channels, as I learned we should not have tests force pref on for all channels - is that right, Peter?

Exactly, I am running try test and ask someone for review this after getting the result. We can choose expected failed on release_and_beta or enable prefs on all builds

Comment 17

[Alexandru Michis [:malexandru]]

The patch attached here doesn't solve the issue (central as beta simulation for reference): https://treeherder.mozilla.org/#/jobs?repo=try&selectedJob=267934142&resultStatus=testfailed%2Cbusted%2Cexception%2Crunnable&revision=db301cdd1eb3f15428c484c58aada07c7b4a889d&searchStr=%28wpt

Comment 18

[Thomas Nguyen (:tnguyen)]

Thanks. I could mark the test fail but I have no idea the reason.
Only one possible reason I know is bug 1510437

Comment 19

[Thomas Nguyen (:tnguyen)]

try nightly and Beta Simulation:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=f659860631e5d986ee1f26e3e4328b5235e847cc
https://treeherder.mozilla.org/#/jobs?repo=try&revision=007547c32f9e624c51ea7695e934df749f394e7d

Comment 20

[Olli Pettay [:smaug][[e-mail]]]

This is weird.
bz, do you recall any recent changes which might require the changes in https://phabricator.services.mozilla.com/D46754 ?
Or is this just about bug 1496510 and that we've got some new wpt tests?

Comment 21

[Boris Zbarsky [:bzbarsky]]

Hmm. So this failure:

[Window interface: window must have own property "window"]

is definitely due to bug 1496510 (aka bug 1510437). This used to be properly annotated in the .ini file: see https://searchfox.org/mozilla-beta/rev/b57a9e23559c27580b7fbf0870e29c03d9fdefc7/testing/web-platform/meta/html/dom/interfaces.https.html.ini#71-75

Then the test file got renamed from "interfaces.https.html" to "idlharness.https.html" (amongst others) and the resulting wpt expectation update nuked the annotations, as far as I can tell. See https://hg.mozilla.org/mozilla-central/rev/dd47a0b2457c609974032d469e951e8774b0df84 for the update that did that. It also nuked various other metadata like some "bug:" annotation in "css/cssom/interfaces.html.ini".

This failure:

[HTMLIFrameElement interface: attribute allow]

Is because that same WPT expectation update nuked a bunch of "prefs: " lines from the .ini files, including this one from "testing/web-platform/meta/html/dom/interfaces.https.html.ini":

prefs: [dom.security.featurePolicy.enabled:true]

I think I'd prefer the conditional failure annotation to the pref-setting thing, so we should basically take the patch in this bug, or better yet the annotations that used to be there before bug 1577456, which included links to relevant bugs, etc.

Comment 22

[James Graham [:jgraham]]

So, when files move we try to also move the metadata [1], psecfically so we don't lose prefs or other annotations. git seems to correctly recognise the commit here as containing renames. So something went wrong; I guess there's a bug in the implementation. I've filed an issue on the wpt sync bot [2]. In this case it seems like the solution might be to copy over the old versions of the metadata files to the new location by hand, and then update them for any subsequent changes using the results of a try run covering just those tests.

[1] https://github.com/mozilla/wpt-sync/blob/master/sync/downstream.py#L452-L473
[2] https://github.com/mozilla/wpt-sync/issues/501

Comment 23

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/76d03f0c306d
Mark correct expected result idlharness tests on different builds r=smaug

Comment 24

[Boris Zbarsky [:bzbarsky]]

Er... could we please restore the bug annotations that used to be there too?

Comment 25

[Thomas Nguyen (:tnguyen)]

Asking for backout, thanks

Comment 26

[Andreea Pavel [:apavel]]

Backed out changeset 76d03f0c306d to restore old bug annotations

Backout: https://hg.mozilla.org/integration/autoland/rev/065260671bea3007c8ed103496a8653c4a5f1eb8

Comment 27

[Intermittent Failures Robot]

13 failures in 4530 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 13

Platform breakdown:

• linux64-qr: 2
• linux64: 4
• android-em-7-0-x86_64: 4
• windows10-64: 2
• windows10-64-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1578750&startday=2019-09-23&endday=2019-09-29&tree=all

Comment 28

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d5bd7a1c2846
Mark correct expected result idlharness tests on different builds r=smaug

Comment 29

[Raul Gurzau (:RaulG)]

https://hg.mozilla.org/mozilla-central/rev/d5bd7a1c2846

Comment 30

[Andreea Pavel [:apavel]]

Verified fixed in recent beta sims: https://treeherder.mozilla.org/#/jobs?repo=try&selectedJob=269592261&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel%2Crunnable&revision=b6507172914b14d3270456c80ee292abfc3994da&searchStr=wpt7

Comment 31

[Intermittent Failures Robot]

1 failures in 4292 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 1

Platform breakdown:

• macosx1014-64-devedition: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1578750&startday=2019-09-30&endday=2019-10-06&tree=all