Closed Bug 1578750 Opened 1 year ago Closed 5 months ago

Perma /html/dom/idlharness.https.html?include=HTML.* | HTMLIFrameElement interface: attribute allow - assert_true: The prototype object must have a property "allow" expected true got false when Gecko 71 merges to Beta on 2019-10-14

Categories

(Core :: DOM: Core & HTML, defect, P1)

defect

Tracking

()

VERIFIED FIXED
mozilla71
Tracking Status
firefox-esr60 --- unaffected
firefox-esr68 --- unaffected
firefox69 --- unaffected
firefox70 --- unaffected
firefox71 + verified

People

(Reporter: intermittent-bug-filer, Assigned: tnguyen)

References

(Regression)

Details

(Keywords: regression, Whiteboard: [stockwell unknown])

Attachments

(1 file)

Filed by: shindli [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=264937506&repo=try
Full log: https://queue.taskcluster.net/v1/task/NY0_MUqPRDS-ko1uVVyziw/runs/0/artifacts/public/logs/live_backing.log


[task 2019-09-04T13:01:25.027Z] 13:01:25 INFO - Certificate Nickname Trust Attributes
[task 2019-09-04T13:01:25.027Z] 13:01:25 INFO - SSL,S/MIME,JAR/XPI
[task 2019-09-04T13:01:25.027Z] 13:01:25 INFO -
[task 2019-09-04T13:01:25.027Z] 13:01:25 INFO - web-platform-tests CT,,
[task 2019-09-04T13:01:25.027Z] 13:01:25 INFO -
[task 2019-09-04T13:01:27.808Z] 13:01:27 INFO - adb Granting important runtime permissions to org.mozilla.geckoview.test
[task 2019-09-04T13:01:29.075Z] 13:01:29 INFO - adb launch_application: am start -W -n org.mozilla.geckoview.test/org.mozilla.geckoview.test.TestRunnerActivity -a android.intent.action.MAIN --es env9 MOZ_DISABLE_NONLOCAL_CONNECTIONS=1 --es env8 R_LOG_DESTINATION=stderr --es args "-no-remote -profile /sdcard/tests/profile --marionette about:blank" --es env3 MOZ_HIDE_RESULTS_TABLE=1 --es env2 R_LOG_VERBOSE=1 --es env1 MOZ_WEBRENDER=0 --es env0 MOZ_CRASHREPORTER=1 --es env7 MOZ_CRASHREPORTER_SHUTDOWN=1 --es env6 MOZ_IN_AUTOMATION=1 --es env5 MOZ_LOG=signaling:3,mtransport:4,DataChannel:4,jsep:4 --es env4 STYLO_THREADS=4 --ez use_multiprocess True --es env12 R_LOG_LEVEL=6 --es env11 MOZ_PROCESS_LOG=/tmp/tmpSysbrepidlog --es env10 MOZ_CRASHREPORTER_NO_REPORT=1
[task 2019-09-04T13:01:30.473Z] 13:01:30 INFO - Starting runner
[task 2019-09-04T13:01:30.584Z] 13:01:30 INFO - TEST-START | /html/dom/idlharness.https.html?include=(Document|Window)
[task 2019-09-04T13:01:31.545Z] 13:01:31 INFO -
[task 2019-09-04T13:01:31.545Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: attribute sessionStorage
[task 2019-09-04T13:01:31.545Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: attribute localStorage
[task 2019-09-04T13:01:31.545Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: internal [[SetPrototypeOf]] method of global platform object - setting to a new value via Object.setPrototypeOf should throw a TypeError
[task 2019-09-04T13:01:31.545Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: internal [[SetPrototypeOf]] method of global platform object - setting to a new value via proto should throw a TypeError
[task 2019-09-04T13:01:31.545Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: internal [[SetPrototypeOf]] method of global platform object - setting to a new value via Reflect.setPrototypeOf should return false
[task 2019-09-04T13:01:31.545Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: internal [[SetPrototypeOf]] method of global platform object - setting to its original value via Object.setPrototypeOf should not throw
[task 2019-09-04T13:01:31.545Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: internal [[SetPrototypeOf]] method of global platform object - setting to its original value via proto should not throw
[task 2019-09-04T13:01:31.545Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: internal [[SetPrototypeOf]] method of global platform object - setting to its original value via Reflect.setPrototypeOf should return true
[task 2019-09-04T13:01:31.545Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window must be primary interface of window
[task 2019-09-04T13:01:31.545Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Stringification of window
[task 2019-09-04T13:01:31.545Z] 13:01:31 INFO - TEST-UNEXPECTED-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: window must have own property "window" - expected FAIL
[task 2019-09-04T13:01:31.546Z] 13:01:31 INFO - TEST-INFO | expected FAIL
[task 2019-09-04T13:01:31.546Z] 13:01:31 INFO -
[task 2019-09-04T13:01:31.546Z] 13:01:31 INFO - TEST-FAIL | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: window must inherit property "self" with the proper type - Unrecognized type WindowProxy
[task 2019-09-04T13:01:31.546Z] 13:01:31 INFO - TEST-UNEXPECTED-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: window must have own property "document" - expected FAIL
[task 2019-09-04T13:01:31.546Z] 13:01:31 INFO - TEST-INFO | expected FAIL
[task 2019-09-04T13:01:31.546Z] 13:01:31 INFO -
[task 2019-09-04T13:01:31.546Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: window must inherit property "name" with the proper type
[task 2019-09-04T13:01:31.546Z] 13:01:31 INFO - TEST-UNEXPECTED-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: window must have own property "location" - expected FAIL
[task 2019-09-04T13:01:31.546Z] 13:01:31 INFO - TEST-INFO | expected FAIL
[task 2019-09-04T13:01:31.549Z] 13:01:31 INFO -
[task 2019-09-04T13:01:31.549Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: window must inherit property "statusbar" with the proper type
[task 2019-09-04T13:01:31.549Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: window must inherit property "toolbar" with the proper type
[task 2019-09-04T13:01:31.549Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: window must inherit property "status" with the proper type
[task 2019-09-04T13:01:31.549Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: window must inherit property "close()" with the proper type
[task 2019-09-04T13:01:31.549Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: window must inherit property "closed" with the proper type
[task 2019-09-04T13:01:31.549Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: window must inherit property "stop()" with the proper type
[task 2019-09-04T13:01:31.549Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: window must inherit property "focus()" with the proper type
[task 2019-09-04T13:01:31.549Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: window must inherit property "blur()" with the proper type
[task 2019-09-04T13:01:31.550Z] 13:01:31 INFO - TEST-FAIL | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: window must inherit property "frames" with the proper type - Unrecognized type WindowProxy
[task 2019-09-04T13:01:31.550Z] 13:01:31 INFO - TEST-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: window must inherit property "length" with the proper type
[task 2019-09-04T13:01:31.550Z] 13:01:31 INFO - TEST-UNEXPECTED-PASS | /html/dom/idlharness.https.html?include=(Document|Window) | Window interface: window must have own property "top" - expected FAIL
[task 2019-09-04T13:01:31.550Z] 13:01:31 INFO - TEST-INFO | expected FAIL
[task 2019-09-04T13:01:32.061Z] 13:01:32 INFO - .....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
[task 2019-09-04T13:01:32.061Z] 13:01:32 INFO - TEST-OK | /html/dom/idlharness.https.html?include=(Document|Window) | took 1477ms
[task 2019-09-04T13:01:32.721Z] 13:01:32 INFO - Closing logging queue
[task 2019-09-04T13:01:32.721Z] 13:01:32 INFO - queue closed
[task 2019-09-04T13:01:32.729Z] 13:01:32 INFO - Setting up ssl

[Tracking Requested - why for this release]:

This is really weird since the relevant .webidl files haven't changed and pref should be disabled on non-nightly.
https://searchfox.org/mozilla-central/source/dom/webidl/HTMLIFrameElement.webidl#78-79
https://searchfox.org/mozilla-central/source/modules/libpref/init/StaticPrefList.yaml#2061-2066

Did bug 1567063 break this?

Flags: needinfo?(bugs) → needinfo?(kwright)

Er, no, since that was already 70, and this is 71

Flags: needinfo?(kwright)

If looking at dom.security.featurePolicy.enabled in about:config in 71-as-beta build, is it enabled?

Flags: needinfo?(shindli)
Summary: Perma /html/dom/idlharness.https.html?include=HTML.* | HTMLIFrameElement interface: attribute allow - assert_true: The prototype object must have a property "allow" expected true got false when Gecko 71 merges to Beta on 2019-10-21 → Perma /html/dom/idlharness.https.html?include=HTML.* | HTMLIFrameElement interface: attribute allow - assert_true: The prototype object must have a property "allow" expected true got false when Gecko 71 merges to Beta on 2019-10-14

Downloaded the Windows build which is also affected and dom.security.featurePolicy.enabled is set to false. That's from https://searchfox.org/mozilla-central/rev/e5327b05c822cdac24e233afa37d72c0552dbbaf/modules/libpref/init/StaticPrefList.yaml#2065

Some idlharness changes landed 2 days ago: https://hg.mozilla.org/mozilla-central/log?rev=idlharness

Flags: needinfo?(shindli)

Hi Thomas, my understanding is that you're currently working on Feature Policy 'allow' work. Looks we will need to update the harness to mark the test failed as expected. Could you please help with this?

Flags: needinfo?(tnguyen)
Priority: -- → P1

I think we should enable pref in test instead of mark the test failed. (because the test will pass in nightly and fail in beta)

Duplicate of this bug: 1581830

I was told that :jgraham may know if there's some ini syntax that we can set pass expectation on nightly but fail on other channels, as I learned we should not have tests force pref on for all channels - is that right, Peter?

Flags: needinfo?(peterv)

(In reply to Hsin-Yi Tsai [:hsinyi] from comment #15)

I was told that :jgraham may know if there's some ini syntax that we can set pass expectation on nightly but fail on other channels, as I learned we should not have tests force pref on for all channels - is that right, Peter?

Exactly, I am running try test and ask someone for review this after getting the result. We can choose expected failed on release_and_beta or enable prefs on all builds

Assignee: nobody → tnguyen

Thanks. I could mark the test fail but I have no idea the reason.
Only one possible reason I know is bug 1510437

Flags: needinfo?(tnguyen)
Flags: needinfo?(peterv)
Attachment #9094520 - Attachment description: Bug 1578750 - Enable featurepolicy prefs in idlharness tests → Bug 1578750 - Mark correct expected result idlharness tests on different builds

This is weird.
bz, do you recall any recent changes which might require the changes in https://phabricator.services.mozilla.com/D46754 ?
Or is this just about bug 1496510 and that we've got some new wpt tests?

Flags: needinfo?(bzbarsky)

Hmm. So this failure:

[Window interface: window must have own property "window"]

is definitely due to bug 1496510 (aka bug 1510437). This used to be properly annotated in the .ini file: see https://searchfox.org/mozilla-beta/rev/b57a9e23559c27580b7fbf0870e29c03d9fdefc7/testing/web-platform/meta/html/dom/interfaces.https.html.ini#71-75

Then the test file got renamed from "interfaces.https.html" to "idlharness.https.html" (amongst others) and the resulting wpt expectation update nuked the annotations, as far as I can tell. See https://hg.mozilla.org/mozilla-central/rev/dd47a0b2457c609974032d469e951e8774b0df84 for the update that did that. It also nuked various other metadata like some "bug:" annotation in "css/cssom/interfaces.html.ini".

This failure:

[HTMLIFrameElement interface: attribute allow]

Is because that same WPT expectation update nuked a bunch of "prefs: " lines from the .ini files, including this one from "testing/web-platform/meta/html/dom/interfaces.https.html.ini":

prefs: [dom.security.featurePolicy.enabled:true]

I think I'd prefer the conditional failure annotation to the pref-setting thing, so we should basically take the patch in this bug, or better yet the annotations that used to be there before bug 1577456, which included links to relevant bugs, etc.

Flags: needinfo?(bzbarsky) → needinfo?(james)

So, when files move we try to also move the metadata [1], psecfically so we don't lose prefs or other annotations. git seems to correctly recognise the commit here as containing renames. So something went wrong; I guess there's a bug in the implementation. I've filed an issue on the wpt sync bot [2]. In this case it seems like the solution might be to copy over the old versions of the metadata files to the new location by hand, and then update them for any subsequent changes using the results of a try run covering just those tests.

[1] https://github.com/mozilla/wpt-sync/blob/master/sync/downstream.py#L452-L473
[2] https://github.com/mozilla/wpt-sync/issues/501

Flags: needinfo?(james)
Pushed by tnguyen@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/76d03f0c306d
Mark correct expected result idlharness tests on different builds r=smaug

Er... could we please restore the bug annotations that used to be there too?

Flags: needinfo?(tnguyen)

Asking for backout, thanks

Flags: needinfo?(tnguyen)

Backed out changeset 76d03f0c306d to restore old bug annotations

Backout: https://hg.mozilla.org/integration/autoland/rev/065260671bea3007c8ed103496a8653c4a5f1eb8

Pushed by tnguyen@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/d5bd7a1c2846
Mark correct expected result idlharness tests on different builds r=smaug
Status: NEW → RESOLVED
Closed: 5 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla71
You need to log in before you can comment on or make changes to this bug.