Closed Bug 1457542 Opened 6 years ago Closed 6 years ago

[wpt-sync] Sync PR 10691 - Complete the migration from marionette-specific scripts.

Categories

(Testing :: web-platform-tests, enhancement, P4)

Product:
Testing
Component:
web-platform-tests
Type:
enhancement

Tracking

(firefox62 fixed)

Status:
RESOLVED FIXED
Milestone:
mozilla62
Tracking Flags:
Tracking Status
firefox62 --- fixed

People

(Reporter: mozilla.org, Unassigned)

References

(
URL
)

Details

(Whiteboard: [wptsync downstream]) 

Sync web-platform-tests PR 10691 into mozilla-central (this bug is closed when the sync is complete).

PR: https://github.com/w3c/web-platform-tests/pull/10691
Details from upstream follow.

James Graham <james@hoppipolla.co.uk> wrote:
>  Complete the migration from marionette-specific scripts.
>  
>  Previously we removed the wrappedJSObject from
>  executor_marionette.js. However it turns out that this migration was
>  incomplete; in order to fully avoid the wrappedJSObject, we need to
>  ensure that we never create a sandbox on the marionette side. As a
>  result of this omission we were getting an error trying to call the
>  timeout() function, causing more EXTERNAL-TIMEOUTs and so performance
>  issues.
>  
>  The solution is to use the same approach as geckodriver, which isn't
>  exposed through the js marionette client. Essentially we have to
>  ensure that the sandbox is None for all calls to (content) js. Since
>  this isn't exposed in the client we construct the message directly and
>  ensure that all calls to content js co through the base protocolpart
>  rather than calling marionette.execute_* directly.
Ran 4499[linux32-opt,linux64-debug,linux64-opt], 14604[linux32-debug] tests and 61008[linux32-debug], 61014[linux64-debug], 61013[linux32-opt], 61022[linux64-opt] subtests
OK     : 873[linux32-opt], 875[linux64-opt], 868[linux32-debug], 869[linux64-debug]
PASS   : 57144[linux32-opt], 57153[linux64-opt], 57143[linux64-debug], 64311[linux32-debug]
FAIL   : 7321[linux32-opt,linux64-opt], 7322[linux64-debug], 9117[linux32-debug]
TIMEOUT: 58[linux32-debug], 54[linux32-opt,linux64-debug,linux64-opt]
ERROR  : 4
NOTRUN : 30

Existing tests that now have a worse result (e.g. they used to PASS and now FAIL):
/css/CSS2/floats-clear/clear-on-parent-with-margins-no-clearance.html: FAIL
/css/CSS2/margin-padding-clear/padding-em-inherit-001.xht: FAIL[linux32-debug]
/css/CSS2/zorder/z-index-020.xht: FAIL[linux32-debug]
/css/css-shapes/shape-outside/supported-shapes/polygon/shape-outside-polygon-012.html: FAIL[linux32-debug]
/css/css-shapes/shape-outside/supported-shapes/polygon/shape-outside-polygon-014.html: FAIL[linux32-debug]
/css/css-shapes/shape-outside/supported-shapes/polygon/shape-outside-polygon-015.html: FAIL[linux32-debug]
/css/css-shapes/shape-outside/supported-shapes/polygon/shape-outside-polygon-017.html: FAIL[linux32-debug]
/html/editing/focus/focus-management/focus-events.html: SKIP[linux32-debug]
/webdriver/tests/cookies/delete_cookie.py: SKIP[linux32-debug]
    test_unknown_cookie: FAIL[linux32-opt,linux64-debug,linux64-opt]
/webdriver/tests/element_click/bubbling.py: SKIP[linux32-opt]
/webdriver/tests/execute_async_script/user_prompts.py
    test_handle_prompt_default: FAIL[linux64-debug,linux64-opt]
    test_handle_prompt_twice: FAIL[linux64-debug]
/webdriver/tests/execute_script/cyclic.py: SKIP[linux32-opt]
/webdriver/tests/execute_script/user_prompts.py
    test_handle_prompt_default: FAIL[linux64-debug,linux64-opt]
    test_handle_prompt_twice: FAIL[linux64-debug,linux64-opt]

New tests that have failures or other problems:
/content-security-policy/frame-src/frame-src-redirect.html
    Redirected iframe src should evaluate both enforced and report-only policies on both original request and when following redirect: TIMEOUT
/content-security-policy/frame-src/frame-src-self-unique-origin.html
    Iframe's url must not match with 'self'. It must be blocked.: TIMEOUT
/content-security-policy/inheritance/inherited-csp-list-modifications-are-local.html
    Test that parent document image loads: FAIL
    Test that spv event is fired: NOTRUN
/content-security-policy/inheritance/window.html
    window.open() inherits policy.: TIMEOUT
/dom/events/relatedTarget.window.html
    Untitled: FAIL
/feature-policy/autoplay-allowed-by-feature-policy-attribute-redirect-on-load.https.sub.html
    Feature-Policy allow="autoplay" allows same-origin navigation in an iframe.: TIMEOUT
    Feature-Policy allow="autoplay" disallows cross-origin navigation in an iframe.: TIMEOUT
/feature-policy/autoplay-allowed-by-feature-policy-attribute.https.sub.html
    Feature policy "autoplay" can be enabled in cross-origin iframe using allow="autoplay" attribute: TIMEOUT
    Feature policy "autoplay" can be enabled in same-origin iframe using allow="autoplay" attribute: TIMEOUT
/feature-policy/autoplay-allowed-by-feature-policy.https.sub.html
    Feature-Policy header: autoplay * allows same-origin iframes.: TIMEOUT
    Feature-Policy header: autoplay * allows the top-level document.: TIMEOUT
/feature-policy/autoplay-default-feature-policy.https.sub.html
    Default "autoplay" feature policy ["self"] allows same-origin iframes.: TIMEOUT
    Default "autoplay" feature policy ["self"] allows the top-level document.: TIMEOUT
    Default "autoplay" feature policy ["self"] disallows cross-origin iframes.: FAIL
/feature-policy/autoplay-disabled-by-feature-policy.https.sub.html
    Feature-Policy header: autoplay "none" disallows cross-origin iframes.: FAIL
    Feature-Policy header: autoplay "none" disallows same-origin iframes.: TIMEOUT
    Feature-Policy header: autoplay "none" has no effect on the top level document.: TIMEOUT
/feature-policy/feature-policy-header-policy-allowed-for-all.https.sub.html
    Feature-Policy: fullscreen * -- test fullscreen is allowed on cross-origin subframe: NOTRUN
    Feature-Policy: fullscreen * -- test fullscreen is allowed on same-origin subframe: TIMEOUT
    Feature-Policy: fullscreen *, iframe.allow = fullscreen 'self'; -- test fullscreen is allowed on same-origin subframe: NOTRUN
    Feature-Policy: fullscreen *, iframe.allow = fullscreen 'self'; -- test fullscreen is disallowed on cross-origin subframe: NOTRUN
    Feature-Policy: fullscreen *NaN: FAIL
/feature-policy/feature-policy-header-policy-allowed-for-self.https.sub.html
    Feature-Policy: fullscreen 'self' -- test allowlist is [same_origin]: FAIL
    Feature-Policy: fullscreen 'self' -- test fullscreen is allowed on same-origin subframe: TIMEOUT
    Feature-Policy: fullscreen 'self' -- test fullscreen is disallowed on cross-origin subframe: NOTRUN
    Feature-Policy: fullscreen 'self', iframe.allow = fullscreen 'src'; -- test fullscreen is allowed on cross-origin subframe: NOTRUN
    Feature-Policy: fullscreen 'self', iframe.allow = fullscreen 'src'; -- test fullscreen is allowed on same-origin subframe: NOTRUN
/feature-policy/feature-policy-header-policy-allowed-for-some.https.sub.html
    Feature-Policy: fullscreen 'self' https://www.web-platform.test:8443 https://www.example.com; -- test allowlist is [same_origin, cross_origin, https://www.example.com]: FAIL
    Feature-Policy: fullscreen 'self' https://www.web-platform.test:8443 https://www.example.com; -- test fullscreen is allowed on cross-origin https://www.web-platform.test:8443/feature-policy/resources/feature-policy-allowedfeatures.html subframe: NOTRUN
    Feature-Policy: fullscreen 'self' https://www.web-platform.test:8443 https://www.example.com; -- test fullscreen is allowed on same-origin subframe: TIMEOUT
    Feature-Policy: fullscreen 'self' https://www.web-platform.test:8443 https://www.example.com; -- test fullscreen is disallowed on cross-origin https://www1.web-platform.test:8443/feature-policy/resources/feature-policy-allowedfeatures.html subframe: NOTRUN
    Feature-Policy: fullscreen 'self' https://www.web-platform.test:8443 https://www.example.com;, iframe.allow = fullscreen 'none'; -- test fullscreen is disallowed on same-origin subframe: NOTRUN
    Feature-Policy: fullscreen 'self' https://www.web-platform.test:8443 https://www.example.com;iframe.allow = fullscreen 'none'; -- test fullscreen is disallowed on cross-origin subframe: NOTRUN
/feature-policy/feature-policy-header-policy-disallowed-for-all.https.sub.html
    Feature-Policy: fullscreen 'none' -- test allowlist is []: FAIL
    Feature-Policy: fullscreen 'none' -- test fullscreen is disallowed on cross-origin subframe: NOTRUN
    Feature-Policy: fullscreen 'none' -- test fullscreen is disallowed on same-origin subframe: TIMEOUT
    Feature-Policy: fullscreen 'none', iframe.allow = fullscreen 'src'; -- test fullscreen is disallowed on cross-origin subframe: NOTRUN
    Feature-Policy: fullscreen 'none', iframe.allow = fullscreen 'src'; -- test fullscreen is disallowed on same-origin subframe: NOTRUN
/feature-policy/feature-policy-nested-header-policy-allowed-for-all.https.sub.html
    Test nested header policy with local iframe on policy "fullscreen 'none'": NOTRUN
    Test nested header policy with local iframe on policy "fullscreen 'self'": NOTRUN
    Test nested header policy with local iframe on policy "fullscreen *": TIMEOUT
    Test nested header policy with remote iframe on policy "fullscreen 'none'": NOTRUN
    Test nested header policy with remote iframe on policy "fullscreen 'self'": NOTRUN
    Test nested header policy with remote iframe on policy "fullscreen *": NOTRUN
/feature-policy/feature-policy-nested-header-policy-allowed-for-self.https.sub.html
    Test nested header policy with local iframe on policy "fullscreen 'none'": NOTRUN
    Test nested header policy with local iframe on policy "fullscreen 'self'": NOTRUN
    Test nested header policy with local iframe on policy "fullscreen *": TIMEOUT
    Test nested header policy with remote iframe on policy "fullscreen 'none'": NOTRUN
    Test nested header policy with remote iframe on policy "fullscreen 'self'": NOTRUN
    Test nested header policy with remote iframe on policy "fullscreen *": NOTRUN
/feature-policy/feature-policy-nested-header-policy-disallowed-for-all.https.sub.html
    Test nested header policy with local iframe on policy "fullscreen 'none'".: NOTRUN
    Test nested header policy with local iframe on policy "fullscreen 'self'".: NOTRUN
    Test nested header policy with local iframe on policy "fullscreen *".: TIMEOUT
    Test nested header policy with remote iframe on policy "fullscreen 'none'".: NOTRUN
    Test nested header policy with remote iframe on policy "fullscreen 'self'".: NOTRUN
    Test nested header policy with remote iframe on policy "fullscreen *".: NOTRUN
/feature-policy/picture-in-picture-allowed-by-feature-policy.https.sub.html
    Feature-Policy header: picture-in-picture * allows cross-origin iframes.: TIMEOUT
    Feature-Policy header: picture-in-picture * allows same-origin iframes.: TIMEOUT
    Feature-Policy header: picture-in-picture * allows the top-level document.: TIMEOUT
/feature-policy/picture-in-picture-default-feature-policy.https.sub.html
    Default "picture-in-picture" feature policy [*] allows cross-origin iframes.: TIMEOUT
    Default "picture-in-picture" feature policy [*] allows same-origin iframes.: TIMEOUT
    Default "picture-in-picture" feature policy [*] allows the top-level document.: TIMEOUT
/feature-policy/picture-in-picture-disabled-by-feature-policy.https.sub.html
    Feature-Policy header: picture-in-picture "none" disallows cross-origin iframes.: TIMEOUT
    Feature-Policy header: picture-in-picture "none" disallows same-origin iframes.: TIMEOUT
    Feature-Policy header: picture-in-picture "none" disallows the top-level document.: TIMEOUT
/fetch/security/dangling-markup-mitigation-data-url.tentative.sub.html
    <iframe id="dangling"\n        src="     data:text/html,\n            <img\n              onload='window.parent.postMessage(&quot;loaded&quot;, &quot;*&quot;);'\n              onerror='window.parent.postMessage(&quot;error&quot;, &quot;*&quot;);'\n              src='http://web-platform.test:8000/images/gr&#10;een-256x256.png?&lt;'>\n        ">\n     </iframe>: FAIL
    <iframe id="dangling"\n        src="&#10;data:text/html,\n            <img\n              onload='window.parent.postMessage(&quot;loaded&quot;, &quot;*&quot;);'\n              onerror='window.parent.postMessage(&quot;error&quot;, &quot;*&quot;);'\n              src='http://web-platform.test:8000/images/gr&#10;een-256x256.png?&lt;'>\n        ">\n     </iframe>: FAIL
    <iframe id="dangling"\n        src="\ndata:text/html,\n            <img\n              onload='window.parent.postMessage(&quot;loaded&quot;, &quot;*&quot;);'\n              onerror='window.parent.postMessage(&quot;error&quot;, &quot;*&quot;);'\n              src='http://web-platform.test:8000/images/gr&#10;een-256x256.png?&lt;'>\n        ">\n     </iframe>: FAIL
    <iframe id="dangling"\n        src="\rdata:text/html,\n            <img\n              onload='window.parent.postMessage(&quot;loaded&quot;, &quot;*&quot;);'\n              onerror='window.parent.postMessage(&quot;error&quot;, &quot;*&quot;);'\n              src='http://web-platform.test:8000/images/gr&#10;een-256x256.png?&lt;'>\n        ">\n     </iframe>: FAIL
    <iframe id="dangling"\n        src="\tdata:text/html,\n            <img\n              onload='window.parent.postMessage(&quot;loaded&quot;, &quot;*&quot;);'\n              onerror='window.parent.postMessage(&quot;error&quot;, &quot;*&quot;);'\n              src='http://web-platform.test:8000/images/gr&#10;een-256x256.png?&lt;'>\n        ">\n     </iframe>: FAIL
    <iframe id="dangling"\n        src="data:text/html,\n            <img\n              onload='window.parent.postMessage(&quot;loaded&quot;, &quot;*&quot;);'\n              onerror='window.parent.postMessage(&quot;error&quot;, &quot;*&quot;);'\n              src='http://web-platform.test:8000/images/gr&#10;een-256x256.png?&lt;'>\n        ">\n     </iframe>: FAIL
    <iframe id="dangling"\n        src="data:text/html,\n            <img\n              onload='window.parent.postMessage(&quot;loaded&quot;, &quot;*&quot;);'\n              onerror='window.parent.postMessage(&quot;error&quot;, &quot;*&quot;);'\n              src='http://web-platform.test:8000/images/green-256x256.png?&amp;amp;#10;&lt;'>\n        ">\n     </iframe>: TIMEOUT
/html/browsers/sandboxing/sandbox-disallow-same-origin.html
    Access to sandbox iframe is disallowed: NOTRUN
/html/semantics/document-metadata/the-meta-element/pragma-directives/attr-meta-http-equiv-refresh/parsing.html
    <meta>: "+0": FAIL
    <meta>: "+0; foo": FAIL
    <meta>: "+0; url=foo": FAIL
    <meta>: "+1": FAIL
    <meta>: "+1; foo": FAIL
    <meta>: "+1; url=foo": FAIL
    <meta>: ", foo": FAIL
    <meta>: ",foo": FAIL
    <meta>: "-0": FAIL
    <meta>: "-0.1; url=foo": FAIL
    <meta>: "-0; foo": FAIL
    <meta>: "-0; url=foo": FAIL
    <meta>: "-1": FAIL
    <meta>: "1,\furl=foo": FAIL
    <meta>: "1; url=\"foo'bar": FAIL
    <meta>: "1;\furl=foo": FAIL
    <meta>: "1\f": TIMEOUT
    <meta>: "1\furl=foo": TIMEOUT
    <meta>: "; foo": FAIL
    <meta>: ";foo": FAIL
    Refresh header: "+0": FAIL
    Refresh header: "+0; foo": FAIL
    Refresh header: "+0; url=foo": FAIL
    Refresh header: "+1": FAIL
    Refresh header: "+1; foo": FAIL
    Refresh header: "+1; url=foo": FAIL
    Refresh header: ", foo": FAIL
    Refresh header: ",foo": FAIL
    Refresh header: "-0": FAIL
    Refresh header: "-0.1; url=foo": FAIL
    Refresh header: "-0; foo": FAIL
    Refresh header: "-0; url=foo": FAIL
    Refresh header: "-1": FAIL
    Refresh header: "1; url=\"foo'bar": FAIL
    Refresh header: "; foo": FAIL
    Refresh header: ";foo": FAIL
/webdriver/tests/execute_async_script/user_prompts.py
    test_handle_prompt_accept: FAIL[linux32-debug,linux32-opt]
    test_handle_prompt_accept_and_notify: FAIL[linux32-debug,linux32-opt]
    test_handle_prompt_default: FAIL[linux32-debug,linux32-opt]
    test_handle_prompt_dismiss: FAIL[linux32-debug,linux32-opt]
    test_handle_prompt_dismiss_and_notify: FAIL[linux32-debug,linux32-opt]
    test_handle_prompt_ignore: FAIL[linux32-debug,linux32-opt]
    test_handle_prompt_twice: FAIL[linux32-debug,linux32-opt,linux64-opt]
/webdriver/tests/execute_script/user_prompts.py
    test_handle_prompt_accept: FAIL[linux32-debug,linux32-opt]
    test_handle_prompt_accept_and_notify: FAIL[linux32-debug,linux32-opt]
    test_handle_prompt_default: FAIL[linux32-debug,linux32-opt]
    test_handle_prompt_dismiss: FAIL[linux32-debug,linux32-opt]
    test_handle_prompt_dismiss_and_notify: FAIL[linux32-debug,linux32-opt]
    test_handle_prompt_ignore: FAIL[linux32-debug,linux32-opt]
    test_handle_prompt_twice: FAIL[linux32-debug,linux32-opt]

(truncated for maximum comment length)
Pushed by james@hoppipolla.co.uk:
https://hg.mozilla.org/integration/mozilla-inbound/rev/a91d80ceefbd
[wpt PR 10691] - Complete the migration from marionette-specific scripts., a=testonly
https://hg.mozilla.org/integration/mozilla-inbound/rev/9a0b4feaadfa
[wpt PR 10691] - Update wpt metadata, a=testonly
https://hg.mozilla.org/mozilla-central/rev/a91d80ceefbd
https://hg.mozilla.org/mozilla-central/rev/9a0b4feaadfa
Status: NEW → RESOLVED
Closed: 6 years ago
status-firefox62: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla62
You need to log in before you can comment on or make changes to this bug.