1511193 - Green up and reenable skipped wpt content-security-policy/navigate-to/ tests on Android.

Status: NEW

(Core :: DOM: Security, enhancement, P3)

Description

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

We recently enabled WPT on Fennec, and as part of that, had to skip some frequent intermittently failing tests to get a mostly-green run. Filing this bug to track hopefully fixing whatever flakiness is causing the intermittent failures so these tests can be reenabled.

Running the tests locally on repeat, these ones failed intermittently:
content-security-policy/navigate-to/child-navigates-parent-blocked.sub.html : TIMEOUT 4 out of 5 attempts
content-security-policy/navigate-to/unsafe-allow-redirects/allowed-end-of-chain-because-of-same-origin.sub.html : TIMEOUT 3 out of 5 attempts
content-security-policy/navigate-to/unsafe-allow-redirects/allowed-end-of-chain.sub.html : TIMEOUT 1 out of 5 attempts
content-security-policy/navigate-to/unsafe-allow-redirects/blocked-end-of-chain.sub.html : TIMEOUT 1 out of 5 attempts

Comment 1

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

Attachment: Bug 1511193 - Update test annotations for fennec wpt csp/navigate-to/ a=testonly

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d9dd441d53fd
Update test annotations for fennec wpt csp/navigate-to/ a=testonly

Comment 3

[Raul Gurzau (:RaulG)]

https://hg.mozilla.org/mozilla-central/rev/d9dd441d53fd

Comment 4

[Daniel Veditz [:dveditz]]

Wes: This had leave-open, then something landed, and now it's not 'leave-open'. Does that mean it can be resolved? Or is there something more here?

Firefox does not currently support CSP navigate-to -- these tests should be skipped or annotated as expected fail.

Comment 5

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

This is not resolved, imo, though I don't have any strong feelings about keeping this bug open if that's problematic for triage in this component. 
I had to skip these tests (rather than just annotate as failing) on android because they were frequently randomly failing in ways other than the expected (crash, timeout, notrun, etc), annotated failures expected. 

Having the tests disabled (rather than annotated as expected failures) means we won't see if something starts making the tests pass (expected because we start supporting the feature or unexpected because something randomly tweaks things in ways that make the tests start passing).

In this case, though, we would probably see those changes based on the desktop tests, so if you want to resolve the bug, I won't stop you.

Comment 6

[Christoph Kerschbaumer [:ckerschb]]

In that case we just move it into the backlog for now - thanks!