1436245 - Various CSP web-platform-tests are going to permafail when Gecko 60 merges to Beta on 2018-03-01

Status: VERIFIED FIXED

(Core :: DOM: Security, defect, P1)

Description

[Ryan VanderMeulen [:RyanVM]]

Looks like these tests were recently added in bug 1435337. Hopefully just a pref needing to be forced on?

https://treeherder.mozilla.org/logviewer.html#?job_id=160713552&repo=try
https://treeherder.mozilla.org/logviewer.html#?job_id=160713472&repo=try

TEST-UNEXPECTED-TIMEOUT | /content-security-policy/reporting/report-original-url.sub.html | Direct block, same-origin = full URL in report - Test timed out
TEST-UNEXPECTED-TIMEOUT | /content-security-policy/reporting/report-original-url.sub.html | Block after redirect, same-origin = original URL in report - Test timed out
TEST-UNEXPECTED-TIMEOUT | /content-security-policy/reporting/report-original-url.sub.html | Block after redirect, cross-origin = original URL in report - Test timed out
TEST-UNEXPECTED-TIMEOUT | /content-security-policy/reporting-api/reporting-api-report-only-sends-reports-on-violation.https.sub.html | Event is fired - Test timed out
TEST-UNEXPECTED-TIMEOUT | /content-security-policy/reporting-api/reporting-api-report-only-sends-reports-on-violation.https.sub.html | expected OK
TEST-UNEXPECTED-TIMEOUT | /content-security-policy/reporting-api/reporting-api-report-to-overrides-report-uri-1.https.sub.html | Event is fired - Test timed out
TEST-UNEXPECTED-TIMEOUT | /content-security-policy/reporting-api/reporting-api-report-to-overrides-report-uri-1.https.sub.html | expected OK
TEST-UNEXPECTED-TIMEOUT | /content-security-policy/reporting-api/reporting-api-report-to-overrides-report-uri-2.https.sub.html | Event is fired - Test timed out
TEST-UNEXPECTED-TIMEOUT | /content-security-policy/reporting-api/reporting-api-report-to-overrides-report-uri-2.https.sub.html | expected OK
TEST-UNEXPECTED-TIMEOUT | /content-security-policy/reporting-api/reporting-api-sends-reports-on-violation.https.sub.html | Event is fired - Test timed out
TEST-UNEXPECTED-TIMEOUT | /content-security-policy/reporting-api/reporting-api-sends-reports-on-violation.https.sub.html | expected OK
TEST-UNEXPECTED-TIMEOUT | /content-security-policy/reporting-api/reporting-api-works-on-frame-src.https.sub.html | Event is fired - Test timed out
TEST-UNEXPECTED-TIMEOUT | /content-security-policy/reporting-api/reporting-api-works-on-frame-src.https.sub.html | expected OK

Comment 1

[Christoph Kerschbaumer [:ckerschb]]

Attachment: bug_1436245_fix_csp_wpt_tests.patch

Ryan, tested locally by flipping the pref security.csp.enable_violation_events to false within all.js. Do we need more testing or are we good to check those updates in?

Comment 2

[Ryan VanderMeulen [:RyanVM]]

Sounds fine to me - the only question I have is whether it makes more sense to set this pref at the directory level instead of on the individual tests, which would avoid any surprises from future upstream updates. WDYT?

Comment 3

[Christoph Kerschbaumer [:ckerschb]]

(In reply to Ryan VanderMeulen [:RyanVM] from comment #2)
> Sounds fine to me - the only question I have is whether it makes more sense
> to set this pref at the directory level instead of on the individual tests,
> which would avoid any surprises from future upstream updates. WDYT?

Actually I was thinking about that and would prefer that solution but didn't know if it's possible or not. How do I do that?

Comment 4

[Ryan VanderMeulen [:RyanVM]]

Set it in __dir__.ini instead, similar to the one below.
https://searchfox.org/mozilla-central/source/testing/web-platform/meta/css/css-scoping-1/__dir__.ini

Comment 5

[Christoph Kerschbaumer [:ckerschb]]

Attachment: bug_1436245_fix_csp_wpt_tests.patch

That should do the trick for all CSP wpt tests.

Comment 6

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 8949147 [details] [diff] [review]
bug_1436245_fix_csp_wpt_tests.patch

Assuming you've verified this locally, lgtm!

Comment 7

[Christoph Kerschbaumer [:ckerschb]]

(In reply to Ryan VanderMeulen [:RyanVM] from comment #6)
> Assuming you've verified this locally, lgtm!

I cherry picked several tests and ran them locally.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/c358f1f20f8a
Flip pref explicitly for CSP security policy violation events for newly pulled wpt tests. r=ryanvm

Comment 9

[Stefan Hindli [:stefan_hindli]]

https://hg.mozilla.org/mozilla-central/rev/c358f1f20f8a

Comment 10

[Ryan VanderMeulen [:RyanVM]]

Thanks for the quick fix, Christoph!