1409200 - (csp-external-hashes) Implement CSP-3 support for hashes matching external resources with an integrity attribute

Status: RESOLVED FIXED

(Core :: DOM: Security, task, P3)

Description

[corey]

Attachment: SRI console error.jpg

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13) AppleWebKit/604.1.38 (KHTML, like Gecko) Version/11.0 Safari/604.1.38

Steps to reproduce:

Firefox does not seem to be correctly computing or comparing the sha256 hash for assets when you use both Subresource Integrity and a Content-Security-Policy containing the hash and 'strict-dynamic'.

I've created a test page with the affected policy, load this page in Firefox 56: 

https://scryfall.com/etc/firefox-sha256-test

The `integrity` attribute of the `<script>` tag matches the hash present in the security policy.

You can verify the hash with terminal commands similar to:

$ curl --silent "https://assets.scryfall.com/assets/scryfall-e3385384f25ec87397561982c8e1952260684129ba7b528fc86eda9dc179df14.js" | openssl sha -sha256
$ echo "e3385384f25ec87397561982c8e1952260684129ba7b528fc86eda9dc179df14" | xxd -r -p | base64


Actual results:

Firefox blocks the script in question, and console reports: 

> Content Security Policy: The page’s settings blocked the loading of a resource at [...] 
> None of the “sha256” hashes in the integrity attribute match the content of the subresource.

See attached image of the console.




Expected results:

Firefox should not block this script from loading.

Other browsers:

Chrome 61 loads all page resources with no warnings.

Safari 11 warns that it does not understand 'strict-dynamic', but otherwise loads the script in question

Comment 1

[Christoph Kerschbaumer [:ckerschb]]

Freddy, any chance you could take a look at this one? Since I know we have tests for CSP and SRI to make sure that we compare hashes correctly I assume we are facing some kind of encoding problem. Since you are my person of trust for encoding problems, could you take a quick look?

Comment 2

[Frederik Braun [:freddy]]

Hm, I can do some digging for now.

Going to the website I'm presented with an asset of the URL
> https://assets.scryfall.com/assets/scryfall-e3385384f25ec87397561982c8e1952260684129ba7b528fc86eda9dc179df14.js

Script tag is
> <script src="https://assets.scryfall.com/assets/scryfall-5c8f8b1aa86b393150ab707c6bd3d8e81f1b4dfb5224a3e887db742c54195d44.js" async="async" crossorigin="anonymous" integrity="sha256-XI+LGqhrOTFQq3B8a9PY6B8bTftSJKPoh9t0LFQZXUQ="></script>

Using that script tag in a separate document allows it: I've added an onload=alert(1), which does fire.
> data:text/html,<script src="https://assets.scryfall.com/assets/scryfall-5c8f8b1aa86b393150ab707c6bd3d8e81f1b4dfb5224a3e887db742c54195d44.js" async="async" crossorigin="anonymous" integrity="sha256-XI+LGqhrOTFQq3B8a9PY6B8bTftSJKPoh9t0LFQZXUQ=" onload=alert(1)></script>

Comment 3

[Frederik Braun [:freddy]]

Removing strict-dynamic and using host based whitelisting in the CSP makes it load. Somehow strict-dynamic messes with the SRI functionality.

Comment 4

[Daniel Veditz [:dveditz]]

(In reply to Frederik Braun [:freddyb] from comment #3)
> Removing strict-dynamic and using host based whitelisting in the CSP makes
> it load. Somehow strict-dynamic messes with the SRI functionality.

What if you use strict-dynamic and use _hash_ based whitelisting? The question is whether the problem is our hash processing is simply broken (but we have tests?), or if strict-dynamic interferes with the hash based whitelisting.

Comment 5

[Daniel Veditz [:dveditz]]

Wait, isn't hashes only supported for inline scripts?

Comment 6

[Daniel Veditz [:dveditz]]

Right, CSP 2 (which is where we more or less are) only supports hashes for inline scripts.

CSP 3 (of which we've done strict-dynamic but not much else) supports hashes for external content if you use integrity
https://w3c.github.io/webappsec-csp/#external-hash

So really this is an unsupported feature.

Comment 7

[Frederik Braun [:freddy]]

(In reply to Daniel Veditz [:dveditz] from comment #6)
> So really this is an unsupported feature.


Ah, my bad. Thanks for clarifying in my stead.

Comment 9

[hotaru]

shouldn't strict-dynamic be disabled until this bug is fixed, since this bug causes pages that actually try to use strict-dynamic to break in firefox?

Comment 10

[Colin]

I just ran into this exact problem while testing my CSP in Firefox using strict-dynamic and a series of whitelisted hashes. My script tags have an external source specified, with no inline content. Chromium based browsers already validate external scripts according to hashes specified in the script-src directive of the Content-Security-Policy header.

This is the directive I am specifying in my Content-Security-Policy header. (I split it across multiple lines to make it easier to read)

script-src 
    'self' 
    'sha256-Dxn+cR58x5haydQ1S/lvgMBLRahDCNxsakeubYGDJD0=' 
    'sha256-WiXxwK6878G5m29xzbpoI/6mHv7Otw1epCuigPupglA=' 
    'sha256-B5Xt87JgO41oTYQ2MabCc4UUuiVbcT/ingSs4HJHt1U='
    'sha384-PATEUYk+I+xYJMszEYrKkEyeuJZ04PWuSFUqoPZqptqMabAvHAtyhiRUs2BlHwfC' 
    'sha384-3hJWGurleA0oWvAljgl6WvgG4k8sXcGYwJaiRCjEH7ftI0Sr6es+cPwFlEzhv0Hl' 
    'strict-dynamic' 
    'unsafe-inline' 
    https: 'self';

This is introducing a pain point for an enterprise SPA I am currently working on. Because nonces must be used only once, our static file server must contain middleware to inject the nonce into HTML pages for every request. Also, since our application is supported by a Service Worker, we must also have similar code to generate a cryptographically strong nonce, and again, modify the HTML content before it is delivered from the browser's cache. This is a lot of maintenance and extra work that could be mitigated by allowing the use of whitelisted hashes for external scripts.

I hope my input helps! Safari is also exhibiting similar behavior, which means that I'll be finding another solution to my CSP besides hash whitelists either way.

Comment 11

[Colin]

According to developer.mozilla.org

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/script-src

The following section of Mozilla's docs on CSP mentions that CSP 3.0 allows the use of hashes for external scripts.

'<hash-algorithm>-<base64-value>'

A sha256, sha384 or sha512 hash of scripts or styles. The use of this source consists of two portions separated by a dash: the encryption algorithm used to create the hash and the base64-encoded hash of the script or style. When generating the hash, don't include the <script> or <style> tags and note that capitalization and whitespace matter, including leading or trailing whitespace. See unsafe inline script for an example. In CSP 2.0, this is applied only to inline scripts. CSP 3.0 allows it in the case of script-src for external scripts.

Comment 12

[Tony Davis]

Reading the above, It is unclear exactly how far (how well) the CSP "strict-dynamic" is supported in the current FF versions.
Quoting from Daniel Veditz above:
"CSP 3 (of which we've done strict-dynamic but not much else) supports hashes for external content if you use integrity
https://w3c.github.io/webappsec-csp/#external-hash" --- hence the question.

Certainly I have seen blockages in console logs during site logins lately, where "strict-dynamic" is specified. I notice that some sites use the modernizr-x.x.x.min.js script, to check the supported features in the client browser (to presumably dynamically tailor the cookie and script run-order or selection), and of course older versions of this might not pick up on later enhancments within FF, or other browsers.
Other technical errors can also ensue:
"Layout was forced before the page was fully loaded. If stylesheets are not yet loaded this may cause a flash of unstyled content.
modernizr-2.5.3.min.js:4:3612" (But Not necessarrily related to CSP/cookie processing, more likely a ReCaptcha block).

I feel that there are many reports of users find that site browsing/rendering/login can fail in odd ways in FF, but works on Chrome. I understand it is tough right now with CSP-3 still in draft, but making FF more seamless in these areas, may help to retain loyal users and attract more developers to code for FF, rather than it being an after-thought, or push fixing their site to be compatible with FF, into the "too hard" basket.

Comment 13

[Daniel Veditz [:dveditz]]

Narrowly speaking, Firefox follows the CSP spec algorithms for "strict-dynamic". Roughly speaking, the "strict" part refers to ignoring 'unsafe-inline' and any host expressions, and the "dynamic" refers to allowing approved scripts to insert other scripts at runtime ("non-Parser inserted"). This 'strict-dynamic' mechanism is fully supported.

Of course, you need an allowed script before it can start dynamically inserting more scripts. That's going to be a nonce- or sometimes hash- source since 'strict-dynamic' doesn't disable those. Firefox's lack of support for hashes on remotely loaded scripts does restrict your options for that starting script, but it also restricts the hash options for pages that do NOT use 'strict-dynamic'. It's a separate part of CSP-3.

Firefox is not CSP-3 compliant. It's CSP-2 compliant with a sprinkle of CSP-3 features that could be used if you're careful. In Firefox at the current time 'strict-dynamic' is best used with a nonce- based script approach.

If you find sites that are broken in Firefox (or any other browser) because of CSP (or anything else) you can report that at https://webcompat.org and the team there will figure out what's going on, and work with the site developers and browser developers to find a solution.

Comment 15

[Sebastian Neuner]

Thanks for your updates, Daniel! I was actually pretty surprised that Firefox was the only major browser that doesn't support that.

Is there an ETA when Firefox will be compliant on this part of the CSP3 specs (https://w3c.github.io/webappsec-csp/#external-hash)?

Thanks!

Comment 16

[Lukas Weichselbaum]

FYI: Web platform tests are here: https://wpt.fyi/results/content-security-policy/script-src/script-src-sri_hash.sub.html?label=master&product=chrome%5Bstable%5D&product=edge%5Bstable%5D&product=firefox%5Bexperimental%5D&product=safari%5Bstable%5D&aligned&view=subtest&q=sri

Comment 17

[Sebastian Neuner]

Thanks Lukas!

Comment 18

[zichao.xu]

Hi firefox team, Is there any ETA when Firefox will fix this issue?
We depend a lot on this bugfix to improve our site reliabilty. This issue blocks us implementing CSP inline-script integrity check.

Comment 19

[BugBot [:suhaib / :marco/ :calixte]]

Redirect a needinfo that is pending on an inactive user to the triage owner.
:freddy, since the bug has recent activity, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 20

[Frederik Braun [:freddy]]

We're developing a prototype, but can not commit to an ETA.
Is this going to make a citrix product unusable with Firefox or just less protected?

Comment 21

[zichao.xu]

(In reply to Frederik Braun [:freddy] from comment #20)

We're developing a prototype, but can not commit to an ETA.
Is this going to make a citrix product unusable with Firefox or just less protected?

Thanks for reply Freddy. We have to use another less protected way in our developling product with Firefox because of this bug.

Comment 22

[david]

I have the same problem !

With FIREFOX:

with 'self' only my script files are loaded. But the javascript code from ajax is refused.
with 'strict-dynamic' my script files are refused, on the pretext that none of the sha384 hashes of the integrity attribute correspond to the content of the sub-resource !!???
my simple php code for test. I should see "javascript yes!!!!!!!!!!!!" in the console. It's ok with chrome. Firefox refuses and does not load me jquery

<?
header("Content-Security-Policy: script-src 'strict-dynamic' 'sha384-I7/UTpkJas2maMjJpGmrvEgQecqO8Dta/9Wwh+cQrH6Jj984WRRFhWg4MV/oTkIW' 'nonce-0006c7f26c'; ", true);

if(isset($_GET["ajx"]))
{
?>
YES
<script>
console.log('javascript yes !!!!!!!!!!!')
</script>
<?
exit();
}
?>
<!DOCTYPE html>
<html>
<head>
<script src="../public/plugins/jquery/jquery-3.3.1.min.js" integrity="sha384-I7/UTpkJas2maMjJpGmrvEgQecqO8Dta/9Wwh+cQrH6Jj984WRRFhWg4MV/oTkIW" crossorigin="anonymous"></script>
</head>

<body>
body <div id="zz"></div>
<script nonce=0006c7f26c>
console.log('yes')

$.ajax({
    type:"POST",
    url:'/public/testCSP.php?ajx=1',
    async:false,

    success : function(response, statut)
    {
        if(response)
        {
                $('#zz').html(response);
        }
    },
    error : function(resultat, statut, erreur){ if(erreur) alert("Erreur  : "+erreur)},
    complete : function(resultat, statut){   }
});
</script>

</body>
</html>

Comment 23

[kris.wong]

We would love to see this fixed. It would make using a strict CSP with hashes and subresource integrity a snap for our Angular application. As it stands, not only does FF not support this feature, but it also doesn't use the fallback "unsafe-inline", which means it's just broken completely.

Due to this, we have had to invest quite a large amount of work into injecting a random nonce into what would normally be static pages served with AWS CloudFront. It's definitely not an elegant workaround. It also has a cost associated with it from having to use a lambda@edge function to inject the nonce.

Comment 24

[BugBot [:suhaib / :marco/ :calixte]]

Redirect a needinfo that is pending on an inactive user to the triage owner.
:freddy, since the bug has recent activity, could you please find another way to get the information or close the bug as INCOMPLETE if it is not actionable?

For more information, please visit BugBot documentation.

Comment 25

[Tom Schuster (MoCo)]

Attachment: Bug 1409200 - Add integrityMetadata to nsILoadInfo. r?freddyb

Comment 26

[Tom Schuster (MoCo)]

Attachment: Bug 1409200 - Use nsILoadInfo for nsIContentSecurityPolicy::ShouldLoad. r?freddyb

Depends on D179822

Comment 27

[Tom Schuster (MoCo)]

Attachment: Bug 1409200 - Implement CSP-3 support for hashes matching external resources with an integrity attribute. r?freddyb

Depends on D179823

Comment 28

[Frederik Braun [:freddy]]

Release Note Request
[Why is this notable]: support for a new CSP mechanism that seems to be somewhat desirable
[Affects Firefox for Android]: also supported
[Suggested wording]: Firefox now supports CSP3 "external hashes"
[Links (documentation, blog post, etc)]: N/A, may link to MDN or W3C spec.

Comment 29

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e8a83b1e7e08
Add integrityMetadata to nsILoadInfo. r=freddyb,necko-reviewers,kershaw
https://hg.mozilla.org/integration/autoland/rev/a66ea7d7f812
Use nsILoadInfo for nsIContentSecurityPolicy::ShouldLoad. r=freddyb
https://hg.mozilla.org/integration/autoland/rev/ea10214aa35f
Implement CSP-3 support for hashes matching external resources with an integrity attribute. r=freddyb

Comment 30

[Web Platform Test Sync Bot (Matrix: #interop:mozilla.org)]

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/40469 for changes under testing/web-platform/tests

Comment 31

[Tom Schuster (MoCo)]

I just found out that SRI is in some ways extremely dangerous: If all hashes in the integrity attribute are invalid, it will allow the script to run instead of just blocking it.

After some testing I realized this could potentially be used to exploit a difference between ParseSRIMetadata and SRICheck::IntegrityMetadata. The later has an arbitrary limit of 512 tokens. So by adding 512 invalid hashes (e.g. a bad hash algorithm) to the integrity attribute followed by any hash that appears inside the CSP (it doesn't have to match) the script will run. The CSP implementation will ignore the 512 bad hashes and allow the script to run because hash 513# appears in the CSP. On the SRI side we will only see 512 invalid hashes and not check if (invalid) hash 513# matches and default to running the script as well ...

I think we should probably remove the arbitrary limit from SRICheck::IntegrityMetadata as well as making sure to only pass along the integrity metadata when the SRI is considered valid.

Comment 32

[Norisz Fay [:noriszfay]]

Backed out as requested by dev

Backout link

Comment 33

[Web Platform Test Sync Bot (Matrix: #interop:mozilla.org)]

Upstream PR was closed without merging

Comment 34

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b9f91d8ef9e5
Add integrityMetadata to nsILoadInfo. r=freddyb,necko-reviewers,kershaw
https://hg.mozilla.org/integration/autoland/rev/734a00fc8d22
Use nsILoadInfo for nsIContentSecurityPolicy::ShouldLoad. r=freddyb
https://hg.mozilla.org/integration/autoland/rev/9b5cc2efe332
Implement CSP-3 support for hashes matching external resources with an integrity attribute. r=freddyb

Comment 35

[Iulian Moraru]

https://hg.mozilla.org/mozilla-central/rev/b9f91d8ef9e5
https://hg.mozilla.org/mozilla-central/rev/734a00fc8d22
https://hg.mozilla.org/mozilla-central/rev/9b5cc2efe332

Comment 36

[Web Platform Test Sync Bot (Matrix: #interop:mozilla.org)]

Upstream PR merged by moz-wptsync-bot

Comment 38

[Dianna Smith [:diannaS]]

This has been added to the 116 nightly release notes
https://www.mozilla.org/en-US/firefox/116.0a1/releasenotes/

Comment 39

[Hamish Willee]

FF116 MDN docs for this can be tracked in https://github.com/mdn/content/issues/27749. Comprises addition of browser compatibility data, release note, and a section in CSP scrip-src doc about feature. Most of it is done and being reviewed.