1624914 - ETP Standard Level 2 causes trends.google.com embedded iframe to be "Blocked by X-Frame-Options Policy"

Status: VERIFIED FIXED

(Core :: Privacy: Anti-Tracking, defect, P2)

Description

[Chris Peterson [:cpeterson]]

Steps to reproduce

1. In Firefox Nightly, load https://knowyourmeme.com/memes/awesome-face-epic-smiley
2. Scroll half way down the page to the "Search" section.
3. Look for the Google Trends graph.

Expected result

The "Search" section should include an embedded Google Trends graph.

Actual result

The "Search" section has an embedded iframe with the following error:

Blocked by X-Frame-Options Policy

An error occurred during a connection to trends.google.com.

Nightly prevented this page from loading in this context because the page has an X-Frame-Options policy that disallows it.

I bisected this regression to bug 1584998 in Firefox 72 Nightly:

https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=cd45360fb1ff5110acd8d54329ab45fce7395ba0&tochange=d5bee3db1b4f0ce5ff876f08230336022ae7d8ab

However, I can only reproduce this error in Nightly, e.g. 75 Nightly produces the error but 75 Beta shows the Google Trends graph. This error is reproducible with or without Fission.

Comment 1

[Chris Peterson [:cpeterson]]

Is this a legitimate site error? That it was caused by an XFO fix for Fission makes me think it is not.

Possibly related to bug 1598362, an XFO error on Air Mozilla? That error will be fixed on the Air Mozilla server side.

Comment 2

[Christoph Kerschbaumer [:ckerschb]]

Firefox console displays:

Load denied by X-Frame-Options: “SAMEORIGIN” from “https://trends.google.com/trends/embed/explore/TIMESERIES?re…0face%2522%2C%2520awesome%2520face%2C%2520awesome%2520smiley”, site does not permit cross-origin framing from “https://knowyourmeme.com/memes/awesome-face-epic-smiley”.

which seems correct to me. Though interestingly this is working in Chrome but not in current Firefox. This is also not related to Bug 1599131, so I am wondering why we are blocking that and Chrome doesn't.

Assigning to myself to investigate.

Comment 3

[Christoph Kerschbaumer [:ckerschb]]

I think this Bug will be INVALID in the end because it seems they serve different policies based on user agent (version) sniffing.

In more detail, using mozilla-central, the frame is blocked because:

• the policy is: x-frame-options: SAMEORIGIN
• the document uri is: https://knowyourmeme.com/memes/awesome-face-epic-smiley
• and the blocked iframe uri: https://trends.google.com/trends/embed...

which makes sense, that's clearly cross origin.

Testing with Firefox release and Chrome they don't send x-frame-options headers. (FWIW, they also send a different CSP).

Chris, can you verify that my assumptions are correct? If not, please let me know, otherwise I think we mark this bug as INVALID.

The only thing that beats me is, why would they do that in the first place? I mean, why would https://trends.google.com send an x-frame-options policy of same-origin?

Comment 4

[Chris Peterson [:cpeterson]]

(In reply to Christoph Kerschbaumer [:ckerschb] from comment #3)

Testing with Firefox release and Chrome they don't send x-frame-options headers. (FWIW, they also send a different CSP).

Chris, can you verify that my assumptions are correct? If not, please let me know, otherwise I think we mark this bug as INVALID.

The only thing that beats me is, why would they do that in the first place? I mean, why would https://trends.google.com send an x-frame-options policy of same-origin?

Strange. That's not what I see. On my computer, the trends.google.com response send x-frame-options: SAMEORIGIN in Chrome 80, Edge 80, Firefox 74 Release, 75 Beta, 76 Nightly, and Firefox 76 spoofing Chrome 80's UA. So on my computer, Firefox Nightly appears to be treating x-frame-options: SAMEORIGIN differently. (I am testing with Fission disabled.)

Curiously, Edge (based on Chromium 80) behaves differently than Chrome 80. Edge blocked the embedded iframe, displaying a "trends.google.com refused to connect." error even though trends.google.com did return a proper response.

Browser	x-frame-options	Behavior
Chrome 80	SAMEORIGIN Correction: no XFO header	iframe loaded successfully
Firefox 74 Release	SAMEORIGIN	iframe loaded successfully
Firefox 75 Beta	SAMEORIGIN	iframe loaded successfully
Firefox 72-76 Nightly	SAMEORIGIN	iframe blocked with "Blocked by X-Frame-Options Policy" error
Firefox 76 Nightly spoofing Chrome 80	SAMEORIGIN	iframe blocked with "Blocked by X-Frame-Options Policy" error
Edge 80	SAMEORIGIN	iframe blocked with "trends.google.com refused to connect." error

Comment 5

[Christoph Kerschbaumer [:ckerschb]]

I just checked Safari too, which is also blocking the load and displaying the following message in the console:

Refused to display 'https://trends.google.com/trends/embed/explore/...' in a frame because it set 'X-Frame-Options' to 'SAMEORIGIN'.

I think Firefox behavior is correct by blocking the load but I still couldn't figure out why some versions of Firefox would allow the load. In my opinion we should have always blocked that.

Dan, given comment 3 and comment 4, what's your take on this one?

Comment 6

[Daniel Veditz [:dveditz]]

In Chrome the response does NOT have an XFO. the request headers look similar to ours. Looks like I'm getting a "429" error (with XFO headers). Would be nice to inform about the error on top of the XFO, but it is absolutely important that we honor XFO for 4XX pages. Poorly coded error pages (especially common with 404) are a long-standing source of XSS bugs.

The main difference between Firefox Nightly and Firefox Beta in request headers seems to be the sec-fetch- headers. In Firefox nightly

• I turned those off with the pref: trends showed up.
• turned them back on and did shift-reload: still have trends, yay?
• cleared cache using History: nope, trends are gone again (429 error)

Guess Shift-reload is not as much of a "get it all fresh" as I thought :-(

Firefox Nightly request headers:

Host: trends.google.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:76.0) Gecko/20100101 Firefox/76.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Referer: https://knowyourmeme.com/memes/awesome-face-epic-smiley
Connection: keep-alive
Cookie: [snip]
Upgrade-Insecure-Requests: 1
Sec-Fetch-Dest: iframe
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: cross-site

Chrome request headers:

accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
accept-encoding: gzip, deflate, br
accept-language: en-US,en;q=0.9
cookie: [snip]
dnt: 1
referer: https://knowyourmeme.com/memes/awesome-face-epic-smiley
sec-fetch-dest: iframe
sec-fetch-mode: navigate
sec-fetch-site: cross-site
upgrade-insecure-requests: 1
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36
x-client-data: [base64 stuff]

I don't know what that x-client-data: thing is. Could they see the sec-fetch-, assume it's Chrome and then expect whatever tracking info is in that magic header, then return 429 when they don't get it? They only seem to send it to their own sites: trends, www.google.com, youtube. Not the utility ones though: they don't send it to google-analytics, gstatic, or fonts.googleapis.com

Seems like bug 1508292 is a more likely "regressor", but also this looks like a Chrome server webcompat bug.

Comment 7

[Daniel Veditz [:dveditz]]

That doesn't explain Safari though. Safari is not sending any Sec-Fetch- headers (not many headers at all, really) and also gets the 429 error page.

Note this looks like a not well maintained error page (see comment above about why we need to honor XFO on 4xx pages!). It still has a p3p: CP="This is not a P3P policy! See g.co/p3phelp for more info." header aimed at bypassing IE 3rd-party cookie privacy controls (which I thought they got in trouble for doing).

Safari just has:

Referer: https://knowyourmeme.com/memes/awesome-face-epic-smiley
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.1 Safari/605.1.15

Comment 8

[Daniel Veditz [:dveditz]]

I guess the sec-fetch- header bug was too recent to account for your 72-75 nightly failures. What else do we do differently between nightly and release that the site can detect?

Comment 9

[Christoph Kerschbaumer [:ckerschb]]

(In reply to Daniel Veditz [:dveditz] from comment #8)

I guess the sec-fetch- header bug was too recent to account for your 72-75 nightly failures. What else do we do differently between nightly and release that the site can detect?

But even if we would send lots of different headers, that also means that there is 'no' bug within Firefox itself? In other words, if Firefox is the receiving the XFO: sameorigin header in that case then Firefox blocks correctly. I guess we can't really influence when server set certain headers, we just operate on the ones provided.

Comment 10

[Chris Peterson [:cpeterson]]

(In reply to Christoph Kerschbaumer [:ckerschb] from comment #9)

But even if we would send lots of different headers, that also means that there is 'no' bug within Firefox itself? In other words, if Firefox is the receiving the XFO: sameorigin header in that case then Firefox blocks correctly. I guess we can't really influence when server set certain headers, we just operate on the ones provided.

We can reach out to our Google contacts to ask why trends.google.com is returning different headers.

---

To correct my earlier comment 4 about claiming to see x-frame-options: SAMEORIGIN returned to Chrome:

I see that Chrome is sending two requests to trends.google.com:

1. one to https://trends.google.com/trends/embed/explore/TIMESERIES (which returns status 200 and does not return x-frame-options: SAMEORIGIN)
2. then one to https://trends.google.com/trends/api/widgetdata/multiline (which returns status 200 and x-frame-options: SAMEORIGIN)

I only see Firefox only send the first request (https://trends.google.com/trends/embed/explore/TIMESERIES). Perhaps the second is block by Tracking Protection? The first request returns status 429 (Too Many Requests) and x-frame-options: SAMEORIGIN. So this response is different in Firefox.

Comment 11

[Chris Peterson [:cpeterson]]

btw, if I disable Tracking Protection on the knowyourmeme.com page, then the Google Trends graph loads correctly.

Comment 12

[Qwerty]

(In reply to Daniel Veditz [:dveditz] from comment #6)

I don't know what that x-client-data: thing is.

https://www.theregister.co.uk/2020/02/05/google_chrome_id_numbers/

Comment 13

[Tim Spurway [:tspurway]]

:englehardt does this have anything to do with tracking protection?

Comment 14

[Johann Hofmann [:johannh]]

Looks like ETP strict is causing this somehow. I confirmed this locally. Steve, how are we handling these bugs nowadays?

Chris might not be the correct assignee anymore.

Comment 15

[Steven Englehardt [:englehardt]]

(In reply to Johann Hofmann [:johannh] from comment #14)

Looks like ETP strict is causing this somehow. I confirmed this locally. Steve, how are we handling these bugs nowadays?

Chris might not be the correct assignee anymore.

Agreed. I'll take a look. Since we've still not shipped the level 2 list this will be Nightly-only. We should consider reaching out to Google. I'm happy to take it for now.

Comment 16

[Ksenia Berezina [:ksenia]]

Attachment: Screen Shot 2020-04-20 at 5.02.44 PM.png

There is a similar issue on https://apps.admob.com/v2/payments/overview that was reported in https://github.com/webcompat/web-bugs/issues/51688 (the report is for Firefox Preview Nightly, but also reproducible on desktop).

I get this error in the console:
The loading of “https://payments.google.com/payments/u/0/embedded/buy_flow?w…bW9iLmNvbQ..&mm=e&hl=en_US&style=%3Amd&cn=%24p_axmgtnt4gzdy0” in a frame is denied by “X-Frame-Options“ directive set to “SAMEORIGIN“.

For this request x-frame-options: SAMEORIGIN is returned in the response headers and as the result the iframe is blocked. If I disable ETP, x-frame-options is no longer added and the iframe loads.

Comment 17

[Steven Englehardt [:englehardt]]

This looks similar to the issues we saw in Bug 1550899. Ni st peter to handle outreach to Google.

Comment 18

[Firefox Bug Husbandry Bot]

Because this bug's Severity has not been changed from the default since it was filed, and it's Priority is -- (Backlog,) indicating it has has not been previously triaged, the bug's Severity is being updated to -- (default, untriaged.)

Comment 19

[Firefox Bug Husbandry Bot]

Because this bug's Severity has not been changed from the default since it was filed, and it's Priority is -- (Backlog,) indicating it has has not been previously triaged, the bug's Severity is being updated to -- (default, untriaged.)

Comment 20

[Firefox Bug Husbandry Bot]

Because this bug's Severity has not been changed from the default since it was filed, and it's Priority is -- (Backlog,) indicating it has has not been previously triaged, the bug's Severity is being updated to -- (default, untriaged.)

Comment 21

[Firefox Bug Husbandry Bot]

The severity of these bugs was changed, mistakenly, from normal to S3.

Because these bugs have a priority of --, indicating that they have not been previously triaged, these bugs should be changed to Severity of --.

Comment 22

[Peter Saint-Andre [:stpeter]]

I've re-pinged the mozilla-google coordination list about this. If we don't hear back in ~7 days I'll escalate through other channels.

Comment 23

[Peter Saint-Andre [:stpeter]]

Escalation in progress...

Comment 24

[Tanvi Vyas[:tanvi]]

Hi Peter - Any update from Google?

Comment 25

[Steven Englehardt [:englehardt]]

This is still broken in Firefox Nightly. I did some testing and determined that the request to trends.google.com [0] must have some cookies on the request to return a 200, and will otherwise return a 429 with x-frame-options: SAMEORIGIN. In fact, if one clears their profile and loads the page with tracking protection disabled the iframe will still fail to load on the first pageload because no cookies are attached.

I was able to successfully load the iframe with just the NID (i.e., google preference) cookie present and with just the __Secure-3PSID (apparently an advertising) cookie present. Unfortunately they do appear to validate the cookie value so sending NID=foo doesn't work.

This is broken in Chrome incognito and in Safari since cookies aren't present. If we can find a valid value to hardcode we may be able to get this working. We could also consider exposing a temporary, in-memory cookie jar similar to the compatibility hack we built for localStorage (i.e., the partitionedHosts pref). However, this would still run into the issue that the cookie jar is empty on first-load. Thus, outreach seems like the only path forward here.

[0] https://trends.google.com/trends/embed/explore/TIMESERIES?req=%7B%22comparisonItem%22%3A%5B%7B%22keyword%22%3A%22%5C%22awesome%20face%5C%22%22%2C%22geo%22%3A%22%22%2C%22time%22%3A%22all%22%7D%2C%7B%22keyword%22%3A%22awesome%20face%22%2C%22geo%22%3A%22%22%2C%22time%22%3A%22all%22%7D%2C%7B%22keyword%22%3A%22awesome%20smiley%22%2C%22geo%22%3A%22%22%2C%22time%22%3A%22all%22%7D%5D%2C%22category%22%3A0%2C%22property%22%3A%22%22%7D&tz=240&eq=date%3Dall%26q%3D%2522awesome%2520face%2522%2C%2520awesome%2520face%2C%2520awesome%2520smiley

Comment 26

[Raul Bucata]

This is still happening using the latest version of Firefox Nightly, regardless of the status of ETP.

Tested with:

Browser / Version:Firefox Nightly 87.0a1 (2021-02-17) (64-bit)
Operating System: Windows 10 PRO x64

Comment 27

[Thomas Wisniewski [:twisniewski]]

Attachment: Bug 1624914 - Work around Google Trends sub-frame ETP breakage; r?johannh,denschub

Google Trends time-series subframes are currently broken by dFPI protections,
as they expect the Google NID cookie to be present as per their set-cookie
response header, but it is being blocked.

This patch adds a new custom SmartBlock-style shim which:

• listens for requests made for Google Trends time-series subframes.
• remembers their set-cookie response header.
• redirects the request right back to the same URL.
• subsequently passes along the saved cookie, preventing the breakage.

Comment 28

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/faa3d8badb3a

Comment 29

[Hani Yacoub, Desktop QA]

Verified as fixed on Windows 10 x64, macOS 10.15 and on Ubuntu 20.04.