Closed Bug 1624914 Opened 2 years ago Closed 3 months ago

ETP Standard Level 2 causes trends.google.com embedded iframe to be "Blocked by X-Frame-Options Policy"

Categories

(Core :: Privacy: Anti-Tracking, defect, P2)

defect

Tracking

()

VERIFIED FIXED
93 Branch
Tracking Status
firefox-esr68 --- unaffected
firefox-esr78 --- disabled
firefox-esr91 --- disabled
firefox74 --- disabled
firefox75 --- disabled
firefox76 --- disabled
firefox77 --- disabled
firefox78 --- disabled
firefox91 --- disabled
firefox92 --- disabled
firefox93 --- verified
firefox94 --- verified

People

(Reporter: cpeterson, Assigned: twisniewski)

References

(Blocks 2 open bugs, Regression, )

Details

(Keywords: regression, Whiteboard: [domsecurity-active])

Attachments

(2 files)

Steps to reproduce

  1. In Firefox Nightly, load https://knowyourmeme.com/memes/awesome-face-epic-smiley
  2. Scroll half way down the page to the "Search" section.
  3. Look for the Google Trends graph.

Expected result

The "Search" section should include an embedded Google Trends graph.

Actual result

The "Search" section has an embedded iframe with the following error:

Blocked by X-Frame-Options Policy

An error occurred during a connection to trends.google.com.

Nightly prevented this page from loading in this context because the page has an X-Frame-Options policy that disallows it.

I bisected this regression to bug 1584998 in Firefox 72 Nightly:

https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=cd45360fb1ff5110acd8d54329ab45fce7395ba0&tochange=d5bee3db1b4f0ce5ff876f08230336022ae7d8ab

However, I can only reproduce this error in Nightly, e.g. 75 Nightly produces the error but 75 Beta shows the Google Trends graph. This error is reproducible with or without Fission.

Is this a legitimate site error? That it was caused by an XFO fix for Fission makes me think it is not.

Possibly related to bug 1598362, an XFO error on Air Mozilla? That error will be fixed on the Air Mozilla server side.

See Also: → 1598362

Firefox console displays:

Load denied by X-Frame-Options: “SAMEORIGIN” from “https://trends.google.com/trends/embed/explore/TIMESERIES?re…0face%2522%2C%2520awesome%2520face%2C%2520awesome%2520smiley”, site does not permit cross-origin framing from “https://knowyourmeme.com/memes/awesome-face-epic-smiley”.

which seems correct to me. Though interestingly this is working in Chrome but not in current Firefox. This is also not related to Bug 1599131, so I am wondering why we are blocking that and Chrome doesn't.

Assigning to myself to investigate.

Assignee: nobody → ckerschb
Status: NEW → ASSIGNED
Priority: -- → P2
Whiteboard: [domsecurity-active]

I think this Bug will be INVALID in the end because it seems they serve different policies based on user agent (version) sniffing.

In more detail, using mozilla-central, the frame is blocked because:

which makes sense, that's clearly cross origin.

Testing with Firefox release and Chrome they don't send x-frame-options headers. (FWIW, they also send a different CSP).

Chris, can you verify that my assumptions are correct? If not, please let me know, otherwise I think we mark this bug as INVALID.

The only thing that beats me is, why would they do that in the first place? I mean, why would https://trends.google.com send an x-frame-options policy of same-origin?

Flags: needinfo?(cpeterson)

(In reply to Christoph Kerschbaumer [:ckerschb] from comment #3)

Testing with Firefox release and Chrome they don't send x-frame-options headers. (FWIW, they also send a different CSP).

Chris, can you verify that my assumptions are correct? If not, please let me know, otherwise I think we mark this bug as INVALID.

The only thing that beats me is, why would they do that in the first place? I mean, why would https://trends.google.com send an x-frame-options policy of same-origin?

Strange. That's not what I see. On my computer, the trends.google.com response send x-frame-options: SAMEORIGIN in Chrome 80, Edge 80, Firefox 74 Release, 75 Beta, 76 Nightly, and Firefox 76 spoofing Chrome 80's UA. So on my computer, Firefox Nightly appears to be treating x-frame-options: SAMEORIGIN differently. (I am testing with Fission disabled.)

Curiously, Edge (based on Chromium 80) behaves differently than Chrome 80. Edge blocked the embedded iframe, displaying a "trends.google.com refused to connect." error even though trends.google.com did return a proper response.

Browser x-frame-options Behavior
Chrome 80 SAMEORIGIN Correction: no XFO header iframe loaded successfully
Firefox 74 Release SAMEORIGIN iframe loaded successfully
Firefox 75 Beta SAMEORIGIN iframe loaded successfully
Firefox 72-76 Nightly SAMEORIGIN iframe blocked with "Blocked by X-Frame-Options Policy" error
Firefox 76 Nightly spoofing Chrome 80 SAMEORIGIN iframe blocked with "Blocked by X-Frame-Options Policy" error
Edge 80 SAMEORIGIN iframe blocked with "trends.google.com refused to connect." error
Flags: needinfo?(cpeterson) → needinfo?(ckerschb)

I just checked Safari too, which is also blocking the load and displaying the following message in the console:

Refused to display 'https://trends.google.com/trends/embed/explore/...' in a frame because it set 'X-Frame-Options' to 'SAMEORIGIN'.

I think Firefox behavior is correct by blocking the load but I still couldn't figure out why some versions of Firefox would allow the load. In my opinion we should have always blocked that.

Dan, given comment 3 and comment 4, what's your take on this one?

Flags: needinfo?(ckerschb) → needinfo?(dveditz)

In Chrome the response does NOT have an XFO. the request headers look similar to ours. Looks like I'm getting a "429" error (with XFO headers). Would be nice to inform about the error on top of the XFO, but it is absolutely important that we honor XFO for 4XX pages. Poorly coded error pages (especially common with 404) are a long-standing source of XSS bugs.

The main difference between Firefox Nightly and Firefox Beta in request headers seems to be the sec-fetch- headers. In Firefox nightly

  • I turned those off with the pref: trends showed up.
  • turned them back on and did shift-reload: still have trends, yay?
  • cleared cache using History: nope, trends are gone again (429 error)

Guess Shift-reload is not as much of a "get it all fresh" as I thought :-(

Firefox Nightly request headers:

Host: trends.google.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:76.0) Gecko/20100101 Firefox/76.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Referer: https://knowyourmeme.com/memes/awesome-face-epic-smiley
Connection: keep-alive
Cookie: [snip]
Upgrade-Insecure-Requests: 1
Sec-Fetch-Dest: iframe
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: cross-site

Chrome request headers:

accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
accept-encoding: gzip, deflate, br
accept-language: en-US,en;q=0.9
cookie: [snip]
dnt: 1
referer: https://knowyourmeme.com/memes/awesome-face-epic-smiley
sec-fetch-dest: iframe
sec-fetch-mode: navigate
sec-fetch-site: cross-site
upgrade-insecure-requests: 1
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36
x-client-data: [base64 stuff]

I don't know what that x-client-data: thing is. Could they see the sec-fetch-, assume it's Chrome and then expect whatever tracking info is in that magic header, then return 429 when they don't get it? They only seem to send it to their own sites: trends, www.google.com, youtube. Not the utility ones though: they don't send it to google-analytics, gstatic, or fonts.googleapis.com

Seems like bug 1508292 is a more likely "regressor", but also this looks like a Chrome server webcompat bug.

Flags: needinfo?(dveditz)
Regressed by: 1508292

That doesn't explain Safari though. Safari is not sending any Sec-Fetch- headers (not many headers at all, really) and also gets the 429 error page.

Note this looks like a not well maintained error page (see comment above about why we need to honor XFO on 4xx pages!). It still has a p3p: CP="This is not a P3P policy! See g.co/p3phelp for more info." header aimed at bypassing IE 3rd-party cookie privacy controls (which I thought they got in trouble for doing).

Safari just has:

Referer: https://knowyourmeme.com/memes/awesome-face-epic-smiley
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.1 Safari/605.1.15

I guess the sec-fetch- header bug was too recent to account for your 72-75 nightly failures. What else do we do differently between nightly and release that the site can detect?

(In reply to Daniel Veditz [:dveditz] from comment #8)

I guess the sec-fetch- header bug was too recent to account for your 72-75 nightly failures. What else do we do differently between nightly and release that the site can detect?

But even if we would send lots of different headers, that also means that there is 'no' bug within Firefox itself? In other words, if Firefox is the receiving the XFO: sameorigin header in that case then Firefox blocks correctly. I guess we can't really influence when server set certain headers, we just operate on the ones provided.

(In reply to Christoph Kerschbaumer [:ckerschb] from comment #9)

But even if we would send lots of different headers, that also means that there is 'no' bug within Firefox itself? In other words, if Firefox is the receiving the XFO: sameorigin header in that case then Firefox blocks correctly. I guess we can't really influence when server set certain headers, we just operate on the ones provided.

We can reach out to our Google contacts to ask why trends.google.com is returning different headers.


To correct my earlier comment 4 about claiming to see x-frame-options: SAMEORIGIN returned to Chrome:

I see that Chrome is sending two requests to trends.google.com:

  1. one to https://trends.google.com/trends/embed/explore/TIMESERIES (which returns status 200 and does not return x-frame-options: SAMEORIGIN)
  2. then one to https://trends.google.com/trends/api/widgetdata/multiline (which returns status 200 and x-frame-options: SAMEORIGIN)

I only see Firefox only send the first request (https://trends.google.com/trends/embed/explore/TIMESERIES). Perhaps the second is block by Tracking Protection? The first request returns status 429 (Too Many Requests) and x-frame-options: SAMEORIGIN. So this response is different in Firefox.

btw, if I disable Tracking Protection on the knowyourmeme.com page, then the Google Trends graph loads correctly.

(In reply to Daniel Veditz [:dveditz] from comment #6)

I don't know what that x-client-data: thing is.

https://www.theregister.co.uk/2020/02/05/google_chrome_id_numbers/

:englehardt does this have anything to do with tracking protection?

Component: DOM: Security → DOM: Selection
Flags: needinfo?(senglehardt)

Looks like ETP strict is causing this somehow. I confirmed this locally. Steve, how are we handling these bugs nowadays?

Chris might not be the correct assignee anymore.

Component: DOM: Selection → Privacy: Anti-Tracking
Priority: P2 → --
Summary: trends.google.com embedded iframe is "Blocked by X-Frame-Options Policy" (Nightly only?) → ETP strict causes trends.google.com embedded iframe to be "Blocked by X-Frame-Options Policy"

(In reply to Johann Hofmann [:johannh] from comment #14)

Looks like ETP strict is causing this somehow. I confirmed this locally. Steve, how are we handling these bugs nowadays?

Chris might not be the correct assignee anymore.

Agreed. I'll take a look. Since we've still not shipped the level 2 list this will be Nightly-only. We should consider reaching out to Google. I'm happy to take it for now.

Assignee: ckerschb → senglehardt
Flags: needinfo?(senglehardt)
Summary: ETP strict causes trends.google.com embedded iframe to be "Blocked by X-Frame-Options Policy" → ETP Standard Level 2 causes trends.google.com embedded iframe to be "Blocked by X-Frame-Options Policy"

There is a similar issue on https://apps.admob.com/v2/payments/overview that was reported in https://github.com/webcompat/web-bugs/issues/51688 (the report is for Firefox Preview Nightly, but also reproducible on desktop).

I get this error in the console:
The loading of “https://payments.google.com/payments/u/0/embedded/buy_flow?w…bW9iLmNvbQ..&mm=e&hl=en_US&style=%3Amd&cn=%24p_axmgtnt4gzdy0” in a frame is denied by “X-Frame-Options“ directive set to “SAMEORIGIN“.

For this request x-frame-options: SAMEORIGIN is returned in the response headers and as the result the iframe is blocked. If I disable ETP, x-frame-options is no longer added and the iframe loads.

See Also: → 1550899

This looks similar to the issues we saw in Bug 1550899. Ni st peter to handle outreach to Google.

Flags: needinfo?(stpeter)

The severity of these bugs was changed, mistakenly, from normal to S3.

Because these bugs have a priority of --, indicating that they have not been previously triaged, these bugs should be changed to Severity of --.

Severity: S3 → --
Severity: -- → S3
Priority: -- → P2

I've re-pinged the mozilla-google coordination list about this. If we don't hear back in ~7 days I'll escalate through other channels.

Escalation in progress...

Hi Peter - Any update from Google?

This is still broken in Firefox Nightly. I did some testing and determined that the request to trends.google.com [0] must have some cookies on the request to return a 200, and will otherwise return a 429 with x-frame-options: SAMEORIGIN. In fact, if one clears their profile and loads the page with tracking protection disabled the iframe will still fail to load on the first pageload because no cookies are attached.

I was able to successfully load the iframe with just the NID (i.e., google preference) cookie present and with just the __Secure-3PSID (apparently an advertising) cookie present. Unfortunately they do appear to validate the cookie value so sending NID=foo doesn't work.

This is broken in Chrome incognito and in Safari since cookies aren't present. If we can find a valid value to hardcode we may be able to get this working. We could also consider exposing a temporary, in-memory cookie jar similar to the compatibility hack we built for localStorage (i.e., the partitionedHosts pref). However, this would still run into the issue that the cookie jar is empty on first-load. Thus, outreach seems like the only path forward here.

[0] https://trends.google.com/trends/embed/explore/TIMESERIES?req=%7B%22comparisonItem%22%3A%5B%7B%22keyword%22%3A%22%5C%22awesome%20face%5C%22%22%2C%22geo%22%3A%22%22%2C%22time%22%3A%22all%22%7D%2C%7B%22keyword%22%3A%22awesome%20face%22%2C%22geo%22%3A%22%22%2C%22time%22%3A%22all%22%7D%2C%7B%22keyword%22%3A%22awesome%20smiley%22%2C%22geo%22%3A%22%22%2C%22time%22%3A%22all%22%7D%5D%2C%22category%22%3A0%2C%22property%22%3A%22%22%7D&tz=240&eq=date%3Dall%26q%3D%2522awesome%2520face%2522%2C%2520awesome%2520face%2C%2520awesome%2520smiley

See Also: → 1672494
Flags: needinfo?(stpeter)

This is still happening using the latest version of Firefox Nightly, regardless of the status of ETP.

Tested with:

Browser / Version:Firefox Nightly 87.0a1 (2021-02-17) (64-bit)
Operating System: Windows 10 PRO x64

Assignee: se → twisniewski

Google Trends time-series subframes are currently broken by dFPI protections,
as they expect the Google NID cookie to be present as per their set-cookie
response header, but it is being blocked.

This patch adds a new custom SmartBlock-style shim which:

  • listens for requests made for Google Trends time-series subframes.
  • remembers their set-cookie response header.
  • redirects the request right back to the same URL.
  • subsequently passes along the saved cookie, preventing the breakage.
Status: ASSIGNED → RESOLVED
Closed: 3 months ago
Resolution: --- → FIXED
Target Milestone: --- → 93 Branch

Verified as fixed on Windows 10 x64, macOS 10.15 and on Ubuntu 20.04.

Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.