Open Bug 1626616 Opened 4 years ago Updated 4 months ago

document.referrer not set in some cross domain cases involving service workers

Categories

(Core :: DOM: Service Workers, defect, P3)

Product:
Core
Component:
DOM: Service Workers
Type:
defect

Tracking

()

People

(Reporter: psiinon, Unassigned)

References

(Regression)

Details

(Keywords: regression, Whiteboard: [necko-triaged])

OK, so this is a weird use case, but I think it could apply to more standard situations as well.
The OWASP ZAP security tool has a Heads Up Display (https://github.com/zaproxy/zap-hud) which works by injecting content into the target application. Its a consensual mitm situation so thats fine, honestly :)
The HUD is supported on Firefox and Chrome, but stopped working with FX 74.
It broke because we were relying on the document.referrer being set. Previously this worked but with FX 74 it is no longer set across domains for us.
I've done some simple cross domain tests and with those its set fine.
We know that its not being affected by any related security headers as we remove those.
After some investigations we worked out that it was broken by bug 1583700 and that the workaround was to set the browser.tabs.documentchannel pref to false. This is not ideal, and no similar settings are required on chrome.
FYI turning off eTP did not fix the problem, so I dont think its related to that.
I can supply instructions for reproducing with ZAP, although this will require installing Java :/
I havnt reproduced it yet in a standalone environment - the HUD is a bit complicated and involves multiple domains, but that doesnt mean that it wont be possible to reproduce in a more straightforward manner.
Happy to provide any other diagnostics required.

Flags: needinfo?(matt.woodrow)

Steps to reproduce the issue would be really helpful, I don't think I can make much progress otherwise.

Java would be workable if it's not possible to reproduce this any other way, but a minimal standalone testcase would make life much easier.

Flags: needinfo?(matt.woodrow)

Matt, can you take this bug?

Flags: needinfo?(matt.woodrow)
Priority: -- → P1
Whiteboard: [necko-triaged]

OK, will try to find time to create a minimal standalone test case, but in the meantime the instructions for reproducing are:

  1. Download and install ZAP: https://www.zaproxy.org/download/ (requires min Java 8)
  2. Run ZAP desktop
  3. Select 'Manual Explore' on the 'Quick Start' tab
  4. "Launch Browser" with "Firefox" selected and HUD enabled (should be the defaults)
  5. Open a suitable page, eg "mozilla.com"

At this point the HUD should be overlayed on top of the target site.
Something may appear but it will be broken, and there should be a load of errors like "SyntaxError: An invalid or illegal string was specified".
These will occur at lines like https://github.com/zaproxy/zap-hud/blob/develop/src/main/zapHomeFiles/hud/management.js#L79 as the document.referrer is empty.
You can launch Chrome to see how it should look.

I've suggested launching Firefox from ZAP as this automatically sets FX to proxy though ZAP and ignore certificate errors (ZAP has its own install specific Root CA cert). But you can manually configure FX to proxy through ZAP and import the ZAP cert as well.

If you update ZAP (via the blue arrows and bolt icon on the toolbar) and install any updates then the above process should then work correctly. Thats because the updates will cause FX to be launched with the browser.tabs.documentchannel pref to false. Applying the pref manually after launching FX should have the same effect.

I'm usually around UK office hours if you need any help with this :)

Assigning to Matt as a P1 bug should have an assignee.

Matt, if you think someone else should work on this, or it's not a P1, feel free to NI me back.

Assignee: nobody → matt.woodrow
Status: NEW → ASSIGNED

Hi Nhi.

Given that this is rare, and has only been seen to be broken with an external app that does mitm injection, then I don't think this should be P1.

We still should try to figure it out though, I've just been really busy with fission recently sorry.

I suspect this would be easier for someone who knows the referrer code well, but I'll try to get to it when I have time.

Flags: needinfo?(matt.woodrow) → needinfo?(nhnguyen)
Flags: needinfo?(nhnguyen)
Priority: P1 → P2

As of FX 81 this is failing even with the browser.tabs.documentchannel pref to false :(
Anyone know of any other settings we can use to force the referer to be set?
Can anyone explain why the referer is not being set, or ways to find this out?
The HUD still works fine in Chrome, but as an ex Mozilla employee I really dont want to drop FX support...

Flags: needinfo?(matt.woodrow)

Dragana, is there anyone working Necko that would have the time to debug this?

Flags: needinfo?(matt.woodrow) → needinfo?(dd.mozilla)

Jens, can you find someone to look into this?

Flags: needinfo?(dd.mozilla) → needinfo?(jstutte)

Firefox 74 switched browser.tabs.documentchannel to true in bug 1596682.

(In reply to Sebastian Hengst [:aryx] (needinfo on intermittent or backout) from comment #9)

Firefox 74 switched browser.tabs.documentchannel to true in bug 1596682.

(In reply to Simon Bennetts from comment #6)

As of FX 81 this is failing even with the browser.tabs.documentchannel pref to false :(

81 removed support for the pref (bug 1654922).

What's the frame situation here? Like, what is the ZAP document, and how was it requested/opened such that it expects that document.referrer is the target site?

Flags: needinfo?(psiinon)
Flags: needinfo?(jstutte)
Regressed by: 1596682, 1583700
Has Regression Range: --- → yes

There's an old case where we don't send Referer: for a meta refresh (see bug 1454022, bug 266554). Nothing to do with documentChannel, though.

Does ZAP put the inspected site inside an iframe? There's bug 1423974 and bug 1677386. The former would pre-date documentChannel so again, maybe not.

FYI I've rewritten the bit of ZAP functionality that was failing so that it no longer uses this technique - instead we pass the target URL around and inject it where necessary.
ZAP actually does the opposite - it injects ZAP iframes into the target site (hey, we're a consensual manipulator in the middle tool, we can do all sort of nasty things;)
We were using the referrer header to find out which URL the ZAP iframes were injected into so that we could be sure that we communicated with the right frames via postMessages.

Flags: needinfo?(psiinon)

(In reply to Simon Bennetts from comment #12)

FYI I've rewritten the bit of ZAP functionality that was failing so that it no longer uses this technique - instead we pass the target URL around and inject it where necessary.
ZAP actually does the opposite - it injects ZAP iframes into the target site (hey, we're a consensual manipulator in the middle tool, we can do all sort of nasty things;)
We were using the referrer header to find out which URL the ZAP iframes were injected into so that we could be sure that we communicated with the right frames via postMessages.

Does this converge towards a WORKSFORME now ?

Flags: needinfo?(psiinon)

I'm fine with that.
It is a weird edge case but in theory other people might hit it in less unusual situations.
But in that case it might be easier to gather evidence ;)

Flags: needinfo?(psiinon)

It seems from Daniel's comment we have other, more actionable bugs on file then.

Status: ASSIGNED → RESOLVED
Closed: 3 years ago
Resolution: --- → WORKSFORME

(In reply to Jens Stutte [:jstutte] from comment #13)

Does this converge towards a WORKSFORME now ?

I mean, I don't think this really makes sense - the web app is working around our bug. We should still fix it.

That said, I've been fighting ZAP for like 30 minutes now and it's pretty unwieldy:

  • I can't easily tell it to invoke a debuggable binary (it'll just pick the "normal" Firefox install, which I can't debug)
  • it takes port 8080 which means it interferes with running a local webserver and setting up a reproducible testcase
  • having shut down both my webserver and ZAP, and restarted ZAP, it now refuses to work - any website I load (either in a useful binary, by manually configuring the proxy per https://www.zaproxy.org/docs/desktop/start/proxies/, or in the copy of Firefox ZAP starts) gets me either a network login captive portal page, or a java connection error (if the other site, e.g. my previous webserver, doesn't exist)
  • having restarted a few more times and figured out that I need to click "advanced" on the network captive portal page to allow the MITM cert, even after accepting all the certs, in current Nightly ZAP doesn't work - it hides all its own iframes and there are console errors:
Some cookies are misusing the “SameSite“ attribute, so it won’t work as expected 2
Cookie “ZAP-HUD” has been rejected because it is in a cross-site context and its “SameSite” is “Lax” or “Strict”. panel.html
Cookie “ZAP-HUD” has been rejected because it is in a cross-site context and its “SameSite” is “Lax” or “Strict”. panel.html
Cookie “ZAP-HUD” has been rejected because it is in a cross-site context and its “SameSite” is “Lax” or “Strict”. growlerAlerts.html
Cookie “ZAP-HUD” has been rejected because it is in a cross-site context and its “SameSite” is “Lax” or “Strict”. drawer.html
Cookie “ZAP-HUD” has been rejected because it is in a cross-site context and its “SameSite” is “Lax” or “Strict”. display.html
14:36:26 GMT+0000 (Greenwich Mean Time) ERROR errorHandler: TypeError: can't access property "tools", panel is null
	 loadPanelTools    https://zap//zapCallBackUrl/6776912977684198840/file/utils.js 314:: {} utils.js:716:35
14:36:26 GMT+0000 (Greenwich Mean Time) ERROR errorHandler: TypeError: can't access property "forEach", tools is undefined
	 created    https://zap//zapCallBackUrl/6776912977684198840/file/panel.js 160:: {} utils.js:716:35
14:36:26 GMT+0000 (Greenwich Mean Time) ERROR errorHandler: TypeError: can't access property "tools", panel is null
	 loadPanelTools    https://zap//zapCallBackUrl/6776912977684198840/file/utils.js 314:: {} utils.js:716:35
14:36:26 GMT+0000 (Greenwich Mean Time) ERROR errorHandler: TypeError: can't access property "forEach", tools is undefined
	 created    https://zap//zapCallBackUrl/6776912977684198840/file/panel.js 160:: {} utils.js:716:35
14:36:26 GMT+0000 (Greenwich Mean Time) ERROR errorHandler: SecurityError: The operation is insecure.
	 startServiceWorker@https:    https://zap//zapCallBackUrl/6776912977684198840/file/management.js 214:2
	 @https:    https://zap//zapCallBackUrl/6776912977684198840/file/management.js 78:: {} utils.js:716:35
Failed to register/update a ServiceWorker for scope ‘https://zap//zapCallBackUrl/6776912977684198840/file/’: Storage access is restricted in this context due to user settings or private browsing mode. management.js:214:26
Partitioned cookie or storage access was provided to “<URL>” because it is loaded in the third-party context and storage partitioning is enabled. 7

Funnily enough, in this configuration, on current nightly, document.referrer is not the empty string but the correct page URL, ie this bug doesn't reproduce.

To fix the partitioned storage thingy, I went to Firefox's privacy & security, and set the tracking settings to "custom", then switched cookies to "Cross-site and social media cookies" (without the "and isolate remaining cookies"), which made ZAP work again, but broke the thing in this bug.

When looking at the network console, it indicates that now, the iframes are loaded by service workers.

So as far as I can tell, if service workers are blocked, the referrer is correct, and otherwise, it is not. If this works reliably in other browsers, and the service worker is used in the other browsers, that still feels like a bug we should investigate. I don't know how to investigate how the service worker is interfering here, but hopefully this comment is helpful in terms of getting this bug on track.

Simon, can you perhaps explain briefly how ZAP uses service workers / where the implementation lives?

Status: RESOLVED → REOPENED
Flags: needinfo?(psiinon)
Resolution: WORKSFORME → ---
Summary: document.referrer not set in some cross domain cases → document.referrer not set in some cross domain cases involving service workers

OK, so I can help with some of those things:)

ZAP listens on 8080 by default but thats easy to change: start ZAP with -port 8090 or in the UI you can change it via Options/Local Proxies - Port

ZAP uses Selenium to launch and control browsers. In the Options / Selenium you can change which Firefox binary it will use and the webdriver as well if you need to.

The latest version of the HUD (v0.12.0) will have the fix I mentioned in.
If you've got that then uninstall it (via the Manage Add-ons button), download the previous one via https://github.com/zaproxy/zap-hud/releases/download/v0.11.0/hud-beta-0.11.0.zap and install it via File/Load Add-on File...

Happy to jump on a call if that would help...

Flags: needinfo?(psiinon)

(In reply to Simon Bennetts from comment #17)

The latest version of the HUD (v0.12.0) will have the fix I mentioned in.
If you've got that then uninstall it (via the Manage Add-ons button), download the previous one via https://github.com/zaproxy/zap-hud/releases/download/v0.11.0/hud-beta-0.11.0.zap and install it via File/Load Add-on File...

I mean, I just check document.referrer via the devtools console, which still shows it being the empty string.

Happy to jump on a call if that would help...

It might, but probably with someone more familiar with how our serviceworker / referrer handling works. Maybe this is a dupe of bug 1626192 ? bug 1340659 might also be relevant.

I'm also still curious about how ZAP uses service workers - do you have any pointers into the relevant source code for that?

Yaron, are you the right person to investigate this (I looked for people who had recently fixed service worker bugs :-) ) ?

Flags: needinfo?(ytausky)
Flags: needinfo?(psiinon)

Thanks if we can get this actionable. I understood, that the scenario (benign MITM) is an edge case we can ignore. But if we can get this reproducible also in a normal situation that would help.

The ZAP HUD source code is in https://github.com/zaproxy/zap-hud but you may find the structure strange because we inject the JavaScript into HTTP(S) responses.
The JS code is all under https://github.com/zaproxy/zap-hud/tree/develop/src/main/zapHomeFiles/hud and the service worker is defined in https://github.com/zaproxy/zap-hud/blob/develop/src/main/zapHomeFiles/hud/serviceworker.js

There is some documentation on how the HUD works in the wiki eg https://github.com/zaproxy/zap-hud/wiki/Initialization

Let me know if you have any other questions.

Flags: needinfo?(psiinon)
Flags: needinfo?(ytausky) → needinfo?(jstutte)

Eden, can you please check if you see something actionable here?

Flags: needinfo?(jstutte) → needinfo?(echuang)

The bug assignee didn't login in Bugzilla in the last 7 months and this bug has priority 'P2'.
:dragana, could you have a look please?
For more information, please visit auto_nag documentation.

Assignee: matt.woodrow → nobody
Flags: needinfo?(dd.mozilla)

Eden will not be able to look into this for a while.

Flags: needinfo?(echuang)
Flags: needinfo?(dd.mozilla)
Severity: normal → S3

I've tried to track where the empty referrer is coming from but I didn't dig too far into it.
It's empty here and I didn't find any place where it's set to something else.
WorkerLoadInfo is set to that which presumably means it will just be empty?

Severity: S3 → --
Status: REOPENED → NEW
Component: Networking → DOM: Service Workers
Priority: P2 → --
Severity: -- → S3
Priority: -- → P3

It seems I face the same problem, document.referrer is empty (only in Firefox).
Referrer policy explicitly set in strict-origin-when-cross-origin, steps to reproduce very straightforward:

  • open https://www.weltbild.de/
  • click any link on the page
  • check document.referrer in console
    • it's set in 'https://www.weltbild.de/' in Chrome, Edge, and Safari
    • it's empty in Firefox (I'm using fresh installation of 114.0.2 (64-bit) version)

(In reply to github from comment #25)

It seems I face the same problem, document.referrer is empty (only in Firefox).
Referrer policy explicitly set in strict-origin-when-cross-origin, steps to reproduce very straightforward:

  • open https://www.weltbild.de/
  • click any link on the page
  • check document.referrer in console
    • it's set in 'https://www.weltbild.de/' in Chrome, Edge, and Safari
    • it's empty in Firefox (I'm using fresh installation of 114.0.2 (64-bit) version)

I can't reproduce this - the referrer is set normally for me (116 beta 2, on macOS, but also not on 114 as launched through mozregression).

Can you doublecheck on a clean profile?

Flags: needinfo?(github)

Clear a needinfo that is pending on an inactive user.

Inactive users most likely will not respond; if the missing information is essential and cannot be collected another way, the bug maybe should be closed as INCOMPLETE.

For more information, please visit BugBot documentation.

Flags: needinfo?(github)
You need to log in before you can comment on or make changes to this bug.