970136 - HTTP referer: Allow to send target host as referer when crossing domains

Status: RESOLVED WORKSFORME

(Core :: Networking: HTTP, enhancement, P5)

Description

[Ben Bucksch (:BenB)]

Relevance: HTTP referers can be a privacy issue, because they report information about my visit on site A to site B.

Both bug 55477 (dormant since > 10 years) and bug 822869 (actually commited) implemented a number of options for HTTP referers. They can limit the referer to the hostname only, or to the same domain only.

One option is missing:
- if source and target hosts match: send full referer
- otherwise (i.e. it's a cross-origin request): send the current host's root
This keeps certain hosts happy.

Example:
* Link: http://www.no-more-anonymity.com/why-all-that-happens.html -> http://www.newspaper.com/article/the-presidents-secret.html
* Currently: http://www.newspaper.com/article/the-presidents-secret.html sees http://www.no-more-anonymity.com/why-all-that-happens.html as referer
* With cross-domain referers disabled: http://www.newspaper.com/article/the-presidents-secret.html sees no referer
* With shortened referer disabled: http://www.newspaper.com/article/the-presidents-secret.html sees http://www.no-more-anonymity.com/ as referer
* This proposal: http://www.newspaper.com/article/the-presidents-secret.html sees http://www.newspaper.com/ as referer

Advantages:
* Referers within sites continue to work. These allow sites to follow user trails. They pose no additional privacy threat, because the site sees both requests anyway, and with cookies can combine them anyway.
* Information across sites is completely suppressed, this preserving privacy. The domain can in some cases already be revealing, like in the www.no-more-anonymity.com -> newspaper case.

Comment 1

[Dan Auerbach]

Increasing the granularity of referer options seems like a positive thing overall and I have no objection to this proposal. However, I'm curious if you have empirical evidence to the extent to which sending the current host's root keeps certain hosts happy? I ask because my intuition is that the vast majority of cases where a referer "needs" to be sent occur in requests within the same domain (e.g. CSRF+HTTPS protection, hotlinking prevention), and hence setting network.http.referer.XOriginPolicy ought to do the trick.

In any case, now that 822869 has finally landed (woo!) it would be nice to do some testing to arrive at the most restrictive policy we can get away with with a minimum of breakage. I'd love to hear from the privacy team at Mozilla about the best way to test and push this forward.

Comment 2

[eyal gruss]

by "source and target hosts match" do you mean just base (root) or full domains?
by "send the current host's root" does current mean target host?

Comment 3

[Ben Bucksch (:BenB)]

> by "source and target hosts match" do you mean just base (root) or full domains?

I leave this open.
We could make it configurable, too.

> by "send the current host's root" does current mean target host?

Yes:

> > * This proposal: http://www.newspaper.com/article/the-presidents-secret.html sees http://www.newspaper.com/ as referer

Comment 4

[Tsudojon Yurohenyo]

All other network.http.referer options are at their default values, for each presented case.

============================================================
CASE 1: Pixiv.net breaks if XOriginPolicy=1
(works if only using spoofSource=true)
------------------------------------------------------------

network.http.referer.XOriginPolicy = 1

- Will send proper referer if domain name is the same.
- Will not send any referer header if domain name is different.

This breaks Pixiv.net (https://en.wikipedia.org/wiki/Pixiv -major image publishing platform in Japan)

http://www.pixiv.net/member_illust.php?id=47488

No picture thumbnails are listed in the page content.

These pictures come from the pximg.net domain, which checks if the referrer is either pixiv.net or pximg.net.

============================================================

============================================================
CASE 2: Maif.fr breaks if spoofSource=true
(works if only using XOriginPolicy=1)
------------------------------------------------------------

network.http.referer.spoofSource = true

- Will not send any referer header if addressed was entered manually from the address bar.
- Will send the FQDN of the requested URL itself, as a referer, when first following a link from another FQDN.
- Will send the requested URL itself (FQDN + path), as a referer, in all other cases (links from same FQDN and external resources).

This breaks Maif.fr (https://fr.wikipedia.org/wiki/Mutuelle_d'assurance_des_instituteurs_de_France -One of the main French insurance company):

https://www.maif.fr/

After one second, a menu bar which should appear at the very top of the page, does not appear. This menu bar includes the only login form accessible from this page.

Even going manually to the secondary login form (https://www.maif.fr/service/identification/valideridentification.action) will not work, because it is refusing login (error message when trying to login, saying "the requested URL does not exist"), probably checking for a specific referer URL (this secondary form works without issue with default referer options).

============================================================


Using just spoofSource=true for Pixiv.net works.

Using just XOriginPolicy=1 for Maif.fr works.

(Using both options at the same time will make both websites fail).


PROPOSED SOLUTIONS:

A) If using both XOriginPolicy=1 (or 2, to send proper referrer only the full FQDN is the same, while 1 matches only the main domain name) and spoofSource=true, Firefox should send the proper referer if the domain name is the same (or full FQDN, if XOriginPolicy=2), which does not seem like a big privacy issue (you are on the same domain -or FQDN if XOriginPolicy=2-, they already know all your GETs... they would only have your more precise path through their pages), but spoof the referer in all the other cases.

B) Otherwise, another option will have to be created. Like:
network.http.referer.XOriginFallback
0 = Don't send any referer header if XOriginPolicy would prevent it (default, current behavior).
1 = If XOriginPolicy prevents sending a proper referer, spoof referer header (send the destination's own FQDN+path).

(With both solutions, XOriginTrimmingPolicy would have to be respected too, so as not to send the query string, or path, if so desired, although this might fail on some websites).

Thus, Maif.fr would receive the proper referer it awaits, and the image hosting domain used by Pixiv.net would receive a spoofed referer as its required referer (luckily it accepts itself too, not just pixiv.net).


There would be only one case left: resources on completely different domains requiring the domain name of their main website, as a referer (without having to enable sending the proper referer to all external resources, which would in part ease tracking by third-parties -the website sure can always send the referer URL or some page identifier through the query string, but that's slightly more complex, and few do it, at least for now...). But I supposed you'll say this is left to addons (that seems to be the conclusion of bug #966505 anyway...).

Comment 5

[Firefox Bug Husbandry Bot]

Bulk change to priority: https://bugzilla.mozilla.org/show_bug.cgi?id=1399258

Comment 6

[Anne (:annevk)]

As shown in bug 1589074 this is possible these days thanks to Referrer Policy.