1458183 - Content Script XHR does not set Referer header - but does in Chrome

Status: RESOLVED WONTFIX

(Core :: Networking: HTTP, defect)

Description

[dw-dev]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:59.0) Gecko/20100101 Firefox/59.0
Build ID: 20180310025718

Steps to reproduce:


I am the developer of the Save Page WE add-on.

I have done some testing with a slightly modified version of this add-on - Save Page WE 9.5.2 (attached).

My testing was done with Firefox Nightly 61.0a1, Save Page WE 9.5.2 and this page: https://www.pixiv.net/

The  4 resources, that could not be loaded by the content script using Firefox, were successfully loaded by the content script using Chrome.

Using Firefox, the tests showed that:

- in the background script, an XHR request does not have Origin or Referer headers.
- in a content script, an XHR request does not have Origin or Referer headers.
- in a content script, a content.XHR request does have both Origin and Referer headers.

Using Chrome, the tests showed that:

- in the background script, an XHR request does not have Origin or Referer headers.
- in a content script, an XHR request does not have an Origin header, BUT DOES have a Referer header.

When sending an XHR request from a content script, why does Firefox not set a Referer header, when Chrome does?

This seems to be an inconsistency in the implementation of the APIs.


Actual results:

When sending an XHR request from a content script, Firefox not set a Referer header, but Chrome does.



Expected results:

It was expected that Firefox would perform in the same way as Chrome.

Comment 1

[dw-dev]

Attachment: Save Page WE 9.5.2.zip

Comment 2

[dessant]

Please edit the title, it's the other way around.

Comment 3

[dw-dev]

No, this is the correct title.

This bug report relates to using XHR in a content script, NOT to using content.XHR in a content script.

See Comment 21 in Bug 1454190.

Comment 4

[dessant]

Yes, we're talking about window.XHR, and it does not currently set the Referer, while in Chrome it does. The bug report title says the exact opposite.

The expected result is for window.XHR to set the Referer header when the request is made from a content script.

Comment 5

[dw-dev]

Apologies, I re-read the bug title a couple of times before my last post, but somehow I thought it was correct.  Too much haste, or possibly brain doing some tranposition!

It is now corrected.

Comment 6

[Cosmin Muntean [:cmuntean], Ecosystem QA]

I am assigning a component to this issue in order to involve the development team and get an opinion on this.

Comment 7

[Kris Maglione [:kmag]]

If you want the referer header to be set, use content.XMLHttpRequest.

Comment 8

[Shane Caraveo (:mixedpuppy)]

Kris, you missed the important difference and what dw-dev is encountering.

chrome xhr: only referer
fx xhr: neither origin or referer
fx context.xhr: both origin and referer

There is a difference in behavior between chrome and firefox with XHR in content scripts, and that affects cors behavior.

Comment 9

[Shane Caraveo (:mixedpuppy)]

I'm guessing that the logic for setting the referrer doesn't take extension principals into account, but that's just a guess.

https://searchfox.org/mozilla-central/rev/c0d81882c7941c4ff13a50603e37095cdab0d1ea/netwerk/protocol/http/HttpBaseChannel.cpp#1609

Comment 10

[Kris Maglione [:kmag]]

I didn't miss the difference. We don't set eithet header in the sandbox XHR because expanded principals have no origin URI. We set both with content.XHR because setting both makes sense. If the extension wants to rely on CORS, it can use content.XHR. If not, it can use the sandbox XHR.

Comment 11

[Shane Caraveo (:mixedpuppy)]

So is there a way to get an XHR with referrer and without origin?  Per bug 1454190 comment 0 the lack of that prevents fetching some resources.

Comment 12

[Kris Maglione [:kmag]]

Not without using the webRequest API. But sending a request with Referer but not Origin is... silly.

I suppose if you can find someone like smaug who disagrees, though, we can re-open.

Comment 13

[Shane Caraveo (:mixedpuppy)]

(In reply to Kris Maglione [:kmag] (long backlog; ping on IRC if you're blocked) from comment #12)

> I suppose if you can find someone like smaug who disagrees, though, we can
> re-open.

And that is why I moved it to core networking, to get input from that team.

Comment 14

[dessant]

Having a valid referrer sent is also useful for downloading content with hotlink protection. When you send a request to a URL, you already have permission for the host, and you should not have to fiddle with CORS to send the Referer header (content.*).

We have to resort to intercepting our own requests (which requires new permissions) just to set the Referer header while still benefiting from the privileged context to request cross-origin resources:
https://bugzilla.mozilla.org/show_bug.cgi?id=1454190#c20

Comment 15

[Olli Pettay [:smaug][[e-mail]]]

(don't expect any too fast reply from me atm. So you may want to ask someone else)

Comment 16

[dessant]

Kris, I can see how certain things seem silly initially, but once you use these APIs, valid use cases do emerge. If downloading content from the internet reliably and without extraneous permissions (webRequest and webRequestBlocking) is silly, the least to be expected is to explain why before closing the bug report repeatedly.

Comment 17

[Olli Pettay [:smaug][[e-mail]]]

(This is really a question to web extensions folks. Not sure what kind of feedback I could give here.)