424875 - Deal with server-side script access via XMLHttpRequest from local files

Status: RESOLVED WORKSFORME

(Core :: Networking, defect)

Description

[Foteos Macrides]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9b5pre) Gecko/2008032405 Minefield/3.0b5pre
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9b5pre) Gecko/2008032405 Minefield/3.0b5pre

IE and Safari support http URLs for the XMLHttpRequest
object via local files, which is useful when one wishes to access
server-side scripts in that context.  Firefox through 2.0.0.12 treats
that as a cross-domain access violation.  Fx3.0b5pre and Fx3.0b4 yield readyState 4 with .status 200 and .statusText 'OK' in that context, but everything else for the XMLHttpRequest object is null or the null string, so typical onreadystatechange handlers proceed as if the request were successful when functionally it was not.  If there is no valid security issue in that context, it would be good to make Fx3 compatible with the other browsers.  Otherwise, the .status and .statusText values should be restored to what Fx2 is loading.


Reproducible: Always

Steps to Reproduce:
1.Atempt to access an http URL with XMLHttpRequest via a local file.
2.
3.


Expected Results:  
With .status set to 200 and .statusText to 'OK' one would expect the response headers, .responseText and .responseXML to be loaded appropriately for a successful http response, but they are null or the null string.

Comment 1

[Foteos Macrides]

Attachment: testcase

Adding testcase as Bug424875.zip.  Compare its Bug424875.html as a local file versus via http://www.macridesweb.com/oltest/Bug424875.html

Comment 2

[Foteos Macrides]

Attachment: replacement testcase

Oops. Hadn't changed all of the file names in the test case to correspond with the current bug number.  Adding a replacement testcase.  Again, compare Bug424875.html as a local file versus via http://www.macridesweb.com/oltest/Bug424875

Comment 3

[Foteos Macrides]

(In reply to comment #2)
> http://www.macridesweb.com/oltest/Bug424875

Ugh! That should be http://www.macridesweb.com/oltest/Bug424875.html
(hope this report is all set now :-)

Comment 4

[Boris Zbarsky [:bzbarsky]]

Confirming.  At the very least, the "status == 200, no data" combination seems like a bad idea, and is a regression.

Comment 5

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

Don't think this is a blocker, but would be great to have fixed since its a regression.

Comment 6

[Foteos Macrides]

(In reply to comment #2)
> Oops. Hadn't changed all of the file names in the test case to correspond with
> the current bug number.

Note that in that first testcase, via the local file it correctly loads status 404, but statusText 'OK' instead of 'Not Found' and everything else null or the null string (should be loading my server's http headers, and responseText 'No input file specified.').  Check that out, too, when this bug is fixed.

Comment 7

[Foteos Macrides]

(In reply to comment #6)
> Check that out, too, when this bug is fixed.

The html file in that first testcase can be accessed via http on my server as:
http://www.macridesweb.com/oltest/Bug424875_404.html

Fx2 and Fx1.5 also load 'OK' instead of 'Not Found' in statusText (but everything else correctly) when using that http URL, so the statusText part of the problem appears to be a long-standing bug rather than a regression.

Comment 8

[Boris Zbarsky [:bzbarsky]]

> Fx2 and Fx1.5 also load 'OK' instead of 'Not Found' in statusText

The server doesn't include status text in its reply in this case.  If you look in  
nsHttpResponseHead::ParseStatusLine, you see the following code handling that situation:

188         if (!(line = PL_strchr(line, ' '))) {
189             LOG(("mal-formed response status line; assuming statusText = 'OK'\n"));
190             mStatusText.AssignLiteral("OK");

Note that per RFC 2616 section 6.1, the status text is required to be present in the response.

In other words, fix your server.

Comment 9

[Foteos Macrides]

(In reply to comment #8)
> The server doesn't include status text in its reply in this case.

That must be due to the hosting service's configuration for php.  The server does include status text when the missing files have other extensions.  It's an Apache server, but rather than trying to deal with php extensions myself via htaccess, I added the missing php file and have it do:

header('Status: 404 Not Found');

I presume that's an equivalent test.

Comment 10

[Foteos Macrides]

(In reply to comment #8)
(In reply to comment #8)
> 188         if (!(line = PL_strchr(line, ' '))) {
> 189             LOG(("mal-formed response status line; assuming statusText =
> 'OK'\n"));
> 190             mStatusText.AssignLiteral("OK");

The often inappropriate consequences of this code are not directly related to the present bug, and so I have filed a spin-off: Bug 425403

Comment 11

[Daniel Veditz [:dveditz]]

> IE and Safari support http URLs for the XMLHttpRequest
> object via local files, which is useful when one wishes to access
> server-side scripts in that context.

We don't want to do that, it is insecure. Admittedly handy for a very small number of people who know what they're doing, but a hazard waiting to happen for the vast majority of people.

The attack scenario is a random web page saved locally, or perhaps a mail attachment that forces being opened in a web browser (which means it's opened from a local file in the /tmp directory). Alternate attacks scenarios include RSS readers that save and render content from a local file, though most do their best to sanitize scripts entirely these days. When the page is opened it can then try to load data from any of your logged-in web sites, including banks, web-mail, or social sites.


> Firefox through 2.0.0.12 treats that as a cross-domain access violation.
> Fx3.0b5pre and Fx3.0b4 yield readyState 4 with .status 200 and .statusText
> 'OK' in that context, but everything else for the XMLHttpRequest object is
> null or the null string

FF3 is broken, we should fix it to work like FF2 but not like IE or Safari.

For people who _do_ know what they're doing they can explicitly grant specific files privileges to do what you're asking (and I believe that will work better in FF3 because the grant can be limited to a specific file and not just to "file://" in general. Eventually, anyway: see bug 427051).

Comment 12

[Foteos Macrides]

(In reply to comment #11)
> FF3 is broken, we should fix it to work like FF2 but not like IE or Safari.
> For people who _do_ know what they're doing they can explicitly grant specific
> files privileges to do what you're asking (and I believe that will work better
> in FF3 because the grant can be limited to a specific file and not just to
> "file://" in general. Eventually, anyway: see bug 427051).

I agree that this bug should be treated as "fix [the regression] to work like FF2 but not like IE or Safari" but in that case I think it should be made a blocker for 1.9 as I originally requested.  Hopefully jonas will agree and change that back.

I also agree that an ability to "explicitly grant specific files privileges" is the best way to enable access to one's server-side scripts via local files, and Bug 427051 is marked wanted 1.9.0.x which I presume means that the feature need not hold up the Fx3.0 formal release, but we also won't need to wait for the subsequent major release to have this capability in Fx.

Comment 13

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

Shouldn't .open simply throw if you try to XHR from a file:// url to a http:// url? I don't understand how status gets set at all?

Comment 14

[Boris Zbarsky [:bzbarsky]]

I think it should, yes.  Comment 0 makes it sound like for a bit there it didn't, because of the cross-site support.

Comment 15

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

So then I don't understand what we need to do here. What is the difference between the FF2 behavior and FF3? (Apart from that FF3 for the moment does not throw when .open is called, but rather when .send is called)

Comment 16

[Boris Zbarsky [:bzbarsky]]

With cross-site XHR backed out, I don't think there is a difference anymore.  Reporter, can you confirm?

Comment 17

[Foteos Macrides]

(In reply to comment #16)
> With cross-site XHR backed out, I don't think there is a difference anymore. 
> Reporter, can you confirm?

Yes, with tonight's Minefield it is failing to throw when .open() is called but no longer loading false data into .status and .statusText, so presumably a more complete fix for Bug 425201 is what's needed now.

Comment 18

[Boris Zbarsky [:bzbarsky]]

And we should add this as a dep on the cross-site bug so we make sure not to regress it.  Or better yet, add a testcase to mochitest if we can write one that will pass on throw and also pass on the right behavior if cross-site is supported.

Comment 19

[Foteos Macrides]

(In reply to comment #17)
> [snip] so presumably a more
> complete fix for Bug 425201 is what's needed now.

The 2008-04-17 "Followup fix with test" for Bug 425201 fixes the "regression" component of this bug, i.e., if the XHR .open() is called for an http URL via a local file, .status no longer is set to 200, .statusText no longer is set to "OK" and an exception is thrown for a cross-domain access violation.

But the feature request component of this bug, to create an adequately secure procedure for granting access for particular http URLs (especially for particular server-side scripts) via particular local files still needs to be handled in the current Minefield (Gecko/2008041907 Minefield/3.0pre rv:1.9pre).

Comment 20

[Foteos Macrides]

(In reply to comment #11)
> For people who _do_ know what they're doing they can explicitly grant specific
> files privileges to do what you're asking (and I believe that will work better
> in FF3 because the grant can be limited to a specific file and not just to
> "file://" in general. Eventually, anyway: see bug 427051).

I tried putting:

if(navigator.appName=='Netscape'&&window.location.protocol.indexOf('file')==0)
 netscape.security.PrivilegeManager.enablePrivilege('UniversalBrowserRead');

before the try/catch for doing an XHR .open() with an http URL specified, and:

if(navigator.appName=='Netscape'&&window.location.protocol.indexOf('file')==0)
 netscape.security.PrivilegeManager.revertPrivilege('UniversalBrowserRead');

after it.  I was able to do the .open() via a local file on clicking the "Allow" button in the dialog this invoked.  Also, if I checked "Remember this decision" in the dialog, thereafter the dialog stopped being presented when I used that local file to access the (server-side script) http URL.  As far as I can tell, the permission is specific for that local file.  I can edit the file, and thereby change the http URL in the .open() call, and if save the edited file in the same directory so that its file URL still applies, then the .open() still works, so the permission for an http URL is not utterly "specific" in that respect.  Is that an "unacceptible" security risk, and if not, is this already available means of meeting this bug's "feature request" component expected to persist for the formal Fx3.0 release?

Comment 21

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

Yes, UniveralBrowserRead isn't specific to loading a single URL. In fact, when you grant permission to do that you grant much more than file reading using XMLHttpRequest.

Comment 22

[Foteos Macrides]

(In reply to comment #21)
> Yes, UniveralBrowserRead isn't specific to loading a single URL. In fact, when
> you grant permission to do that you grant much more than file reading using
> XMLHttpRequest.

Yes, it enables much more, but doesn't my doing the enable just before the XHR .open() call and reverting it immediately afterwards make it functionally specific to that particular call?  I'm not familiar with the underlying Fx code, but the "Using Expanded Privileges" section of http://www.mozilla.org/projects/security/components/signed-scripts.html indicates that the enablePrivilege / revertPrivilege sequence functionally restricts the enable for only the javascript which is sandwiched in between.

Of course is would be better to have a way for invoking a dialog that shows the URL of the document which wants to make an XHR .open() call, and the cross-domain URL in the .open() call, and offers an option to approve the open only for that particular pair of URLs.  Is there an existing bug for that, and if there is, can one expect it to become RESOLVED FIXED for the formal Fx3.0 release or a formal Fx3.0.x release soon thereafter?

Comment 23

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

User dialogs suck in general, and when it comes to security they are outright dangerous. The fact that we have the dialog for enablePrivilege is something that we really want to change, and it's a hold-over from the olden netscape days.

So no, I would not expect us to add a dialog for loading a single url. Note however that there are other solutions in the work, such as Cross-site XMLHttpRequest using the Access-Control spec (please google for more info).

Comment 24

[Foteos Macrides]

(In reply to comment #23)
> User dialogs suck in general, and when it comes to security they are outright
> dangerous. The fact that we have the dialog for enablePrivilege is something
> that we really want to change, and it's a hold-over from the olden netscape
> days.
>
> So no, I would not expect us to add a dialog for loading a single url. Note
> however that there are other solutions in the work, such as Cross-site
> XMLHttpRequest using the Access-Control spec (please google for more info).

The "new feature" component for this bug seeks a "reasonably" secure way to enable a document which has been accessed via a file URL to make an XHR for an http URL (typically a server-side script, e.g., php), as can be done with IE and Safari (albeit with inadequate security as explained in Comment #11).  It is not seeking a more general solution for Cross-site XHR.  If one is simply working on the local file and then will be transfering it to the server, one commonly would not want Cross-site access at that point. While working on the document as a local file, how would setting up the server-side script to return an Access-Control header help, when the local document is being prevented from sending a request to the server-side script in the first place, i.e., what am I missing here?

The invocation of a dialog in this situation serves as an alert and thus a form of security against the attack senerio described in Comment #11 when the (less than ideal) enablePrivilege / revertPrivilege sandwich is being used.  I hope it is not eliminated before other solutions which don't "suck" are actually available :-).

Comment 27

[sworddragon2]

(In reply to Daniel Veditz [:dveditz] from comment #11)
> When the page is
> opened it can then try to load data from any of your logged-in web sites,
> including banks, web-mail, or social sites.

This ticket ( https://bugzilla.mozilla.org/show_bug.cgi?id=692677#c68 ) introduced flags to disable sending authentication data and to allow to make requests without needing the server to setup CORS (documented here: https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest#XMLHttpRequest%28%29 ). The latter flag is restricted to reviewed apps and don't currently work on the file-protocol. It also seems that the first flag was standardized here ( https://www.w3.org/Bugs/Public/show_bug.cgi?id=17242 ) to replace AnonXMLHttpRequest() but I'm not sure what the current status is.

Since applications are anyway capable to make unrestricted requests to any server if they want I'm trying to find an attack scenario that would be opened if we would make the mozSystem flag for the file-protocol work. If there is none we could make this flag work for the file-protocol to make local web applications more competitive (ideally both flags would then get standardized so that all browsers would get this ability earlier or later).

Comment 28

[Daniel Veditz [:dveditz]]

Nothing is changed from that comment: "Again, this is *only* available to reviewed apps, and can't be used in web pages opened in the Firefox browser since it wouldn't be safe."

Local files are not reviewed apps, they are much more like web pages (and in fact are often just web pages that have been saved locally). The Firefox desktop equivalent to "reviewed apps" are add-ons or extensions--also reviewed--and those _are_ allowed to do cross-origin XHR.

Comment 29

[sworddragon2]

(In reply to Daniel Veditz [:dveditz] from comment #28)
> Nothing is changed from that comment: "Again, this is *only* available to
> reviewed apps, and can't be used in web pages opened in the Firefox browser
> since it wouldn't be safe."

This is what I'm pointing to: With these flags authentication data (like cookies) are not sended anymore. Can you explicate which security problem does now block enabling this feature for the file-protocol?

Comment 30

[Boris Zbarsky [:bzbarsky]]

The same security problem that necessitates CORS with preflights even for anonymous XHR, no?

Put another way, if I save some web page to a file and then open it, it should NOT be able to read everything on my not-publicly-routable intranet.

Comment 31

[sworddragon2]

(In reply to Boris Zbarsky [:bz] from comment #30)
> The same security problem that necessitates CORS with preflights even for
> anonymous XHR, no?

Isn't this just a sanity check to ensure both sides are CORS-aware?


(In reply to Boris Zbarsky [:bz] from comment #30)
> Put another way, if I save some web page to a file and then open it, it
> should NOT be able to read everything on my not-publicly-routable intranet.

There is no listdir capability for any protocol so a local site can't simply access data (and the file-protocol is limited to the current directory or any below it). And if there are still security concerns for unwanted intranet access possibly we could then still refuse access from the file-protocol to private addresses.

Comment 32

[Boris Zbarsky [:bzbarsky]]

> Isn't this just a sanity check to ensure both sides are CORS-aware?

For the cases that require a preflight, yes.  But why is CORS needed in the first place for anonymous XHR?

> so a local site can't simply access data

It can't list what data is there, but it would be able to access any data it guesses the location of, given your proposal.

> we could then still refuse access from the file-protocol to private addresses

The problem is defining "private addresses".

Comment 33

[sworddragon2]

(In reply to Boris Zbarsky [:bz] from comment #32)
> But why is CORS needed in the
> first place for anonymous XHR?

This is what this ticket is aiming for: With an anonymous XHR-request we don't need to rely on the CORS-capability of the server.


(In reply to Boris Zbarsky [:bz] from comment #32)
> but it would be able to access any data it
> guesses the location of, given your proposal.

Firefox allows already on the file-protocol unrestricted file access to any data of the current directory and any bellow it. Actually nothing would get changed here with that proposal that would give an application more access as it already has on accessing a file over file://.


(In reply to Boris Zbarsky [:bz] from comment #32)
> The problem is defining "private addresses".

Not a real problem as we would simply refuse access to the addresses that are theoretically not publicly accessable. I guess they would be:

- 0.0.0.0/8
- 10.0.0.0/8
- 127.0.0.0/8
- 169.254.0.0/16
- 172.16.0.0/12
- 192.168.0.0/16
- 224.0.0.0/24
- 239.0.0.0/8
- 255.255.255.255
- ::
- ::1
- fc00::/7
- fe80::/10
- ff01::/16
- ff02::/16
- ff03::/16
- ff04::/16
- ff05::/16
- ff08::/16
- ff11::/16
- ff12::/16
- ff13::/16
- ff14::/16
- ff15::/16
- ff18::/16
- ff21::/16
- ff22::/16
- ff23::/16
- ff24::/16
- ff25::/16
- ff28::/16
- ff31::/16
- ff32::/16
- ff33::/16
- ff34::/16
- ff35::/16
- ff38::/16


No guarantee of absolute correctness (if we go this way this has to be re-evaluated from a network expert).

Comment 34

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

Sadly there are lots of intranets which use IP addresses that aren't on that list or any such list.

Anonymous XHR already exists. That was added as part of CORS. However the CORS spec still requires the remote server to opt in to exactly because no one has been able to create a comprehensive list of intranet IPs. I can attest to this because I was one of the CORS designers (not sure if that makes me one of the network experts that you call for or not).

Comment 35

[sworddragon2]

(In reply to Jonas Sicking (:sicking) from comment #34)
> Sadly there are lots of intranets which use IP addresses that aren't on that
> list or any such list.

If the request goes out to a public address the server should probably be protected and if it goes to a private address (even if not on my list) it is solvable.


(In reply to Jonas Sicking (:sicking) from comment #34)
> Anonymous XHR already exists. That was added as part of CORS.

Now we are missing only a way to do this anonymous request without requiring the server to opt-in CORS. We have just to ensure that this introduces no security problems or that they are solvable.

Comment 36

[Boris Zbarsky [:bzbarsky]]

> With an anonymous XHR-request we don't need to rely on the CORS-capability of the server.

But we do.  That's why anonymous requests from web pages have to do CORS anyway.

> Firefox allows already on the file-protocol unrestricted file access to any data of the
> current directory and any bellow it.

Right, but we're talking about accessing random other things on the intranet.

> If the request goes out to a public address the server should probably be protected

No, you don't understand.  There are intranets that use public IP ranges internally and route stuff to them internally.  From the outside those IP ranges point to somewhere else entirely.  Yes, this is moderately insane, but people do it anyway.

> Now we are missing only a way to do this anonymous request without requiring the server
> to opt-in CORS.

We're missing it precisely because it introduces security problems.

Comment 37

[sworddragon2]

(In reply to Boris Zbarsky [:bz] from comment #36)
> But we do.

We wouldn't to have to with such a feature we are talking here.


(In reply to Boris Zbarsky [:bz] from comment #36)
> No, you don't understand.  There are intranets that use public IP ranges
> internally and route stuff to them internally.  From the outside those IP
> ranges point to somewhere else entirely.  Yes, this is moderately insane,
> but people do it anyway.

This is indeed a problem here. But the potential malicious site would still to have to make up to around 4 billion guesses since nearly every IPv4-address would be possible then. It are even much more guesses if the sensitive unprotected data isn't in the root directory of that address or if such a setup is used with IPv6.


There may also be another possibility to achieve this feature: A system-wide configuration file that can grant permissions (for example specific ones or all) to specific files and directories. A potential malicious site saved and opened locally can't do evil things then because it is not registered in this file but an installer invoked by the user or the user itself could enter locations that should be partly or fully unrestricted to this file. This wouldn't also have all the issues netscape.security.PrivilegeManager had.

Comment 38

[Boris Zbarsky [:bzbarsky]]

> We wouldn't to have to with such a feature we are talking here.

Why not?  How is this feature different from anonymous XHR from a website, exactly?  What's special about file:// ?

> But the potential malicious site would still to have to make up to around 4 billion
> guesses

Not at all.  The problem is targeted attacks, nor random ones.

> that can grant permissions (for example specific ones or all) to specific files and
> directories.

We have that.  It's called "put it into an extension".

Comment 39

[sworddragon2]

(In reply to Boris Zbarsky [:bz] from comment #38)
> Why not?  How is this feature different from anonymous XHR from a website,
> exactly?  What's special about file:// ?

We are talking about fetching data without requiring the server to opt-in CORS. Not sure what you do now to expect me to answer here.


(In reply to Boris Zbarsky [:bz] from comment #38)
> We have that.  It's called "put it into an extension".

Hard to standardize that. And with HTML5 in mind and the way web applications evolve (even complex games are going to be possible) it looks really like a tinker-solution if such an application would then have to rely on non-standardized browser-specific extensions.

Comment 40

[Boris Zbarsky [:bzbarsky]]

> We are talking about fetching data without requiring the server to opt-in CORS

Right, and Jonas and I are saying that this opens up security holes in servers and therefore isn't something we're willing to ship.

> Hard to standardize that.

That's fair, but what it sounds like you're asking for is a way to standardize some sort of web app privilege grant.  There is existing standardization effort along those lines.

Comment 41

[sworddragon2]

(In reply to Boris Zbarsky [:bz] from comment #40)
> Right, and Jonas and I are saying that this opens up security holes in
> servers and therefore isn't something we're willing to ship.

With servers you mean the intranet ones? Then yes, this is currently an issue. I wonder if there are also another issues as the ones mentioned by Daniel Veditz in comment#11 should have no effect anymore if the request is anonymous and thus don't contain the authentication data.


(In reply to Boris Zbarsky [:bz] from comment #40)
> That's fair, but what it sounds like you're asking for is a way to
> standardize some sort of web app privilege grant.  There is existing
> standardization effort along those lines.

Oh, this has already been tried or is even be more or less discussed/worked on? Hopefully this will not/has not been rejected for DRM reasons.

Comment 42

[Boris Zbarsky [:bzbarsky]]

> With servers you mean the intranet ones? 

Yes.

> Oh, this has already been tried or is even be more or less discussed/worked on?

http://w3c-webmob.github.io/installable-webapps/

Comment 43

[Marcos Caceres [:marcosc]]

(In reply to Boris Zbarsky [:bz] from comment #42)
> > With servers you mean the intranet ones? 
> 
> Yes.
> 
> > Oh, this has already been tried or is even be more or less discussed/worked on?
> 
> http://w3c-webmob.github.io/installable-webapps/

Note that the above work resulted in exactly the opposite than what is being described here:
http://w3c.github.io/manifest/

(i.e., no file:// + moar CORS!)

Comment 44

[Daniel Bodea [:danibodea]]

Attachment: reproduced in Nightly 96.png

Apparently still reproduces on the latest Nightly 96.
Does this screenshot confirm reproduction or is this bug more complicated than it seems? Thanks!

Comment 45

[Daniel Bodea [:danibodea]]

Tested with Windows 10 and Mac OS 11.

Comment 46

[Hsin-Yi Tsai (she/her) (REO Fx 129) [:hsinyi]]

(In reply to Bodea Daniel [:danibodea] from comment #44)

Created attachment 9252535 [details]
reproduced in Nightly 96.png

Apparently still reproduces on the latest Nightly 96.
Does this screenshot confirm reproduction or is this bug more complicated than it seems? Thanks!

Can you help answer this question?

Comment 47

[Valentin Gosu [:valentin] (he/him) {{ FYI: OOO 14 June - 4 August }}]

If there is no valid security issue in that context, it would be good to make Fx3 compatible with the other browsers.

I don't fully understand the context of this bug, but Firefox, Chrome and Safari seem to all agree on the same behaviour, so I think it's safe to close as WFM.