The default bug view has changed. See this FAQ.

IndexedDB same-origin policy implementation for local files with query string

UNCONFIRMED
Unassigned

Status

()

Core
DOM: IndexedDB
UNCONFIRMED
3 years ago
2 years ago

People

(Reporter: Paul Fernhout, Unassigned)

Tracking

Trunk
x86
Mac OS X
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

3 years ago
User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:29.0) Gecko/20100101 Firefox/29.0 (Beta/Release)
Build ID: 20140421221237

Steps to reproduce:

Using Firefox Nightly (tested 2014-05-04) on Mac OS X 10.6.8, I opened a locally-stored HTML file which uses an IndexedDB database for a single-page application. The application works fine when working without adding a query string. However, when I start to add query strings to the URL after the file name, things start to behave unexpectedly.

The single-page application I am developing and testing with is at: https://github.com/pdfernhout/Pointrel20140331/blob/master/source/PointrelBootstrapLoader.html

I am testing with commit: 39af1a4881. That single-page FOSS application is inspired by Mozilla Webmaker ideals, that anyone should be able to create and use web pages locally. For this test, you would need to add a new item with some text such as "Hello world" (could be anything, including HTML with JavaScript), call it "test" as the Data ID, and "Store" it locally via that button. Then click on the "Browse" button to open the saved content in another tab (or window) to view only that added content (with the app's editing features hidden). This bug report is what happens when using that generated URL with a query string.

However, this issue should be duplicate-able with any simpler IndexedDB application that runs as a local file and for which you append a query string. Just open up such an application, run it, and then add a query string to the URL and press enter in the URL bar and your data will probably appear to go away. I just tested that on Firefox 29 by saving the Mozilla IndexedDB tutorial demo to my local machine. I then opened the file in Firefox, added an entry (title and Bibliographic ID, clicked "Add Publication"). The new entry listed OK with "List database content". Then I added a query string part ("?q=foo") to the URL and reloaded the page via pressing the "Enter" key. I observed the same behavior as for the application I am working on described bedlow. That is, the data previously entered was not accessible via the URL with the query string, and a second IndexedDB SQLite database is created under storage/persistent with the query string as part of the file name.

Here are those Mozilla URLs:
https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API/Using_IndexedDB
https://mdn.mozillademos.org/en-US/docs/Web/API/IndexedDB_API/Using_IndexedDB$samples/Full_IndexedDB_example?revision=543435


Actual results:

Under Firefox 28, it seems that any query string probably made the security domain for the same origin policy use a different SQLite database file. The initially stored data was not accessible with different query strings at all, and the application reports creating a new database when accessed with a different query string. I did not test all the permutations of this before upgrading to Firefox 29, so perhaps I missed a nuance of this similar to later versions.

Under Firefox 29 and also the Firefox Nightly I tried, what seems to happen is that if I launch a new page directly from the single-page app which has the same file but adds query string, the data is accessible via the same IndexedDB database. That is is I expect. The data remains even if I press the reload icon next to the URL area. Firefox in this case seems to propagate the security domain to the new page that is opened from the previous page.

However, if I launch the single-page app via pasting in the file URL or via a bookmark, or by reloading the page by pressing the enter key at the end of the URL, then the data is not accessible if the URL has a query string. It seems Firefox is trying to load some other SQLite database file in this case. I can save data with the bookmarked version, and it seems it is saved in a different database (listed below). I can then flip between tabs which have the same URL (file and query string) and they show different content (or errors) based on how the URL (file plus query string) was loaded.

It seems that under the user profile .../storage/persistent directory, a new database was made for the second test with the query string as: file++++Users+pdf+Documents+workspace+Pointrel20140331+source+PointrelBootstrapLoader.html+view=test

The earlier database file was named:
file++++Users+pdf+Documents+workspace+Pointrel20140331+source+PointrelBootstrapLoader.html


Expected results:

It seems to me that the same local file should be in the same same-origin security domain regardless of the query string or how the file was opened. I would expect never to see such SQLite database files created with the query string in them at the end. There should only have been one database file associated with the file URL's file name on disk.

One can certainly make an argument for default restrictive same-origin policies for files compared to how some other browsers do this. However including the query string in the security domain for a file just seems problematical to me. One reason is the unexpected behavior when looking at two tabs with identical URLs referencing the same file depending on how they were opened. Another reason is because it prevents bookmarking the state of a local single-page application by including a query string -- without opening up the security policy much more broadly with setting security.fileuri.strict_origin_policy to false. As above with the app I've been working on, a use case for this expected behavior is single-page web apps that can display different locally-added content depending on the query string and which can be bookmarked to display a subset of all the added data. I would expect interest in writing such apps to continue to grow as part of the Webmaker movement.

The current behavior also does not seem to me to accord with what is described here, which does not mention query strings as part of the security restriction:
https://developer.mozilla.org/en-US/docs/Same-origin_policy_for_file:_URIs

If the current behavior is not changed, then I'd suggest at least updating that web page to be clearer about the behavior of file URLs with a query string part.
(Reporter)

Comment 1

3 years ago
As a correction to the section of the report on the expected behavior, I had mistakenly assumed that turning off "security.fileuri.strict_origin_policy" in "about:config" would change the behavior of this as a workaround so that it worked more like I expected. However, in testing that just now, nothing has changed that I've noticed about the behavior under Firefox 29 or Firefox Nightly. Apparently that setting does not affect the IndexedDB behavior in this regard. So, there seems to be no way under Firefox at the moment that I know of to make a local-file-based single-page web application that uses the query string in a "file://..." bookmark or pasted-in link to select the desired subset of previously-added data in an IndexedDB database added under a different query string for the same file.

Comment 2

3 years ago
Hi Paul,

Did you managed to find any workaround for this problem. My application works inside an iframe of different domain website and need access to local storage. Any help would be highly appreciated.

Thanks
(Reporter)

Comment 3

2 years ago
This IndexedDB issue reported for FireFox 29.0 seems to be fixed as of FireFox 40.0.2 (or likely earlier, but I just noticed it working now). So I am marking this issue resolved/worksforme.

Another workaround for my use case could have been to use a hash ("#id=foo") instead of a query string ("?id=foo") on the assumption hashs are treated differently for resolving same origins. That also works with 40.0.2, however I did not test that with earlier versions of FireFox where I'm just assuming it would work. 

@Neeraj I think the issue you mentioned (iframe use and local storage) is a different issue. You could possibly solve it by using postMessage to sending messages between windows.
https://developer.mozilla.org/en-US/docs/Web/API/Window/postMessage
Status: UNCONFIRMED → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → WORKSFORME

Comment 4

2 years ago
For me this issue persists on 41.0.1 with a fragment (hash) in the URL. I guess I'll open a new issue then.
(Reporter)

Comment 5

2 years ago
(In reply to christoph.burgmer from comment #4)
> For me this issue persists on 41.0.1 with a fragment (hash) in the URL. I
> guess I'll open a new issue then.

Thanks for the comment. I looked again at this, and it turns out I was testing using a file being served from the web, not using a locally stored file as in the original report. (It's been about a year and a half since I filed the report, so I forgot all the specifics, sorry.) When I tested with the local file according to the original instructions, it still did not work. So I am reopening this issue by setting the status back to "unconfirmed".

To address your comment, I then tested locally under Mac OS X 10.6.8 with FireFox 40.0.2 with the code modified to use a fragment/hash ("#") instead of a query string ("?"). That did work for me though however. So if fragments are causing IndexedDB to see a different origin for each fragment for your configuration, you may be seeing a different but related issue? Personally, I'd be very surprised if FireFox considered the fragment as part of the URL for IndexedDB. You could check if FireFox is creating a new sqlite database with the fragment as part of the name. I checked my own system just now and did not see any databases with obvious fragments in the name despite various testing. I did still see ones with query strings in the name though from related testing. Here is a version of my code using hash fragments instead of query strings that seems to work (which requires using the specific PointrelBootstrapLoader.html file locally):
https://github.com/pdfernhout/Pointrel20140331/tree/FireFoxBug1005634WithFragment

In general, and given that this (in my opinion) bug has been sitting around for so long (both on the user side and on the Mozilla side), it seems to me this situation relates in part to changing cultural expectations on the use of a web browser. For me, I increasingly see the web browser with JavaScript as a new non-proprietary well-supported cross-platform technology to deliver applications of all sorts for the desktop, mobile, and embedded (a bit like the proprietary VisualWorks Smalltalk could do in the 1980s way before Java). I can think that even if at the same time I feel we should have better standards for exchanging information in structured ways. To me, the app part of that means a web browser should fully support running applications from local files including all functionality -- but in a "sandbox" with fine-grained security permissions (something any OS should ideally be supporting from the ground up for all apps and subapps, but that's another story). Full functionality could include support for peer-to-peer web browser interactions without the need for a central server (like WebRTC moves towards). However, I get the feeling most people using web browsers (including likely many at Mozilla) still see a web browser as something always connecting to servers which host web pages. Even Mozilla's Webmaker movement focuses on using a server to make content, not to edit local files. To make things worse, web site creators have adopted approaches involving loading code from many sites just to make basic functionality work (which is also in part based on a third-party-advertising-based revenue model). So, thinking through this sort of security issues related to running code from local files is presumably not a priority or seems just to big an issue to wrestle with. So, it is easier to just deny all access as much as possible when loading from files (as a choice between security and convenience, as opposed to devoting substantial resources to innovation to deliver both security and convenience). In many ways the entire web browser security model (including with per-origin cookies) was unfortunately just not well thought through from the start. As Jeremiah Grossman suggests, "... Web Security is Fundamentally Broken".[1][2] Opening files saved from the web in FireFox pushes **** pain points from a fundamentally broken web security model. One might expect many web pages will be downloaded together in some common "Downloads" directory and so might otherwise interact with each other in unexpected ways regarding IndexedDB unless they are kept separate somehow -- except web apps typically would be made of many files unless they are a "single page app" (like mine at the moment), so getting them all the pages of a multi-page app loaded from files to work together usign commonly shared local data is problematical. To make the activity of running apps from files "work" generally would probably require appropriate fine-grained security restrictions, perhaps in the direction of the latest work on Android 6 with granular permission for apps or similar.[3] It might also take even more than that to think through access to common resources like a database from multiple "subapp" files (perhaps a bit like NoScript?). Personally, I don't see how anyone could surf the web without something like NoScript or its equivalent to reduce threats by controlling fine-grained permissions, and in my opinion that sort of support should be baked into the browser and the web in general. If everyone did that, then web site owners would likely reduce the number of external libraries their pages load from other domains by hosting them locally. But that is all just part of some more general solution going forward and something that would take many years to play out. It might even end up with local pages being served from some internal webserver in FireFox and other browsers as some sort of new "local web server for browsers" standard? Given Mozilla's recent emphasis on FireFox OS, perhaps permission ideas from FireFox OS will find their way into the FireFox browser eventually as part of something like that?[4]

Until then, it seems the most predictable behavior results from running apps from local servers (like say NodeJS wrapped into a dekstop application), which at least then just creates the usual security and user expectation issues and not extra ones and additional confusion from loading directly from files. It's a sad situation though. My current workaround for local files is presumably now to use hash strings (which means they can't be used for in-page movement) -- but since you say they don't work for you, even if they seem to work for me on my particular setup, I wonder if that will break on other platforms.

[1] http://www.slideshare.net/jeremiahgrossman/why-web-security-is-fundamentally-broken
[2] https://www.youtube.com/watch?v=rKmUUcyYIsk
[3] http://www.phonearena.com/news/Android-6.0-Marshmallow-app-permissions-a-closer-look_id72787 
[4] https://developer.mozilla.org/en-US/docs/Mozilla/Firefox_OS/Security/Application_security
Status: RESOLVED → UNCONFIRMED
Resolution: WORKSFORME → ---
You need to log in before you can comment on or make changes to this bug.