Closed Bug 507361 Opened 15 years ago Closed 13 years ago

localStorage doesn't work in file:/// documents

Categories

(Core :: DOM: Core & HTML, defect)

Product:
Core
Component:
DOM: Core & HTML
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla8

People

(Reporter: jay, Assigned: mayhemer)

References

(Depends on 1 open bug)

Details

Attachments

(3 files, 3 obsolete files)

Example file of how localStorage does not work on file:// protocol
1.36 KB, text/plain
Details
queryinterface tweak
1.64 KB, patch
mayhemer
: review+
Details | Diff | Splinter Review
testcase
399 bytes, text/html
Details 
URL works in a local directory with cookies for Opera and Safari,
but Mozilla using localStorage requires the file to be online to be read or written to.

from a developer perspective this can be frustrating.
presumably users working offline might also expect to be able to read and write as per cookies...

raising with webapps, apologies if componenent is 'wrong' only just beginning to get a handle on this...
Do you mean "offline" like "http url but not on a network" or do you mean file:// url?

This bug needs clear steps to reproduce, for what it's worth (and they would presumably have made my question above unnecessary).

This also seems to have nothing to do with cookies...
Component: Networking: Cookies → DOM: Mozilla Extensions
QA Contact: networking.cookies → general
#2 Zbarsky,

as I mentioned in the description, my knowledge of components is limited.
please move to Web Storage, but where is that?

"offline" like "http url but not on a network" works as expected
ie open attachment, disconnect, open from history, click on failure, refressh for 'success'.
I raised this issue with the WG as it seems relevant, is closely related, but not explicitely stated afaict.

this bug relates to "file:// url"

steps to reproduce:

open attachment, click on failure, refresh page, note 'success'
save attachment locally, 
open file, click on failure, refresh page, note 'failure'
Ah.  For file://, the question is what constitutes "same domain", given that file:// URIs don't have domains.  Storage binds to domains, after all...  

Note that this means that all sorts of things don't work "right" from file:// URIs (XMLHttpRequest, document.domain, etc, etc), so testing from file:// URIs is not really useful in any case.
Likely will be fixed in bug 495337
Depends on: 495337
#4

'best effort' is the general expectation,
not miracles which XMLHttpRequest might.
(In reply to comment #5)
> Likely will be fixed in bug 495337

I'm not sure it will be. There is a problem to obtain an origin for files on some place of code (I don't exactly remember where). There is a discussion in bug 357323 about this.
Shouldn't be a problem if all your stuff is in a single file (as here).  If you have multiple files, then you get worries. Maybe.  If it's just keyed by principal, it should Just Work.
OK, so I'm trying to make sense of this bug. I'm removing the testcase keyword and the URL, since they don't appear to have anything to do with this bug.

I have also changed summary to what I believe correctly represents what this bug is about. Honza, Boris, is the new summary accurate? I'm also not sure how this is different from bug 357323 - only that this one is about localStorage and that one is about sessionStorage?

Also, if the new summary is correct, this seems to be a duplicate of bug 469192 or vice versa.
URL: https://bug506639.bugzilla.mozilla.or...
Keywords: testcase
Summary: Is localStorage read and write-able offlne? → localStorage doesn't work in file:/// documents
Assignee: nobody → honzab.moz
Honza: are there any tests I can write or anything else that help with this bug? Be glad to help.
Given that localStorage doesn't play well in a multiprocess world (it implied a global lock last I recall), shouldn't anything desiring the feature here (that is part of the browser proper) be using something else?  I seem to recall there being an alternative; we should use that if we can.
Aha. I see the great need for a more robust storage API now.
It would seem to me that, just like when using 'file://' based cookies, the only logical way to proceed here is that all 'pages' launched from 'file://' URLs use the same DB. As Comment #4 points out, pages launched from 'file://' URLs don't really have a domain.

In any case, it looks like the Webkit guys have solved this in some fashion, as this works in both Safari and Chrome. My guess is that they punted and just made all 'file://' pages share the same DB.
At the very least, if it’s not going to work for the time being, why isn’t a warning triggered into the error console? or an error triggered when setItem is called?

Firefox currently has no chrome to list domain/objects stored localStorage, so you cant easily test why its failing - is this a subtle signal that localStorage isn’t being encouraged at all?

I have a single document application that should be able to work when served over HTTP or sat on your local machine. 

file://machine/path/file domain machine origin obvious
file:///home/john/2011-02-10.webgl.html domain unknown - obviously shouldnt use `localhost` as domain for security reasons, pick firefox-machine. put origin as the path, and be done with it. Then developers can use the damn thing.
I work on applications that we run locally as demos, stand-alone on a computer from the local file system. When we learned of the HTML5 localStorage option, we thought that it alleviated the limitations of using cookies. Then trying it today, we found that it doesn't work on FireFox. Although it does work on chrome, they won't load dojo locally. Safari does work for us, but we have standardized on using FireFox for demos as most people have it and it is usually quite advanced. 

We want to load some data with index.html and then use it on subsequent pages. Although I was able to load the data on index.html and read it there, when I went to the second page, localStorage was empty.

Setting up a localhost server isn't really an option for us as we distribute our demos to non-technical people.  We currently ask them to unzip the archive and load index.html in FireFox. Asking them to set up a localhost server would be DOA for our demos.

Therefore, I'm voting for finding some solution enabling localStorage to run locally, the local file system.
is there any progress ?
is it likely to reach firefox 4 ?
This behavior will not be changed in Firefox 4.

It's not clear whether it will ever be changed at all.  It's certainly a low priority to look into ways of changing it without breaking the relevent spec in the process.

If someone wrote a patch for this, it would probably be accepted, though.
What kind of patches would be acceptable without breaking the spec too much?
What John Evans above said - same here. 

I see a browser as a hyper-card-based-multi-media viewer, not a sever-centric internet channel. It should not be limited to, or care where, the data resides. 

It has been evident for a long time that cookies are inadequate for storing complex page and web app states. The localStorage spec is designed for this issue, and reading it you expect it to act locally - to the client.
(In reply to comment #19)
> What John Evans above said - same here. 
> 
 The localStorage spec is designed for this
> issue, and reading it you expect it to act locally - to the client.

Sorry, not "locally to the client", but locally to wherever the site/project/app resides.
i think the app storage was meant to replace cookies
and it should be the standard for html5 web applications
It's kind of ridiculous that this hasn't been fixed in about 2 years. It's not brain science or rocket surgery. The file:// protocol needs to stop being treated as the black sheep poor step-child of the supported family, especially with "web" apps, "work off-line" and related issues becoming more prominent. Browser are not just for browsing the web anymore.

This can't be more than a few lines of code to use the first directory in a file:// specification instead of the domain name as the "name-group".
It's not, but that approach is a giant security hole (e.g. on any of the current OSes that would put all local files in the same localStorage bin!).
ok, for every problem there is a solution out there:

I understand the security issue,
we can workaround it:

for example:
limit the access to just one flat dir - never allow descending to subdirs
and named it so it is obvious the files there are in danger:
.localStorageAreaKeepPersonalFilesOut

if you support this it will make a lot of serverless usecases possible
Boris -

So, in that, you all have a difference of opinion from the Webkit folks?

https://bugs.webkit.org/show_bug.cgi?id=20701

Cheers,

- Bill
> limit the access to just one flat dir

Sure, but that becomes a lot more work at that point, which is why no one has done it yet.

> So, in that, you all have a difference of opinion from the Webkit folks?

Yep, we think their behavior is a security hole.
How can this be any more of a security hole than http:// or https://? All you have to do is treat file:// EXACTLY the same way and there is no security issue. All of the data would be stored in the same local location however it is currently stored for other protocols.

The only issue is what to use as the "domain" differentiator (I don't know the exact terminology here). I mentioned the "first directory" above. It should actually be the drive specifier and first directory. Example:

file:///C:/Users/Test/subdir/local.html

The "domain" would be "C:/Users". (or "file:///C:/Users", however it currently works, or maybe "normalized" to "C__Users")

(instead of "http://example.com" or "example.com")

That would give the user as many "local domains" for the file:// protocol as they want to set up in their file system as first-sub-root directories under C: or D: or F:, etc.
> the only issue is what to use as the "domain" differentiator

That's the big issue, yes.

> The "domain" would be "C:/Users".

That is NOT acceptable.  That allows any web page you happen to save to disk to read the local storage of all file:/// pages.  We are NOT going to implement that.

> That would give the user as many "local domains" for the file:// protocol as
> they want

No normal user would ever set up extra drives to save web pages to.  Ever.  It's no OK to expose those users to security/privacy bugs by default.

Now can we please stop adding noise to this bug rehashing what's already been said?
It would probably be even more complicated to implement, but somehow specifying a manifest file for the 'domain' would be ideal.

On the non-ideal but possibly sufficient side, maybe the app could be allowed to specify it's own document.domain, which would then be used to separate it in localStorage.

I don't want to 'spam' this bug -- the goal with the above is to determine the contours of a patch that mozilla would be willing to accept.  I for one used this 'security hole' in firefox before it was closed to make a now-broken application.  It's ashame that mozilla doesn't prioritize this considering that it's still probably the easiest way to creating 'web apps' on the desktop.
?I don't want to 'spam' this bug

Then please stop.
Schuyler, those sort of solutions might work, yes. Overloading document.domain probably wouldn't, but specifying some sort of file that you can load anyway from your app (which means it can't be an arbitrary file on disk) might work.

As you say, it'd be a good bit of work.
(In reply to comment #30)

But, you are wrong. (and don't seem to understand what I'm saying)

> > The "domain" would be "C:/Users".
> 
> That is NOT acceptable.  That allows any web page you happen to save to disk
> to read the local storage of all file:/// pages.  We are NOT going to
> implement that.
> 

It would ONLY allow web pages to access local storage for all web pages stored under that directory. As for "security", you're not supposed to store actual "secure data" in there, anyway. (anyone who does that, even in the current system, "deserves what they get") If you want to have "better security", just use the whole directory spec as the "domain". (that might be more preferable? -- or the top N directories, IF present)

> > That would give the user as many "local domains" for the file:// protocol as
> > they want
> 
> No normal user would ever set up extra drives to save web pages to.  Ever. 
> It's no OK to expose those users to security/privacy bugs by default.

It's not a matter or setting up extra drives. It's a matter of HAVING multiple drives and wanting to differentiate between them. (as would be desired and needed)

To make it (more) "secure", you could just create a config value that is "Off" by default (or not even present, so it would have to be manually created, which is even more 'secure'), so only advanced users who know what they are doing have to go into about:config and turn it on. You might want to even require that they create a value specifying each directory they want to allow to have local storage. (that would be my preference)

> Now can we please stop adding noise to this bug rehashing what's already
> been said?

This isn't noise. The fact that you think it is "says something". (and I've never seen the issues / "solutions" I've brought up mentioned anywhere)
> for all web pages stored under that directory.

Under C:/Users, yes?  So every single web page users on the machine save.  Please do think through how your proposals would interact with how users actually use computers...
I'm fine with using the first directory for the domain, c:\Users on Windows and /Users on Linux/OS X.  Just using the drive would be a Windows only solution. Please don't make every URL as a separate domain as Webkit did for cookies; you can only pass cookies form one page to itself because every page is its own domain. The goal of localstorage is making it available to the application which is usually composed of more than one page.
Comment on attachment 533757 [details] [diff] [review]
patch to make file:/// localStorage domain scope exact path

Won't work for multi-page websites, but better than nothing....  Thank you for the patch!

Honza, what do you think?
Attachment #533757 - Flags: review?(honzab.moz)
if we tweaked it to be the exact directory (but no children or parent dirs) would that be ok, too?
The problem is that you actually want children in many cases....  But yes, other stuff in the same dir would be no worse than what file:// same-origin policy allows, iirc.
Attachment #533757 - Attachment is patch: true
Although I appreciate improvement on this issue, without having the local storage available to multiple pages, I don't see that it will be usable. I can't image more than 1% of web pages are one page sites, those using the same URL, and those are either simple sites with little information or dynamic pages where the information is replaced as needed. Limiting access to local storage to one URL without also making it available to children at least, is hardly an improvement because considerable storage is available in javascript as long as you don't reload. I still prefer using the first folder in the URL, but I even be happy with anything in file:///
(No time to review until Tuesday, if not urgent for closing aurora merge)
(In reply to comment #41)
{snip}

It would be a pain to not allow sub-directories, but I could live with that
for the stuff I want to use it for.

How about either using the first directory and not allowing it to be "user",
which would be somewhat "safer",
OR
requiring that the first directory be some known, chosen name,
like "firefox_file_local".

I would be fine with either of those. (although I suppose not "ideal")

(however, again, having to specify the top-level directory manually in
about:config would be even better and "safer" and usable for power-users --
for code, just a quick get of the value, compare, and allow or disallow)
Well, this makes it the directory instead of the single file.  

I'd like to do parent directories, but don't see how we could 'know' where to root files in child directories.  

I'd like to use the manifest file, but manifests and offline resources don't seem to work (yet) on file:/// (another bug?  maybe, but it does seem weird to use offline resources on file:/// ...)

Blacklisting c:\Users still would be too leaky, but maybe a white-listed 'magic name' like your 'firefox_file_local' would be ok?

configuring a top-level directory assumes that you have a single file:/// web app, and that doesn't make any sense (you should be able to have as many as you want).

So I think current directory is both sane, semi-sufficient, and anything else will probably be a lot more work.  Please consider this patch.
Attachment #533757 - Attachment is obsolete: true
Attachment #533757 - Flags: review?(honzab.moz)
Attachment #536635 - Flags: review?(honzab.moz)
I appreciate your consideration of power-users, but I have to ship demos to non-power-users as a zip file and have them upzip it somewhere, creating a directory sturcture ../output/index.html (as the starting point for other html files).  Then I have them double click on or launch index.html in a browser.  Sometimes that is challenging. Having to instruct them to set some environment variable or create such a directory structure would complicate installation and running. Our demos usually have over a hundred html pages which I'd want each to access the local storage.

In development, I'll have multiple projects that will be in separate directory structures, and I need to be able to run them with little configuration because I'm often switching from one project to another.

Even using the first directory, c:\project1\files.html, I'll have to do some ocnfiguration.

Schuyler, maybe I don't understand your suggestion about using the current directory.  Do you mean c:\projects\file1.html would be its own domain and have a local storage and c:\projects\file2.html would be another domain and have a different local storage?
The current proposed patch has the same local storage domain for:
 c:\projects\file1.html
 c:\projects\file2.html
but not for either:
 c:\file3.html
nor
 c:\projects\subdirectory\file2.html

I believe our use cases are similar.  When your users go to ../output/index.html and navigate from there to ../output/file1.html and back, localStorage will be the same.  If you have a different project that goes to ../output2/index.html then the localStorage is not shared.  For my use case, that's a feature -- one project will not conflict with another.  The downside is that all your html files in a particular project need to live in a flat directory.
Unfortunately, for me the are not in a flat directory structure, but almost; all html except index.html are in a sub-directory called html.
../output/index.html
../output/html/file1.html
../output/html/others.html

Therefore, since index.html only launches to the others, I'd have no access to localstorage when I needed it.

Since our file structure is build with a tool, changing would be difficult.

Thanks for the explanation.
If storage used a principal, not just a URI (which it may; I'm not sure), and if your users always go through index.html to get to the other pages, then the setup in comment 47 would actually work.
It does use a principal:
http://hg.mozilla.org/mozilla-central/file/54f7bd940f82/dom/src/storage/nsDOMStorage.cpp#l1383

and after doing a test, it works, but I don't entirely understand why.  It's because when you navigate in file:/// you stay in the same principal?  So at some level, this means the patch sort-of does multi-hierarchical domains as long as you always start in the same place....
When you navigate from a file:// URI to another file in the same directory or in a subdirectory, the new page gets the same file:// principal as the original page.  It's basically a hack to the general "each file is a separate origin" policy that allows situations like the one described in comment 47 work for DOM access across frame boundaries.
Hello, just bumped into this bug for a local HTML5 app I am writing.

It is amazing to see it has been filed more than 2 years ago ...
Is there any workaround yet? I have prepared a new demo file I'll try to upload it ASAP.
Comment on attachment 536635 [details] [diff] [review]
localstorage based on url->GetDirectory()

Review of attachment 536635 [details] [diff] [review]:
-----------------------------------------------------------------

Sorry this took soooo long time to review.

So, I played with this a bit and it really works if you go from the root page by clicking links to reach dirs down in the tree.  Storage is shared by pages in sub-dirs.

Only problem that could potentially rise is (how else...) on windows, where file/dir names are not case-sensitive.  So, when user renames folder from "Folder" to say "FOLDER", then for the system it is the same path, but for localstorage keying it is different (caps change).  Extracting the file from the url and doing Normalize() on it doesn't solve this issue.  I don't know about any other clean way to de-capitalize names on the windows platform.

So, just using the Directory portion of the URL seems to be OK here.

r+ with the comments addressed and added a test (mochitest, probably).  If you don't know how to write a test, let me know, I can help.

::: dom/src/storage/nsDOMStorageDBWrapper.cpp
@@ +355,5 @@
>    if (domainScope.IsEmpty()) {
>      // About pages have an empty host but a valid path.  Since they are handled
>      // internally by our own redirector, we can trust them and use path as key.
> +    // if file:/// protocol, let's make the exact directory the domain
> +    PRBool useUrlPath = PR_FALSE;

I would rather call this |isScheme|

@@ +365,5 @@
>        // thus need to force the casing.
>        ToLowerCase(domainScope);
>      }
> +    else if (NS_SUCCEEDED(aUri->SchemeIs("file", &useUrlPath)) && useUrlPath) {
> +      nsCOMPtr<nsIURL> url = do_QueryInterface(aUri);

NS_ENSURE_SUCCESS(rv, rv); here please.
Attachment #536635 - Flags: review?(honzab.moz) → review+
Attached patch modified patch after review (obsolete) — — Splinter Review 
Thanks for reviewing!  This is the modified patch based on the small changes.

I'm familiar with mochitest, but can't see how to make a test that runs as file:///
e.g. there are no file:/// urls available in build/pgo/server-locations.txt
and I can't find any other examples that test file-schemes.  Help?
Attachment #536635 - Attachment is obsolete: true
Attachment #550694 - Flags: review?(honzab.moz)
We don't have a sane way to run file:// tests.  That's bug 424484.  Add this one as a dependency?  :(
Comment on attachment 550694 [details] [diff] [review]
modified patch after review

Review of attachment 550694 [details] [diff] [review]:
-----------------------------------------------------------------

::: dom/src/storage/nsDOMStorageDBWrapper.cpp
@@ +366,5 @@
>        ToLowerCase(domainScope);
>      }
> +    else if (NS_SUCCEEDED(aUri->SchemeIs("file", &isScheme)) && isScheme) {
> +      nsCOMPtr<nsIURL> url = do_QueryInterface(aUri);
> +      NS_ENSURE_SUCCESS(rv, rv);

I didn't specify correctly, please:

nsCOMPtr<nsIURL> url = do_QueryInterface(aUri, &rv);  // <-- add &rv arg
NS_ENSURE_SUCCESS(rv, rv);
Attachment #550694 - Flags: review?(honzab.moz)
(In reply to comment #55)
> We don't have a sane way to run file:// tests.  That's bug 424484.  Add this
> one as a dependency?  :(

I was afraid it was not easy.  OK, let's go in w/o a test.  This is highly wanted feature and it's not a good idea to block it on a test.
Depends on: 424484
Ok, added &rv as do_QueryInterface second argument.  I don't completely understand what this does, but the patch still works :-)

Maybe after this goes in, I'll tackle the file:/// testing issue.
Attachment #550694 - Attachment is obsolete: true
Attachment #550808 - Flags: review?(honzab.moz)
Comment on attachment 550808 [details] [diff] [review]
queryinterface tweak

Review of attachment 550808 [details] [diff] [review]:
-----------------------------------------------------------------

To explain: &rv on do_QueryInterface stores the error result of the QueryInterface operation on the object.  If it failed, then without the NS_ENSURE_SUCCESS check we would end up with |url| being null and would crash on the next line.
Attachment #550808 - Flags: review?(honzab.moz) → review+
http://hg.mozilla.org/mozilla-central/rev/9547e620e571
Status: NEW → RESOLVED
Closed: 13 years ago
Resolution: --- → FIXED
Whiteboard: [inbound]
Target Milestone: --- → mozilla8
Forgive the mozilla newbyness, but does this mean it's fixed in Firefox8?
It should be, barring any unforeseen issues arising.
Attached file testcase — 
I'm still having trouble using sessionStorage from a file:/// uri.

If I open this test file in a current Nightly build, I see the following error logged to the error console:
Timestamp: 1/10/2012 11:29:27 AM
Error: uncaught exception: [Exception... "Operation is not supported"  code: "9" nsresult: "0x80530009 (NS_ERROR_DOM_NOT_SUPPORTED_ERR)"  location: "file:///C:/Users/admin/Desktop/mytest.html Line: 11"]


If I change from using sessionStorage to using localStorage in the test file, it works just fine.

Did this bug only fix this for localStorage, or was it fixed for sessionStorage and this is a new regression?
From reading the code, it doesn't intentional to me. Either way, could you file comment 66 as a new bug, Wes?
(In reply to Josh Matthews [:jdm] from comment #67)
> From reading the code, it doesn't intentional to me. Either way, could you
> file comment 66 as a new bug, Wes?

Filed bug 716963. Thanks.
Does not work with jar:file:///

I was using a local file stored inside a jar, hence I used jar:file:/// instead of file:/// which resulted in the unexpected behavior, that localStore only worked as long as the session was active. I.e. it behaved like sessionStorage. It did not throw any error. webappsstore.sqlite was not created.

As soon as I unzip the jar and use file:/// to access the file, it works and webappsstore.sqlite is created.

This is on XULRunner 18.0.2 and 19.0.
(In reply to Mike Kronenberg from comment #69)
> Does not work with jar:file:///
> 
Can file a new bug?
Component: DOM: Mozilla Extensions → DOM
Component: DOM → DOM: Core & HTML

localStorage doesn't work in file:// because the port is not connect.

You have to got the code editor (VS code) and have a live server running.

For VS code download extension Live Server by Ritwick Dey it works great and run the serve live after downloading it will be on the bottom right and click Go Live.

Or you can run localhost.

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: