Last Comment Bug 774250 - IndexedDB: encoding error writing quota in sqlite file
: IndexedDB: encoding error writing quota in sqlite file
Status: VERIFIED FIXED
:
Product: Core
Classification: Components
Component: DOM: IndexedDB (show other bugs)
: Trunk
: x86 Windows 7
: -- major (vote)
: mozilla18
Assigned To: Kyle Huey [:khuey] (Exited; not receiving bugmail, email if necessary)
:
Mentors:
: 766489 (view as bug list)
Depends on:
Blocks: 766489
  Show dependency treegraph
 
Reported: 2012-07-16 06:05 PDT by Eric Abouaf
Modified: 2012-11-22 07:03 PST (History)
12 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---
-
wontfix
fixed
verified
verified
wontfix


Attachments
As landed (2.46 KB, patch)
2012-09-20 08:34 PDT, Kyle Huey [:khuey] (Exited; not receiving bugmail, email if necessary)
khuey: review+
akeybl: approval‑mozilla‑aurora+
akeybl: approval‑mozilla‑beta+
Details | Diff | Splinter Review

Description Eric Abouaf 2012-07-16 06:05:01 PDT
User Agent: Mozilla/5.0 (Windows NT 5.1; rv:13.0) Gecko/20100101 Firefox/13.0.1
Build ID: 20120614114901

Steps to reproduce:


 * create a Windows user containing accents in its name ( ex: "Propriétaire" (aka "Owner") is the default user name in French, so its really frequent ), and launch Firefox with this user.

 * Re-open an existing indexedDB database



Actual results:

Trying to reopen the database will raise an exception with errorCode 1 (unknown error)

Same issue than in https://bugzilla.mozilla.org/show_bug.cgi?id=766489



Expected results:

The database should open correctly.


Details :

The error comes from the "UpdateQuotaInformationForFile" function in http://hg.mozilla.org/releases/mozilla-beta/file/c92edb5a0ac2/storage/src/mozStorageService.cpp#l923

    int rc = ::sqlite3_quota_file(PromiseFlatCString(path).get());

Here, the result code returned by sqlite is an error (unable to open the file), so I first thought that the issue was from sqlite.


I built a simple test case to test the sqlite code, and I could reproduce the bug :

   int rc = ::sqlite3_quota_file("C:\Documents and Settings\Proprietaire\...path to the indexedb sqlite file..."); // SUCCESS

   int rc = ::sqlite3_quota_file("C:\Documents and Settings\Propriétaire\...path to the indexedb sqlite file..."); // FAILS IF NOT IN UTF 8, SUCCESS IF IN UTF-8


So, sqlite expects utf8 filenames in the sqlite3_quota_file and sqlite3_quota_set calls (lines 905 and 923)

So I don't know which encoding is PromiseFlatCString(path).get() in, but we should convert it to UTF-8 before making the sqlite code.

This makes no difference unless you have accents in the path of the firefox Profile, but fails for most French users :(
Comment 1 Ben Turner (not reading bugmail, use the needinfo flag!) 2012-07-16 07:12:34 PDT
Jan or Kyle, can you poke around a little here before I get back from the east coast? Would be good to know what's going wrong here.
Comment 2 Jan Varga [:janv] 2012-07-16 08:32:56 PDT
I think we need to change GetNativePath() to GetPath() and
 then convert the path to UTF8
Comment 3 Maxime RETY 2012-09-10 08:51:14 PDT
Any update ?

It's a very common bug on computers of non-English-speaking people.

IndexedDB should not ship unprefixed before it has been fixed.
Comment 4 arnaud 2012-09-17 11:32:18 PDT
quite annoying bug for us, a lot of our users have to create another windows username without any accent. Is there another workaround ? Can we do something to help ?
Comment 5 Andrew Overholt [:overholt] (back Aug 31) 2012-09-19 10:21:25 PDT
Jonas doesn't think this is critical (or affecting?) B2G so blocking- for Basecamp.
Comment 6 Kyle Huey [:khuey] (Exited; not receiving bugmail, email if necessary) 2012-09-19 14:03:58 PDT
https://hg.mozilla.org/integration/mozilla-inbound/rev/c1cd7da16cd6
Comment 7 Kyle Huey [:khuey] (Exited; not receiving bugmail, email if necessary) 2012-09-19 15:26:01 PDT
With that fix the entire IndexedDB test suite that we have passes when run from a profile directory on a user named 'Propriétaire'.
Comment 8 Kyle Huey [:khuey] (Exited; not receiving bugmail, email if necessary) 2012-09-19 16:28:28 PDT
Though, it turns out our test suite can never hit the scenario in this bug, because it requires restarting the browser, so that's not a very useful test.
Comment 10 :Ehsan Akhgari 2012-09-19 18:52:30 PDT
(In reply to comment #9)
> Backed out because of Linux opt Moth orange:
> https://hg.mozilla.org/integration/mozilla-inbound/rev/d6716083397b
> 
> Example log:
> https://tbpl.mozilla.org/php/getParsedLog.php?id=15355150&tree=Mozilla-Inbound

And, it seems like the orange has been intermittent?!  Kyle, please reland if you think that's appropriate, I'm not sure what needs to be done here :/
Comment 11 Phil Ringnalda (:philor) 2012-09-19 21:34:16 PDT
It doesn't look to me like it was the least bit intermittent, it looks like it failed on every single Linux32 run which started between 14:30 and 15:30 (or, ran that test between some times after those).

Relanded in https://hg.mozilla.org/integration/mozilla-inbound/rev/9404e57a5ce6.
Comment 13 Kyle Huey [:khuey] (Exited; not receiving bugmail, email if necessary) 2012-09-20 08:34:18 PDT
Created attachment 663026 [details] [diff] [review]
As landed
Comment 14 Kyle Huey [:khuey] (Exited; not receiving bugmail, email if necessary) 2012-09-20 08:36:51 PDT
Comment on attachment 663026 [details] [diff] [review]
As landed

[Approval Request Comment]
Bug caused by (feature/regressing bug #): Was in the original indexedDB implementation (in Firefox 4)
User impact if declined: IndexedDB will not work properly on Windows machines where the username has non-ASCII characters.  At the very least, the default username in the French locale hits this bug.
Testing completed (on m-c, etc.): Landed on m-c.
Risk to taking this patch (and alternatives if risky): Very low risk patch.  We just get the unicode path in two places where we used to get the path in the system codepage.
String or UUID changes made by this patch: None.
Comment 15 Ryan VanderMeulen [:RyanVM] 2012-09-20 18:37:03 PDT
https://hg.mozilla.org/mozilla-central/rev/9404e57a5ce6
Comment 16 Alex Keybl [:akeybl] 2012-09-21 15:49:43 PDT
Comment on attachment 663026 [details] [diff] [review]
As landed

[Triage Comment]
Approving for Aurora 17 due to the risk evaluation, and Beta 16 due to the unprefixing in bug 726378.
Comment 17 Kyle Huey [:khuey] (Exited; not receiving bugmail, email if necessary) 2012-09-22 06:42:27 PDT
https://hg.mozilla.org/releases/mozilla-aurora/rev/940b7d1491e5
https://hg.mozilla.org/releases/mozilla-beta/rev/135a522896cb
Comment 18 Ioana (away) 2012-11-12 00:01:23 PST
*** Bug 766489 has been marked as a duplicate of this bug. ***
Comment 19 Paul Silaghi, QA [:pauly] 2012-11-12 08:00:28 PST
While trying to reproduce this:
1. Go to http://www.html5rocks.com/en/tutorials/indexeddb/todo/
2. Load the html example in FF
3. Press the button

Actual results:
nothing happens

Not the same result in Chrome.
Comment 20 Paul Silaghi, QA [:pauly] 2012-11-14 07:57:10 PST
I've tested this on Nightly 2012-09-04 before the fix:
1. Create user "Propriétaire"
2. Open http://jensarps.github.com/IDBWrapper/example/index/ in FF
3. Add some random customers
4. Set FF to remember tabs and windows from last time
5. Restart FF
6. Click again to add a customer
Actual results:
The DB won't open. "Unknown error" in error console

Tested again on FF 17b6. The DB is not automatically opened after restart, but the list shows up after clicking to add a new customer. I'm considering this verified fixed.
Comment 21 Paul Silaghi, QA [:pauly] 2012-11-14 08:07:32 PST
bug 811736 filed for comment 19
Comment 22 Paul Silaghi, QA [:pauly] 2012-11-22 07:03:14 PST
Verified fixed on Mozilla/5.0 (Windows NT 6.1; WOW64; rv:18.0) Gecko/18.0 Firefox/18.0b1

Note You need to log in before you can comment on or make changes to this bug.