774250 - IndexedDB: encoding error writing quota in sqlite file

Status: VERIFIED FIXED

(Core :: Storage: IndexedDB, defect)

Description

[Eric Abouaf]

User Agent: Mozilla/5.0 (Windows NT 5.1; rv:13.0) Gecko/20100101 Firefox/13.0.1
Build ID: 20120614114901

Steps to reproduce:


 * create a Windows user containing accents in its name ( ex: "Propriétaire" (aka "Owner") is the default user name in French, so its really frequent ), and launch Firefox with this user.

 * Re-open an existing indexedDB database



Actual results:

Trying to reopen the database will raise an exception with errorCode 1 (unknown error)

Same issue than in https://bugzilla.mozilla.org/show_bug.cgi?id=766489



Expected results:

The database should open correctly.


Details :

The error comes from the "UpdateQuotaInformationForFile" function in http://hg.mozilla.org/releases/mozilla-beta/file/c92edb5a0ac2/storage/src/mozStorageService.cpp#l923

    int rc = ::sqlite3_quota_file(PromiseFlatCString(path).get());

Here, the result code returned by sqlite is an error (unable to open the file), so I first thought that the issue was from sqlite.


I built a simple test case to test the sqlite code, and I could reproduce the bug :

   int rc = ::sqlite3_quota_file("C:\Documents and Settings\Proprietaire\...path to the indexedb sqlite file..."); // SUCCESS

   int rc = ::sqlite3_quota_file("C:\Documents and Settings\Propriétaire\...path to the indexedb sqlite file..."); // FAILS IF NOT IN UTF 8, SUCCESS IF IN UTF-8


So, sqlite expects utf8 filenames in the sqlite3_quota_file and sqlite3_quota_set calls (lines 905 and 923)

So I don't know which encoding is PromiseFlatCString(path).get() in, but we should convert it to UTF-8 before making the sqlite code.

This makes no difference unless you have accents in the path of the firefox Profile, but fails for most French users :(

Comment 1

[Ben Turner (not reading bugmail, use the needinfo flag!)]

Jan or Kyle, can you poke around a little here before I get back from the east coast? Would be good to know what's going wrong here.

Comment 2

[Jan Varga [:janv]]

I think we need to change GetNativePath() to GetPath() and
 then convert the path to UTF8

Comment 3

[Maxime RETY]

Any update ?

It's a very common bug on computers of non-English-speaking people.

IndexedDB should not ship unprefixed before it has been fixed.

Comment 4

[arnaud]

quite annoying bug for us, a lot of our users have to create another windows username without any accent. Is there another workaround ? Can we do something to help ?

Comment 5

[Andrew Overholt [:overholt]]

Jonas doesn't think this is critical (or affecting?) B2G so blocking- for Basecamp.

Comment 6

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/c1cd7da16cd6

Comment 7

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

With that fix the entire IndexedDB test suite that we have passes when run from a profile directory on a user named 'Propriétaire'.

Comment 8

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

Though, it turns out our test suite can never hit the scenario in this bug, because it requires restarting the browser, so that's not a very useful test.

Comment 9

[(no longer active)]

Backed out because of Linux opt Moth orange: https://hg.mozilla.org/integration/mozilla-inbound/rev/d6716083397b

Example log: https://tbpl.mozilla.org/php/getParsedLog.php?id=15355150&tree=Mozilla-Inbound

Comment 10

[(no longer active)]

(In reply to comment #9)
> Backed out because of Linux opt Moth orange:
> https://hg.mozilla.org/integration/mozilla-inbound/rev/d6716083397b
> 
> Example log:
> https://tbpl.mozilla.org/php/getParsedLog.php?id=15355150&tree=Mozilla-Inbound

And, it seems like the orange has been intermittent?!  Kyle, please reland if you think that's appropriate, I'm not sure what needs to be done here :/

Comment 11

[Phil Ringnalda (:philor)]

It doesn't look to me like it was the least bit intermittent, it looks like it failed on every single Linux32 run which started between 14:30 and 15:30 (or, ran that test between some times after those).

Relanded in https://hg.mozilla.org/integration/mozilla-inbound/rev/9404e57a5ce6.

Comment 12

[:Ms2ger (he/him; ⌚ UTC+1/+2)]

https://hg.mozilla.org/mozilla-central/rev/c1cd7da16cd6
https://hg.mozilla.org/mozilla-central/rev/d6716083397b

Comment 13

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

Attachment: As landed

Comment 14

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

Comment on attachment 663026 [details] [diff] [review]
As landed

[Approval Request Comment]
Bug caused by (feature/regressing bug #): Was in the original indexedDB implementation (in Firefox 4)
User impact if declined: IndexedDB will not work properly on Windows machines where the username has non-ASCII characters.  At the very least, the default username in the French locale hits this bug.
Testing completed (on m-c, etc.): Landed on m-c.
Risk to taking this patch (and alternatives if risky): Very low risk patch.  We just get the unicode path in two places where we used to get the path in the system codepage.
String or UUID changes made by this patch: None.

Comment 15

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/9404e57a5ce6

Comment 16

[Alex Keybl [:akeybl]]

Comment on attachment 663026 [details] [diff] [review]
As landed

[Triage Comment]
Approving for Aurora 17 due to the risk evaluation, and Beta 16 due to the unprefixing in bug 726378.

Comment 17

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

https://hg.mozilla.org/releases/mozilla-aurora/rev/940b7d1491e5
https://hg.mozilla.org/releases/mozilla-beta/rev/135a522896cb

Comment 19

[Paul Silaghi, QA [:pauly]]

While trying to reproduce this:
1. Go to http://www.html5rocks.com/en/tutorials/indexeddb/todo/
2. Load the html example in FF
3. Press the button

Actual results:
nothing happens

Not the same result in Chrome.

Comment 20

[Paul Silaghi, QA [:pauly]]

I've tested this on Nightly 2012-09-04 before the fix:
1. Create user "Propriétaire"
2. Open http://jensarps.github.com/IDBWrapper/example/index/ in FF
3. Add some random customers
4. Set FF to remember tabs and windows from last time
5. Restart FF
6. Click again to add a customer
Actual results:
The DB won't open. "Unknown error" in error console

Tested again on FF 17b6. The DB is not automatically opened after restart, but the list shows up after clicking to add a new customer. I'm considering this verified fixed.

Comment 21

[Paul Silaghi, QA [:pauly]]

bug 811736 filed for comment 19

Comment 22

[Paul Silaghi, QA [:pauly]]

Verified fixed on Mozilla/5.0 (Windows NT 6.1; WOW64; rv:18.0) Gecko/18.0 Firefox/18.0b1