Last Comment Bug 1112702 - Switch IndexedDB transactions to be non-durable by default
: Switch IndexedDB transactions to be non-durable by default
Status: RESOLVED FIXED
: dev-doc-complete
Product: Core
Classification: Components
Component: DOM: IndexedDB (show other bugs)
: Trunk
: x86 Windows 8.1
-- normal (vote)
: mozilla40
Assigned To: Ben Turner (not reading bugmail, use the needinfo flag!)
:
: Hsin-Yi Tsai [:hsinyi]
Mentors:
Depends on: 866846 1131766
Blocks:
  Show dependency treegraph
 
Reported: 2014-12-17 11:25 PST by Ben Turner (not reading bugmail, use the needinfo flag!)
Modified: 2016-11-11 01:48 PST (History)
13 users (show)
ryanvm: in‑testsuite+
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---
fixed
40+


Attachments
Patch, v1 (27.62 KB, patch)
2015-02-10 13:18 PST, Ben Turner (not reading bugmail, use the needinfo flag!)
khuey: review+
Details | Diff | Splinter Review

Description User image Ben Turner (not reading bugmail, use the needinfo flag!) 2014-12-17 11:25:09 PST
The proposal is to change our IndexedDB write transactions to more closely align with Chrome/IE which favor performance over durability. We will have an opt-in mechanism to have per-transaction durability for consumers that care.
Comment 1 User image Jan Varga [:janv] 2014-12-17 11:28:00 PST
+1
Comment 2 User image Andrew Sutherland [:asuth] 2014-12-17 14:43:03 PST
Can you clarify the risk profile here?  Specifically, a crash shortly-after a non-durable transaction commits may result in:
A) Database corruption.  The whole database may need to be blown away because the pages of the database may not be in a coherent state. (non-WAL PRAGMA SYNCHRONOUS=0/OFF, low probability on non-WAL PRAGMA SYNCHRONOUS=1/NORMAL)
B) Loss of the most recent transaction but the database should probably be coherent. (non-WAL PRAGMA SYNCHRONOUS=1/NORMAL)
C) Loss of multiple recent transactions but the database will still be coherent at that earlier point.  (WAL with SYNCHRONOUS=1/NORMAL and where a checkpoint is not triggered)

Since IndexedDB currently uses PRAGMA synchronous=2/FULL, I'm assuming this is a proposal to drop to NORMAL or OFF?  Or is the idea to switch to using WAL with PRAGMA synchronous=1/NORMAL and the opt-in mechanism would just make sure to trigger a checkpoint as part of the transaction?

The WAL option sorta sounds preferable since the non-WAL options seem more likely to result in database corruption, requiring apps to make every transaction be durable.  Or to shard things so that there is one IndexedDB database that is where the really important stuff is (config, drafts for email), and another happy-go-lucky DB for stuff that isn't authoritative (synced email).
Comment 3 User image Ben Turner (not reading bugmail, use the needinfo flag!) 2014-12-17 14:50:45 PST
IDB currently runs with non-WAL and SYNCHRONOUS=NORMAL (unless you set an obscure pref that might flip it to FULL).

The change will be to make normal write transactions run with WAL and SYNCHRONOUS=NORMAL. If WAL mode isn't available for some reason then we'll still use SYNCHRONOUS=NORMAL.

The specially-opted-into durable write transactions will essentially run with WAL and SYNCHRONOUS=FULL (it won't actually because we're changing the way we use connections, but that'll be the end result).
Comment 4 User image Andrew Sutherland [:asuth] 2014-12-17 14:52:13 PST
Awesome.  Thanks for the clarification!
Comment 5 User image Kyle Huey [:khuey] (Exited; not receiving bugmail, email if necessary) 2014-12-17 14:52:23 PST
We're absolutely not going to introduce opportunities for database corruption.
Comment 6 User image Ben Turner (not reading bugmail, use the needinfo flag!) 2014-12-17 14:53:21 PST
(In reply to ben turner [:bent] (use the needinfo? flag!) from comment #3)
> (it won't actually because we're changing the way we use connections,
> but that'll be the end result).

Through manual checkpointing.
Comment 7 User image Ben Turner (not reading bugmail, use the needinfo flag!) 2015-02-10 13:18:54 PST
Created attachment 8562320 [details] [diff] [review]
Patch, v1

This gives us an optional mode to say "no, really, I want it on disk".
Comment 8 User image Kyle Huey [:khuey] (Exited; not receiving bugmail, email if necessary) 2015-02-23 13:06:23 PST
Comment on attachment 8562320 [details] [diff] [review]
Patch, v1

Review of attachment 8562320 [details] [diff] [review]:
-----------------------------------------------------------------

We should test that if experimental features are not enabled then readwriteflush throws.

r=me w/that
Comment 9 User image Ben Turner (not reading bugmail, use the needinfo flag!) 2015-03-28 18:30:32 PDT
https://hg.mozilla.org/integration/mozilla-inbound/rev/0b4de21b759f
Comment 10 User image Phil Ringnalda (:philor) 2015-03-29 13:18:28 PDT
Backed out in https://hg.mozilla.org/integration/mozilla-inbound/rev/beb016173f6f - see bug 866846 comment 21
Comment 11 User image Ben Turner (not reading bugmail, use the needinfo flag!) 2015-03-30 15:09:38 PDT
https://hg.mozilla.org/integration/mozilla-inbound/rev/3be650e84eea
Comment 13 User image Ben Turner (not reading bugmail, use the needinfo flag!) 2015-03-31 16:05:08 PDT
https://hg.mozilla.org/integration/mozilla-inbound/rev/b0c5f21c7811
Comment 14 User image Ryan VanderMeulen [:RyanVM] 2015-04-01 09:47:48 PDT
https://hg.mozilla.org/mozilla-central/rev/b0c5f21c7811
Comment 15 User image Lawrence Mandel [:lmandel] (use needinfo) 2015-04-07 09:14:57 PDT
Release Note Request (optional, but appreciated)
[Why is this notable]: This will have an impact on devs.
[Suggested wording]: IndexedDB transactions now non-durable by default
[Links (documentation, blog post, etc)]:
Comment 16 User image Kohei Yoshino [:kohei] 2015-05-15 08:42:43 PDT
Is there any examples that will be affected by this change? I see the new readwriteflush mode in the patch but readwrite is still the default mode, right?
Comment 17 User image Ben Turner (not reading bugmail, use the needinfo flag!) 2015-05-15 10:55:01 PDT
(In reply to Kohei Yoshino [:kohei] from comment #16)
> Is there any examples that will be affected by this change? I see the new
> readwriteflush mode in the patch but readwrite is still the default mode,
> right?

Well, the default mode for transactions is actually 'readonly'. But the new 'readwriteflush' mode is not available unless you have enabled the indexedDB experimental feature preference. Without that preference set you are only able to use 'readonly' and 'readwrite'.
Comment 18 User image Kohei Yoshino [:kohei] 2015-05-15 11:03:06 PDT
Oh I meant 'readonly', of course I know that because my app uses IndexedDB.

> optional IDBTransactionMode mode = "readonly"

So I guess there would be no backward compatibility issues here.
Comment 19 User image Ben Turner (not reading bugmail, use the needinfo flag!) 2015-05-15 11:06:53 PDT
(In reply to Kohei Yoshino [:kohei] from comment #18)
> So I guess there would be no backward compatibility issues here.

Right.
Comment 20 User image Kohei Yoshino [:kohei] 2015-05-15 11:10:23 PDT
Thanks! Removing the relnote-firefox flag given the dom.indexedDB.experimental pref is defaulted to false at this moment.
Comment 21 User image Ben Turner (not reading bugmail, use the needinfo flag!) 2015-05-15 11:15:01 PDT
(In reply to Kohei Yoshino [:kohei] from comment #20)
> Thanks! Removing the relnote-firefox flag given the
> dom.indexedDB.experimental pref is defaulted to false at this moment.

Well, this change is still worthy of a relnote in my opinion (and see comment 15). The change won't affect the code that people write but it will affect the timing of the 'complete' event and the behavior in the event of a system crash.
Comment 22 User image Kohei Yoshino [:kohei] 2015-05-15 11:20:08 PDT
Ah, then put the relnote flag back... I don't add the site-compat flag though.
Comment 23 User image Chris Mills (Mozilla, MDN editor) [:cmills] 2015-06-12 13:08:15 PDT
I've written up a definition of durability and what the current state in browsers is:

https://developer.mozilla.org/en-US/Firefox/Releases/40#IndexedDB

I've also mentioned this on relevant pages:

https://developer.mozilla.org/en-US/docs/Web/API/IDBDatabase/transaction
https://developer.mozilla.org/en-US/docs/Web/API/IDBTransaction/oncomplete

And added a link to the definition in the release notes:

https://developer.mozilla.org/en-US/Firefox/Releases/40#IndexedDB

bent helped me write this, but another tech review would be cool, just as a sanity check.
Comment 24 User image Ben Turner (not reading bugmail, use the needinfo flag!) 2015-06-12 14:39:44 PDT
Thanks Chris! One thing to note though is that the "readwriteflush" mode is only available if the "dom.indexedDB.experimental" pref is set to true.
Comment 25 User image Chris Mills (Mozilla, MDN editor) [:cmills] 2015-06-12 14:54:55 PDT
Added this info; thx Ben.

And the correct link for the 'durable' definition is:

https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API/Basic_Concepts_Behind_IndexedDB#durable

Note You need to log in before you can comment on or make changes to this bug.