508740 - Use mozStorage for SessionStore

Status: RESOLVED WONTFIX

(Firefox :: Session Restore, enhancement)

Description

[Paul O'Shannessy [:zpao] (not bugmail, email directly)]

Bug 485976 moves file writing off the main thread, but we're still writing the whole file every time. This could be to update the scroll position.

Bug 506482 notes how it's pretty excessive to write the whole file. I've heard this outside of bug reports too.

After a chat in the office yesterday, I've concluded that it'd (probably) be more efficient if instead of writing the whole file, we write out only the parts that change.

I'd thought about this before, but was always put off by the complexity of the data we store. It wouldn't be efficient to break down all of the objects, but if we break down to a tab level & just store the tab data as a string, this should be doable.

I haven't thought about all the details but having 2 tables (tabs & windows) with a simple relationship and a couple fields for id, order, closed, maybe a couple others.

I imagine this could hurt Ts a bit since we'll have to build up a JSON object in nsSessionStartup. But the tradeoff might be worth it (especially on mobile devices where disk writes are expensive)

Comment 1

[Paul O'Shannessy [:zpao] (not bugmail, email directly)]

Wrote up a slightly better braindump on wikimo. It's not too technical (yet) but it gives a better overview of the project. The URL may change as this turns into a real project, so use the link on the bug.

Comment 2

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Hmm, I wonder if you can get the same benefits by just appending to the session store file the bits that change, and only regenerate the entire thing once it grows beyond some threshold.  Maybe even tweaking the file format?  I worry that with sqlite we'll end up doing a bunch of small writes which could be more expensive than just one larger write.  Given that the IO is off the main thread now, the gains here might be marginal as well, for a not-insignificant amount of work.

Comment 3

[frankas09]

I have just become aware myself of the hiccough or hesitation in Firefox while it writes the full file, 730kB in my case, and once aware of it, it becomes very irritating; I've noted quite a number of user having very negative experiences with this cyclic lagging in behaviour, especially when they compare it to their experiences with Chrome and IE. Playing with the value of sessionstore.interval is not the real answer, so my vote is definitely for far more intelligent handling of current status logging.

Comment 4

[Paul O'Shannessy [:zpao] (not bugmail, email directly)]

(In reply to comment #3)
> I have just become aware myself of the hiccough or hesitation in Firefox while
> it writes the full file, 730kB in my case, and once aware of it, it becomes
> very irritating; I've noted quite a number of user having very negative
> experiences with this cyclic lagging in behaviour

Your problem should actually be greatly alleviated by bug 485976 which moved the writing of the file off the main thread. That will be in 3.6.

Comment 5

[IU]

I'm assuming the primary purpose of the SessionStore store capability is as a defense against loss due to browser crash and not due to an OS crash.  If that is the case, how about this as a possible architecture to achieve both the aims of reducing disk writes and reducing SQLite writes:

Create a very small, very robust, stand-alone SessionStore Proxy process, with interaction as follows:

_STARTUP_
1. Firefox launches the SessionStore Proxy when it starts
2. SessionStore Proxy independently loads the session information into memory, from SQLite database, and waits to service requests
3. When Firefox is up and ready, it queries SessionStore Proxy and loads whatever sessions need to be loaded.

_OPERATION (heartbeat)_
4(a). SessionStore proxy sends "hello" messages to Firefox/Gecko frequently and Firefox replies.  If no response is received by x seconds, SessionStore Proxy immediately writes it's held session information to the SQLite database.  If the Firefox process is still running, SessionStore Proxy will continue to try and contact Firefox, since Firefox might be hung. If no Firefox process is running, SessionStore Proxy writes any unwritten changes and exits.
4(b). Firefox also expects "hello" messages from SessionStore Proxy and will take action if it receives no such messages by x seconds (e.g. if SessionStore Proxy is running but not responding, send "flush and restart" message, wait, then restart SessionStore Proxy, if it is still running)

_OPERATION (read/write)_
5. Firefox/Gecko writes (as often as necessary) to the SessionStore Proxy 
6. SessionStore Proxy receives, sanitizes, and holds the information in memory (performing only minor operations such as adds/updates/deletes).
7. After a certain time (much less than the writes in #5, SessionStore Proxy writes the session data to SQLite, but holds a copy in memory.
8. If Firefox needs to perform a session restore operation, it reads from SessionStore Proxy.

_SHUTDOWN_
9. Firefox sends shutdown message to SessionStore Proxy.  SessionStore Proxy writes any unwritten data to SQLite and exits.
10. SessionStore Proxy can also shut itself down if it stops receiving responses from Firefox and determines that no Firefox process is still running (see #4(a))


It may look complicated, but it shouldn't be.  I might have missed something, but I hope I've fully covered the gist of the idea.

Comment 6

[James May [:fowl2]]

Personally, I find the power failure case very useful.

Comment 7

[Paul O'Shannessy [:zpao] (not bugmail, email directly)]

(In reply to comment #5)
> I'm assuming the primary purpose of the SessionStore store capability is as a
> defense against loss due to browser crash and not due to an OS crash.

It's for either. It's really just for any time where Firefox doesn't quit cleanly.

As for the rest of your comment. While it's an interesting idea, it's not something we'll be doing now (or likely anytime soon/ever?) and definitely not in this bug. By writing to disk periodically from the Firefox process, there's really no need to have another process running to talk to Firefox. If you feel adamantly about this though, feel free to file a new bug proposing that.

> It may look complicated, but it shouldn't be.

It's certainly more complicated than it needs to be. There's a lot of failure redundancy that you're proposing be added without much benefit, when really all we need is for the data to be on disk.

(In reply to comment #6)
> Personally, I find the power failure case very useful.

That won't be going anywhere. This bug is merely a change in how we write to disk, but should not alter any functionality of session restore as it exists now.

Comment 8

[IU]

(In reply to comment #7)
> It's certainly more complicated than it needs to be. There's a lot of failure
> redundancy that you're proposing be added without much benefit, when really all
> we need is for the data to be on disk.

That settles it then.  Crawling back in my hole. :-)

Comment 9

[(dormant account)]

mozstorage is the wrong way to make session restore incremental. We should write to multiple files instead.