Open Bug 810932 Opened 12 years ago Updated 5 months ago

Investigate how to redesign sessionstore.js for improved performance

Categories

(Firefox :: Session Restore, task)

Product:
Firefox
Component:
Session Restore
Type:
task

Tracking

()

People

(Reporter: Yoric, Unassigned)

References

(Blocks 2 open bugs,
URL
)

Details

(Keywords: main-thread-io, perf)

Attachments

(1 file)

2014-06-20 18_35_32-about_sessionstore.png
19.60 KB, image/png
Details 
Session Store causes a number of performance issues. This is a large (several megabytes), monolithic file, which needs to be fully rewritten every few seconds, which needs to be fully loaded on startup, which needs to be fully rebuilt on shutdown, etc.

We can certainly redesign it for better performance.
Summary: Investigate how to redesign Session Store for improved performance → Investigate how to redesign sessionstore.js for improved performance
Please have an look on my bug:
Bug 937651 - Replace the sessionstore.js with an sessionstore.sqlite
(In reply to David Rajchenbach Teller [:Yoric] (away 20 Dec to Jan 4th - please use "needinfo?") from comment #2)

> I have put together a wiki with a summary of possible redesigns:
> https://wiki.mozilla.org/Firefox/session_restore#Redesigning_sessionstore.js

Thank you, Dave. 
I would add the criterion of human readability. In "Compress", for example. It would be a catastrophe to have an obscure binary file. The current JS file is already one of the worst among browsers for readability. And we need the file to be readable. For investigating problems, of course. But also for normal use. I sometimes search in the session file to find an address.
Recent numbers seem to indicate that while the current sessionstore.js format could be improved, this would let us at best eat into the ~15ms (for the 95th percentile) for serializing and writing data. Not a priority at the moment.
Flags: firefox-backlog?
David, is this still an investigation bug? Aren't we quite sure that we will want a journaled storage sooner or later? Should we close and file a new one for the journaled storage? Morph this bug?
Flags: needinfo?(dteller)
I'd morph this bug in a meta bug.
Flags: needinfo?(dteller)
Flags: firefox-backlog? → firefox-backlog-
Depends on: 937651
Screenshot from "about:sessionstore" with high "Disk writes to sessionstore.js over an 8 hour day".
Is it possible to measure "Disk writes to sessionstore.js over an 8 hour day" of the users with e.g. telemetry ???

And can somebody add some infos to https://wiki.mozilla.org/Firefox/session_restore ???
Especially to https://wiki.mozilla.org/Firefox/session_restore#Data_we_collect ...
And maybe examples to the big data for Google Searches and Facebook Messages ...
https://bugzilla.mozilla.org/show_bug.cgi?id=934935
Severity: normal → major
Status: NEW → ASSIGNED
browser.sessionhistory.max_entries: 50 (default)

Are these the steps that the browser can go back ???

Wouldn't 20 be enough ???
It's actually browser.sessionstore.max_serialize_{back, forward}.
(In reply to David Rajchenbach Teller [:Yoric] from comment #11)
> It's actually browser.sessionstore.max_serialize_{back, forward}.

In 31b, too?
Do somebody know the programmer of the "about:sessionstore"-extension ???
Can we ask him to update the extension?
Can he join this bug?

What about adding this "about:sessionstore" into FF ???

Seems FF write a lot on HDs, when a lot of tabs are open ... But FF also do this, if the tabs are not loaded (after a session-restore, exit FF, restart) and the tabs have not changed!
This was one of my reasons to suggest a SQLite-Solution ... because you can write in a DB just the tabs that change. I don't know if this is possible with an journaled storage, too ... ???

Also the Dev-Team should think about the upper 5% of the statistics ... Because I know from a lot of (power) user, that they have switched to Google Chrome because of the bad performance of FF.
This was also the reason why I used a long time Chrome ...
FF shouldn't get a "Newbie Browser" just because of the statistics.
(In reply to Tobias B. Besemer from comment #12)
> (In reply to David Rajchenbach Teller [:Yoric] from comment #11)
> > It's actually browser.sessionstore.max_serialize_{back, forward}.
> 
> In 31b, too?

Yep, ever since Firefox 30 (added in bug 943339). Specifically:

browser.sessionhistory.max_entries:
  The maximum number of session history entries to keep in working memory (which
  includes the 'current' page).

browser.sessionstore.max_serialize_back:
  The maximum number of 'back button' session history entries to store in
  sessionstore.js (-1 = no limit).


browser.sessionstore.max_serialize_forward:
  The maximum number of 'forward button' session history entries to store in
  sessionstore.js (-1 = no limit).

If you set both of them to 0, only the currently active entry would be saved.
(In reply to Tobias B. Besemer from comment #12)
> (In reply to David Rajchenbach Teller [:Yoric] from comment #11)
> > It's actually browser.sessionstore.max_serialize_{back, forward}.
> 
> In 31b, too?
> Do somebody know the programmer of the "about:sessionstore"-extension ???
> Can we ask him to update the extension?
> Can he join this bug?

He's easy to contact - see https://addons.mozilla.org/en-US/firefox/addon/about-sessionstore/ (you can find his e-mail easily).
 
> What about adding this "about:sessionstore" into FF ???

It's a very useful add-on for you, me and a few others, but I don't think it's useful for enough users to warrant this.

> Seems FF write a lot on HDs [...]
> This was one of my reasons to suggest a SQLite-Solution ... because you can
> write in a DB just the tabs that change. I don't know if this is possible
> with an journaled storage, too ... ???

Yes. Actually, that's pretty much how sqlite does it, too.

> Also the Dev-Team should think about the upper 5% of the statistics ...

Yes, we are working on it.
(In reply to David Rajchenbach Teller [:Yoric] from comment #14)
> (In reply to Tobias B. Besemer from comment #12)
> > What about adding this "about:sessionstore" into FF ???
> It's a very useful add-on for you, me and a few others, but I don't think
> it's useful for enough users to warrant this.

I think that should be just some lines of code. And there are other "about:xxx"'s in FF too, that get not much used by the "mainstream users" ... Maybe the Dev of the Extension will spend his code ... ???


> > Seems FF write a lot on HDs [...]
> > This was one of my reasons to suggest a SQLite-Solution ... because you can
> > write in a DB just the tabs that change. I don't know if this is possible
> > with an journaled storage, too ... ???
> Yes. Actually, that's pretty much how sqlite does it, too.

OK, sorry for my leak of understanding. A "journaled storage" writes also just the changes to the HD?
I can now google for "journaled storage" to read a little bit about what it is, but I think I wouldn't find much about it with this way ... ;-)
Have somebody an URL for me where I can read more about the file/technic that should be used ???


I report in the last days a lot of stopping scripts to Facebook and it seems they work on them ...
But I also think that a part of this "bug" is maybe in FF ...
When the scripts slowing down before I get the error message, I can see a lot of I/O on my HD ...
It seems for me, that the hanging scripts are part of the reloading advertisements on the right side of FB ...
There were also this bug about the ss.json & FB:
https://bugzilla.mozilla.org/show_bug.cgi?id=934935
Now is the question for me, if it is possible that this hanging scripts are also change something in the ss.json and if this is the reason of the high I/O ... ???
Or: If the scripts maybe start hanging because of the hight amount of I/O of the ss.json and that results in a to high delay ???
Have somebody any ideas to this ???
(In reply to Tobias B. Besemer from comment #15)
> I report in the last days a lot of stopping scripts to Facebook and it seems
> they work on them ...
> But I also think that a part of this "bug" is maybe in FF ...
> When the scripts slowing down before I get the error message, I can see a
> lot of I/O on my HD ...
> It seems for me, that the hanging scripts are part of the reloading
> advertisements on the right side of FB ...
> There were also this bug about the ss.json & FB:
> https://bugzilla.mozilla.org/show_bug.cgi?id=934935
> Now is the question for me, if it is possible that this hanging scripts are
> also change something in the ss.json and if this is the reason of the high
> I/O ... ???
> Or: If the scripts maybe start hanging because of the hight amount of I/O of
> the ss.json and that results in a to high delay ???
> Have somebody any ideas to this ???

I wrote an bug on FB about this bug here and asked for joining us here and maybe help to investigate.
David Rajchenbach Teller [:Yoric], https://bugzilla.mozilla.org/show_bug.cgi?id=989393 seems to not work as it should, but seems to be not allowed for me to reopen.
Maybe I just do something else until FF works again ...
Reopening a verified bug doesn't help anyone. If something isn't working as expected then please file a new bug describing the issue. That's much easier to tackle than to work on an already crowded bug.
(In reply to Tim Taubert [:ttaubert] from comment #19)
> Reopening a verified bug doesn't help anyone. If something isn't working as
> expected then please file a new bug describing the issue. That's much easier
> to tackle than to work on an already crowded bug.

How should I call him ???
"[Session Restore] Clean up old closed tabs and windows - now really" ???
Problem not fix for me with this patch!
Telemetry says them same AFAIK ...
Just open a bug, describe the issue.
(In reply to David Rajchenbach Teller [:Yoric] from comment #21)
> Just open a bug, describe the issue.

The issue is, that the existing patch, as you said, seems to don't change anything.
-> The solution don't work.
-> The bug isn't fixed.

I'm sure I know some reasons why ...
... but it seems I have to explain and do here a lot before it can be changed step by step ... :-/

OK, so start at the beginning ...
When is the sessionstore used ???
I think in two cases:
- The browser or the system crashes, the user was stopped this way to work, the user want to work again where he has stopped.
- The user closes the browser, but want to start again in the next session with the same tabs/windows.

The second case is IMHO the best one to look at it first.
The question should be, what the user is expecting to have/see in this case ... ???

I expect only to have the same tabs/windows open then before.
Maybe a backward- & forward-history to each tab ...
So for me, everything else can be deleted on a "normal close" of the browser with a session-restore.

IMHO this includes:
- Closed tabs & windows
- Saved page-content like cookies

I think we can now disuse if somebody else expect in this case something else then the open tabs/windows and the backward- & forward-history of this tabs and then fill bugs to it.
Well, the patch does what it is supposed to do. It just happens that it's not useful, statistically speaking. My hypothesis is that people close windows/tabs often enough that very few windows/tabs remain for two weeks in the closed list.

Now, you have an interesting point.

I would say that SR is designed for the following use cases:
1. Crash recovery without losing state;
2. Quit and restart from the same browsing position (either manually or automatically);
3. Upgrade Firefox without losing state.

For 1. and 3., we need to be able to resume in the exact same state, or users are going to be really annoyed – users can forgive a crash or an update if it just means that they lose the few seconds needed to restart Firefox, but not if they also lose their work. This means that we need to write all the costly stuff, including DOM Session Storage (which is currently what takes most of the disk space, iirc), forms, etc. Since crashes can happen at any time, we are pretty much in a dead-end here.

For 2., since the user has quit and reopened manually, we can afford to save less (actually, we already do). We could probably remove closed tabs and windows. If we decided to remove Session Cookies, Session Storage or the like, however, we would regularly restore tabs and windows that don't work, because they have lost their state. That doesn't sound good.

Now, while 2. is the least-commonly-used codepath (used only once per session), it can certainly be used to shrink the size of sessionstore.js for users who restart Firefox manually.
By the way, I have documented most of the structure of sessionstore.js here: https://wiki.mozilla.org/Firefox/session_restore#The_structure_of_sessionstore.js
(In reply to David Rajchenbach Teller [:Yoric] from comment #23)
> Well, the patch does what it is supposed to do. It just happens that it's
> not useful, statistically speaking. My hypothesis is that people close
> windows/tabs often enough that very few windows/tabs remain for two weeks in
> the closed list.

;-)
We can talk about that later again ...


> For 2., since the user has quit and reopened manually, we can afford to save
> less (actually, we already do). We could probably remove closed tabs and
> windows. If we decided to remove Session Cookies, Session Storage or the
> like, however, we would regularly restore tabs and windows that don't work,
> because they have lost their state. That doesn't sound good.

Sorry, but I see it completely turned-around.
I learned - as a workaround - to reload after a session-restore each tab.
At first, I want to know what is yet on the page - not was there a day before.
This is a problem by pages with fast changing content like communities ... e.g. when I get auto-loged-in with a cookie ...
At second, a lot of things have to be reloaded - e.g. because my IP have changed.
This are e.g. captchers ...
If a page use auto-login, the cookie should be by the other cookies, not in the sessionstore, or ???
(In reply to David Rajchenbach Teller [:Yoric] from comment #23)
> For 2., since the user has quit and reopened manually, we can afford to save
> less (actually, we already do). We could probably remove closed tabs and
> windows.

I would be quite angry if we do that. I often quit the browser to restart it and apply an update, and I don't see why restarting should behave differently.
(In reply to David Rajchenbach Teller [:Yoric] from comment #24)
> By the way, I have documented most of the structure of sessionstore.js here:
> https://wiki.mozilla.org/Firefox/session_restore#The_structure_of_sessionstore.js

Thx! :-)


OK, even if I think it would be the best to "fix" at first the scenario that a user quit and reopen the browser manually ...
... some other things to think about, too.

There are some new settings ...

(In reply to Emanuel Hoogeveen [:ehoogeveen] from comment #13)
> (In reply to Tobias B. Besemer from comment #12)
> > (In reply to David Rajchenbach Teller [:Yoric] from comment #11)
> > > It's actually browser.sessionstore.max_serialize_{back, forward}.
> > 
> > In 31b, too?
> 
> Yep, ever since Firefox 30 (added in bug 943339). Specifically:
> 
> browser.sessionhistory.max_entries:
>   The maximum number of session history entries to keep in working memory
> (which
>   includes the 'current' page).
> 
> browser.sessionstore.max_serialize_back:
>   The maximum number of 'back button' session history entries to store in
>   sessionstore.js (-1 = no limit).
> 
> 
> browser.sessionstore.max_serialize_forward:
>   The maximum number of 'forward button' session history entries to store in
>   sessionstore.js (-1 = no limit).
> 
> If you set both of them to 0, only the currently active entry would be saved.

... so I will add the version number, since the user can use / the browser will use them, to the wiki ... I think this should be done for as much as possible other settings, too.

If the ss needs less HD because of clean-up on exit, I have also no problem to give the user more "security" with e.g. setting the "serialize_back" to a default of 15 (now 10) ...

If the browser should "forget" old, unnecessary things, IMHO we should think about (or measure) how long (in time) a user will go back and expect to see the page as it was before.
E.g. I think that a user don't expect that he can go more back then 12h and that a page (content) gets new loaded when it is older then 6h ...
After knowing this times, when the data should be deleted, we can think/talk about how to do that without much use of CPU & I/O ...
I don't like this "idle thing" and I'm not sure, which case really means "idle" ... but maybe disuse this later ...


P.S. a simple sample about the "reloading thing" when exit and reload:
If somebody closes now his browser on this page of bugzilla and open it tomorrow again, I'm sure that he see this bug as he is now. He don't see that maybe somebody else have added an comment and he don't expect it, because he think that the page shows the bug like it is in the moment of the loading of the page.
(In reply to Tim Taubert [:ttaubert] (away July 7th-18th) from comment #26)
> (In reply to David Rajchenbach Teller [:Yoric] from comment #23)
> > For 2., since the user has quit and reopened manually, we can afford to save
> > less (actually, we already do). We could probably remove closed tabs and
> > windows.
> 
> I would be quite angry if we do that. I often quit the browser to restart it
> and apply an update, and I don't see why restarting should behave
> differently.

If you apply a FF-UPD via the About nothing will change.

The question is, if you ever have quit the browser, restarted it, and then realized, that you have before closed a tab, that you wouldn't ???

I think, if somebody closes a tab/window in a mistake, he undo his action immediately - without quit and restart the browser before!

Keeping to much data for million of users, because one user will have one time in the year a really special case, is not an good idea!

Also there should always be the question, if the really needs the last state of the page, or if not the entry in the history (to find this page again), wouldn't be enough ...
(In reply to Tobias B. Besemer from comment #28)
> If you apply a FF-UPD via the About nothing will change.

Sorry, I have no idea what that is/means?

> The question is, if you ever have quit the browser, restarted it, and then
> realized, that you have before closed a tab, that you wouldn't ???

Yes.

> I think, if somebody closes a tab/window in a mistake, he undo his action
> immediately - without quit and restart the browser before!

I don't think that's a valid assumption, esp. for "power users".

> Keeping to much data for million of users, because one user will have one
> time in the year a really special case, is not an good idea!

I agree with that sentiment but it's hard to talk about things like that without usage data.

> Also there should always be the question, if the really needs the last state
> of the page, or if not the entry in the history (to find this page again),
> wouldn't be enough ...

Maybe. SessionStore has been around for a long time and people got used to it. Changing behavior like that isn't always easy and might upset people that got very used to that functionality.


In general, I'm very reluctant to apply any further band-aids like limiting the amount of shistory entries stored and cleaning up closed window/tab data until we know that sessionstore file size is actually a problem.

I also think that this is not the right place to have this discussion. This is a bug tracker. Please take this to the mailing lists to discuss the feasibility of high-level ideas.

I also don't understand why pruning old data is such a big issue for you? Can you elaborate on why you think that is important? I think it would be good to reset the stage. But please don't do it here. The mailing list is a way better place for this.
(In reply to Tim Taubert [:ttaubert] (away July 7th-18th) from comment #29)
> (In reply to Tobias B. Besemer from comment #28)
> > If you apply a FF-UPD via the About nothing will change.
> 
> Sorry, I have no idea what that is/means?

Firefox (FF) Update (UPD) via the restart on the About-Dialog (About).


> > Keeping to much data for million of users, because one user will have one
> > time in the year a really special case, is not an good idea!
>
> I agree with that sentiment but it's hard to talk about things like that
> without usage data.

Yes, so the questions about measuring.


> I also think that this is not the right place to have this discussion. This
> is a bug tracker. Please take this to the mailing lists to discuss the
> feasibility of high-level ideas.
> 
> I also don't understand why pruning old data is such a big issue for you?
> Can you elaborate on why you think that is important? I think it would be
> good to reset the stage. But please don't do it here. The mailing list is a
> way better place for this.

I think a bug called "Investigate how to redesign sessionstore.js for improved performance" is an excellent place for this.
But if David Rajchenbach Teller as "Assigned To" don't like it, I will stop.


There would also be the option to make one or more settings, that the users can keep the old behavior of FF, if the like too ...
Indeed, the dev-platform mailing-list is probably a better medium for this kind of conversation than bugzilla.
(In reply to David Rajchenbach Teller [:Yoric] from comment #31)
> Indeed, the dev-platform mailing-list is probably a better medium for this
> kind of conversation than bugzilla.

OK, a lot of changes in the last years here ...
Where can I find the "dev-platform mailing-list" ???
Is there already something to it ???


Post this two things there again:

(In reply to Tobias B. Besemer from comment #8)
> Created attachment 8443670 [details]
> 2014-06-20 18_35_32-about_sessionstore.png
> Screenshot from "about:sessionstore" with high "Disk writes to
> sessionstore.js over an 8 hour day".

(In reply to Tobias B. Besemer from comment #15)
> I report in the last days a lot of stopping scripts to Facebook and it seems
> they work on them ...
> But I also think that a part of this "bug" is maybe in FF ...
> When the scripts slowing down before I get the error message, I can see a
> lot of I/O on my HD ...
> It seems for me, that the hanging scripts are part of the reloading
> advertisements on the right side of FB ...
> There were also this bug about the ss.json & FB:
> https://bugzilla.mozilla.org/show_bug.cgi?id=934935
> Now is the question for me, if it is possible that this hanging scripts are
> also change something in the ss.json and if this is the reason of the high
> I/O ... ???
> Or: If the scripts maybe start hanging because of the hight amount of I/O of
> the ss.json and that results in a to high delay ???
> Have somebody any ideas to this ???

???
(In reply to Tobias B. Besemer from comment #25)

> > For 2., since the user has quit and reopened manually, we can afford to save
> > less (actually, we already do). We could probably remove closed tabs and
> > windows. If we decided to remove Session Cookies, Session Storage or the
> > like, however, we would regularly restore tabs and windows that don't work,
> > because they have lost their state. That doesn't sound good.
> 
> Sorry, but I see it completely turned-around.
> I learned - as a workaround - to reload after a session-restore each tab.
> At first, I want to know what is yet on the page - not was there a day
> before.
> This is a problem by pages with fast changing content like communities ...
> e.g. when I get auto-loged-in with a cookie ...

Tobbias, this is your use case. I have exactly the opposite one. A lot of changes can happen to a page over time. And, when I restore my session, I want to get back everything *as it was before*. Not as it is now. A perfect session restore would re-create the situation as if the crash had not happened.

(In reply to David Rajchenbach Teller [:Yoric] from comment #24)

> By the way, I have documented most of the structure of sessionstore.js here:
> https://wiki.mozilla.org/Firefox/
> session_restore#The_structure_of_sessionstore.js

Thank you Dave. It would be nice to add the "userTypedValue" to this page. This is an optional attribute. When it is present, it stores the text entered by the user in the address bar.
(In reply to Nicolas Barbulesco from comment #33)
> (In reply to Tobias B. Besemer from comment #25)
> 
> > > For 2., since the user has quit and reopened manually, we can afford to save
> > > less (actually, we already do). We could probably remove closed tabs and
> > > windows. If we decided to remove Session Cookies, Session Storage or the
> > > like, however, we would regularly restore tabs and windows that don't work,
> > > because they have lost their state. That doesn't sound good.
> > 
> > Sorry, but I see it completely turned-around.
> > I learned - as a workaround - to reload after a session-restore each tab.
> > At first, I want to know what is yet on the page - not was there a day
> > before.
> > This is a problem by pages with fast changing content like communities ...
> > e.g. when I get auto-loged-in with a cookie ...
> 
> Tobbias, this is your use case. I have exactly the opposite one. A lot of
> changes can happen to a page over time. And, when I restore my session, I
> want to get back everything *as it was before*. Not as it is now. A perfect
> session restore would re-create the situation as if the crash had not
> happened.

On a crash: Yes!
But it was about quit (close) the browser "normal" via "Exit" and then start the program new.
(In reply to Nicolas Barbulesco from comment #33)
> (In reply to David Rajchenbach Teller [:Yoric] from comment #24)
> > By the way, I have documented most of the structure of sessionstore.js here:
> > https://wiki.mozilla.org/Firefox/
> > session_restore#The_structure_of_sessionstore.js
> Thank you Dave. It would be nice to add the "userTypedValue" to this page.
> This is an optional attribute. When it is present, it stores the text
> entered by the user in the address bar.

Have I added it on the right position ???
(In reply to Tobias B. Besemer from comment #35)

> > > By the way, I have documented most of the structure of sessionstore.js here:
> > > https://wiki.mozilla.org/Firefox/
> > > session_restore#The_structure_of_sessionstore.js

> > Thank you Dave. It would be nice to add the "userTypedValue" to this page.
> > This is an optional attribute. When it is present, it stores the text
> > entered by the user in the address bar.

> Have I added it on the right position ???

Thank you Tobias. :-) But the attribute has to be moved up one level, sibling of "entries". I see now that there is another attribute "userTypedValue", at the correct level, but saying "test" instead of "text".
(In reply to Nicolas Barbulesco from comment #36)
> (In reply to Tobias B. Besemer from comment #35)
> 
> > > > By the way, I have documented most of the structure of sessionstore.js here:
> > > > https://wiki.mozilla.org/Firefox/
> > > > session_restore#The_structure_of_sessionstore.js
> 
> > > Thank you Dave. It would be nice to add the "userTypedValue" to this page.
> > > This is an optional attribute. When it is present, it stores the text
> > > entered by the user in the address bar.
> 
> > Have I added it on the right position ???
> 
> Thank you Tobias. :-) But the attribute has to be moved up one level,
> sibling of "entries". I see now that there is another attribute
> "userTypedValue", at the correct level, but saying "test" instead of "text".

OK, I removed the first one and fixed the second one. Now all right?
A discussion about changes/improvements to the Session Restore (sessionstore):
https://groups.google.com/forum/#!topic/mozilla.dev.platform/JHrOP3yMgfg
(In reply to Tobias B. Besemer from comment #37)

> OK, I removed the first one and fixed the second one. Now all right?

Yep, thank you !
Comment on attachment 8443670 [details]
2014-06-20 18_35_32-about_sessionstore.png

The sessionstore screenshot posted by Tobias (comment 8) shows a "maybe" estimate of 9+ GB disk writes in an 8 hour day. Mine's double that at 18GB in an "8 hour day". These are huge "invisible" figures for disk writes from one application.

I'd like to check if that's realistic/accurate, or just a scary maximum based on "if Firefox wrote your session at the maximum permitted frequency in <about:config>", for heavy or intensive browser users?

The reason this might matter is that many people have Firefox in use more than 8 hours a day, or have it open for longer periods (not shut down). That can easily push sessionstore writes up to 2 or 3 times the stated amount in <about:sessionstore> (depending on how realistic those figures actually are and how valid the "up to 25-50% less" is for their session). Although disk cache is larger than sessionstore, sessionstore may be written much more often. The problem is that many modern desktops and most laptops are SSD based.  With a few highly engineered exceptions, most end-user SSDs are only designed and warrantied for <= 40 GB or so writes per day, a hard limit based on limited life of NVRAM that's usually described as "much more than any user will normally use".

So it could be a bit eyebrow-raising for a user with an SSD that's said to allow "more than anyone normally needs" pegged at 40 GB write/day, to find their browser stating that its session manager alone might be routinely using most of/more than that, by itself, just writing and rewriting the session store (apparently almost constantly or multi-times a minute).

What I want to flag here, is that somewhere or other, sessionstore + SSD needs a brief double-check, to ensure there are no averse implications, if either 1) sessionstore *is* designed to save sessions in such a way that could cause extreme SSD-impacting write levels "invisible" to the user and "behind the scene" in heavy 24/7 "power user" usage, or 2) if sessionstore *doesn't* do writes at this level but the UI suggests to the user (via about:sessionstore or otherwise) that it is doing so.


(Write amplification ignored for this as its impact for most users isn't clear)
Blocks: 727409
At the moment, it looks like the main performance issues are not related to sessionstore.js, so I'm dropping that bug.
Assignee: dteller → nobody
Depends on: 1147822
Blocks: 1304389
I will say that I'm having a problem the last several months where every 15 seconds, many of my other apps (World of Warcraft for example) pauses every 15 seconds for a second or so, if Firefox is running. Session store seems a likely candidate for being the source of this issue; I'm looking into that now.
(In reply to Eric Shepherd [:sheppy] from comment #42)
> I will say that I'm having a problem the last several months where every 15
> seconds, many of my other apps (World of Warcraft for example) pauses every
> 15 seconds

If you change browser.sessionstore.interval to something other than 15000, does your frequency of pauses also change?

What is size of sessionstore.js?  If it's huge, you might be able to reduce the size with the about:sessionstore addon -  using the "forget" buttons at the bottom.
No longer blocks: sessionRestoreJank
Whiteboard: [fxperf]
Writing to sessionstore.js occurs off of the main thread now. Certainly, for the sake of SSDs everywhere, it might be worth finding more efficient way of writing state to disk in a way that doesn't involve us blasting a huge JSON string out every time one small property changes.

However, I wouldn't consider it an fxperf bug, nor priority.
Whiteboard: [fxperf]
No assignee, updating the status.
Status: ASSIGNED → NEW
No assignee, updating the status.
No assignee, updating the status.
No assignee, updating the status.
QA Whiteboard: qa-not-actionable

In the process of migrating remaining bugs to the new severity system, the severity for this bug cannot be automatically determined. Please retriage this bug using the new severity system.

Severity: major → --
Type: defect → task

Session Store causes a number of performance issues. This is a large (several megabytes), monolithic file, which needs to be fully rewritten every few seconds, which needs to be fully loaded on startup, which needs to be fully rebuilt on shutdown, etc.

I believe I'm experiencing the same issue but the file is now named recovery.jsonlz4. It is being rewritten in its entirety very often. Please correct me if I'm at the wrong place.

I've been reading a bit on this issue, and I like the idea of reimplementing the sessionstore to use SQLite; see bug 937651. I've commented on the concerns that have marked it WONTFIX, hopefully enough to consider it again as a possible resolution of this bug here.

Otherwise what other redesigns of sessionstore are under consideration?

P.S. I really like the side-effect of having the file readable with simple tools. SQLite files are open-able even with GUIs these days. But what non-custom tool can open jsonlz4 files?

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: