Closed Bug 1025924 Opened 10 years ago Closed 7 years ago

Startup time for session restore takes more than 30 seconds (many tabs in many tab groups)

Categories

(Firefox :: Session Restore, defect)

Product:
Firefox
Component:
Session Restore
Version:
29 Branch
Platform:
x86
All
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED DUPLICATE of bug 906076

People

(Reporter: xtremesniper, Unassigned)

References

(Blocks 1 open bug)

Details

(Keywords: perf, regression)

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:30.0) Gecko/20100101 Firefox/30.0 (Beta/Release) Build ID: 20140605174243 Steps to reproduce: Close the browser with many tabs open (anywhere from 10-50 or more) and many tab groups (I have 35 tab groups, though most of the time they remain dormant and not used). Re-open browser and expect session to be restored. Actual results: Browser opens and I see the UI render 1 blank tab, then after a few seconds it shows the outline of all my expected tabs loading in. But then the Firefox process goes 100% CPU and sits there in a non-responsive state for 30-60 seconds before continuing to load the tabs. Note: I have "load tabs on demand" checked and only 3 pinned tabs. This only started happening as of v29 and still happens in v30. Expected results: Browser should load with previous session without going into non-responsive hang.
Something very similar to this happens here with Firefox 30.0 on GNU/Linux x86_64. I don’t know when it started, and I was using Pentadactyl for some time until a few days ago.
QA Whiteboard: [bugday-20140623]
OS: Mac OS X → All
A part of my about:tabs (only one tab is pinned, tabs load on demand): 278 tabs across 26 groups in 1 window 6 tabs have been loaded 243 unique addresses 92 unique hosts 25 empty tabs 158 https: 90 http: 4 about: 1 zotero: Do you use Zotero, proxies?
Component: Untriaged → Session Restore
I have Zotero downloaded but its disabled. I don't have access to any about:tabs page, is that an aurora feature? My current addons: DownThemAll! Flashblock Ghostery HTTPS-Everywhere NoScript Session Manager YouTube High Definition Didn't think it was relevant because there was a distinct change from v28 to v29 without changes to my addons.
about:tabs is from https://addons.mozilla.org/firefox/addon/tab-stats/ I have, among others: DownThemAll! (installed relatively recently) HTTPS-Everywhere NoScript FlashVideoReplacer
Thanks, I installed the addon (and disabled it after grabbing the info): 458 tabs across 33 groups in 1 window 14 tabs have been loaded 453 unique addresses 293 unique hosts 138 https: 318 http: 2 about: I know, I have a tab hoarding problem. =P I'm working on closing a bunch of them slowly but surely. I find that even when I disable or enabled various add-ons, the issue remains pretty consistent. That said, when I start up a fresh profile with no tabs and no add-ons, the issue goes away. I'm fairly certain that it's the tabs, not the add-ons, causing the issue... But I don't know for sure, and this is why I created the bug.
Something I noticed at approximately the same time, but which seems to have been caused by Pentadactyl: bug 1029547 comment 2.
I disabled all extensions (over 20), and the process hasn't improved significantly.
Status: UNCONFIRMED → NEW
Ever confirmed: true
I suspect that we could improve the responsiveness by simply delaying the restoration of all but one windows by a few milliseconds.
(In reply to xtremesniper from comment #0) > Note: I have "load tabs on demand" checked and only 3 pinned tabs. This only > started happening as of v29 and still happens in v30. Can you start version 28 with same sessionstore.js and confirm it works great in version 28?
Flags: needinfo?(xtremesniper)
Keywords: perf, regression
Version: 30 Branch → 29 Branch
fx 28 — 11-12 sec fx 29 — 20? fx 30 — 23? fx 31 — 27 (may be affected by my keypresses, as the first tab was a start page instead of an empty one) my fx 31 — 29 my fx 31 with my profile — >60 "my fx 31" is a copy affected by mandatory access control; "my profile" contains many extensions; the rest are new profiles with "When Firefox starts:" changed.
Summary: Startup time for session restore takes more than 30 seconds → Startup time for session restore takes more than 30 seconds (many tabs in many tab groups)
Currently on FF 36 on startup it presents the current tab group with the shortcut title names. After a pause FF then blanks out the titles and replaces them with "New Tab" like it is performing a reset. Not too long after this, FF then redisplays the shortcut titles and eventually loads the URLs. Strange behavior. Also, with this current version of FF the previous window position and size is not being restored on subsequent launches.
Using 37.0.1 (and before) (on Linux) I have the same issue. Each start up is very very frustrating (really). I have around 990 tabs in my groups. A side effect is that when I modify the size of my Firefox and afterwards I go to the "Group your tabs" view it takes a long long time to reorganize the tabs (and they are improperly reorganized, overlapping each others, it's barely usable... but I keep using it... frustrating). Sometimes the view scripts hangs on prompting me to Force Quit or Wait (waiting usually resolves the issue). My guess is that there is something wrong about the tabs. It is very nice way to bookmark my pages. In fact I don't "bookmark" ctrl+D anymore, I usually put any interesting page in a group. However it seems that at each start up some kind of computation is done on ALL the tabs. I guess it should not be done like that, only the current group should be actively computed ( or I should use the group tabs differently). Same issue with the resizing of the groups view.
I will confirm the observation reported in comment #13 regarding FF manipulating the tab groups. I find that it randomly resizes the size of the group box, will move them somewhat which causes auto positioning to occur. This distorts the alphabetical order I put the tab groups in. Organizing tab groups must be performed manually since FF offers no auto sort by name, date, last used, etc.
There are two issues at hand: - Panorama/Tab Groups needs to open all tabs during startup (this should eventually be handled as part of bug 836758, currently inactive); - opening a tab is slow (actively handled in bug 906076).
Depends on: lazytabs
There are many reported FF bugs related to this one common problem. Having said that there is another issue that implies there is a independent bug that relates to the root cause of the this issue. I reported my initial findings in bug 1140851 which has been determined to be this same bug except for the fact the I reported the group button delay which produces the same lethargic behavior. To confirm that FF is accessing all of the open tabs, I created a new instance of FF by logging in as a different FF user while leaving my current FF instance open. The former had 6 open tabs and no groups while the latter had 16 open tabs and over 30 groups with about 400 tabs overall. The newly created instance with 6 open tabs showed no sign of lethargic performance. All tabs and pages responded immediately whereas with my instance of 16 tabs and 30 groups yields waiting 2 and 3 minutes before I get a response from actions as simple as typing or a refresh from clicking on a different tab. At times "top" shows FF using 350% of cpu time and I am not even interacting with the mouse or keyboard. This confirms my hunch that FF is performing a lot of interactions with all of the open tabs contained in all of the groups and not focusing on the current selected tab. Moreover, under FF preferences I have selected "Don't load tabs until selected." When this option is selected FF should only access the database to retrieve and display the tag title for each tab in the current displayed group and give all focus to the current active tab. The fact that FF does not do this presents another bug that is related to the "Don't load ..." preference as FF is "loading" something in the background for all of the tabs in all of the groups when selecting this preference should block this activity. I do not see a short term solution. Therefore, The immediate fix for this is to revert all of my tab groups into Bookmark groups with subfolders and sublinks.
Update to comment16. I have removed all of my FF groups. Instead of groups I have created subfolders off of main subject folders in the Bookmark menu. Since the last entry of a folder is "open all in tabs", I can open the links contained in the folders just like switching to a group. Furthermore, to keep the newly open links separate from my main FF page, I create a new tab on my FF main window then right click it and select "move to new window." Once the new window is open I can open a Bookmark folder and select the "open all in new tabs" option. This is actually more productive than using FF groups since I can keep my main FF page open and then selectively create new windows for each additional topic I need to work on. The concept of groups versus subfolder groups may need to be debated especially due to the cpu performance hit FF takes when creating many groups. By using Bookmarks, the need for FF to service these links as it does when using groups is eliminated. This greatly improves FF performance. My cpu is almost idle now. I can run a virtual machine while running FF and the fan never kicks on. As I deleted the FF groups I noticed that performance was going up. By the time I had deleted about 20 groups and approximately 200 tabs I really started noticing better performance. There is no doubt about the fact that when many groups with many tabs exist FF is consuming lots of cpu time in the background that reduces the current user's productivity. I still observed that FF does not store the last screen size. I also have discovered why the first FF window displays stale tabs. When FF is closed it does not update the the current open tabs but only uses what was last stored when the "use current pages" button was selected. FF launches this way and then has to use history to find out the last tabs that were open. When the menu item "show my windows and tabs from the last time" is selected FF should perform a programmatic "use current pages" option upon closing. I manually selected the "use current pages" option before closing FF and then restarted FF. This reduced the amount of time required for FF to get my tabs open on launch. This is another indication that accessing a large history database is slowing FF down. Additionally, since FF does not currently invoke the "use current pages" before closing then this too could also be why it is not saving the last window size.
More observations regarding FF and group alternatives. I performed the same test on using Chrome. I found that with 12 tabs open in the idle mode that Chrome was consuming slightly more cpu time than FF but both were below 15% cpu. Chrome creates multiple instances and used slightly more memory. I consider the latter less desirable since I already have C++ FF "helper" applications that will read native files, create the desired html and then launch a new copy of FF under a different user. I want to limit the number of browser instances running at any one time. Chrome has a context Bookmark menu feature that when right clicked gives the option to "open all tabs in new window." FF when performing the same function offers, "New Bookmark" and "New Folder." To open all tabs using FF forces the user to open the subfolder and scroll down to the last entry which is "open all in tabs." This feature and opening "open all in tabs in new window" improves productivity when they are in the context menu at the folder level. These two features replace the groups feature and are actually easier and faster to use. Therefore, I believe the "groups" feature of FF should be removed from FF and become an extension add-on as not all users will embrace this feature especially due to the lethargic behavoir it introduces FF performance. In summary, here the current issues that need addressing 1) FF does not preserve previous window size and location 2) FF does not perform a programmatic "use current windows" preference on closing 3) FF does not offer the capability to "open all tabs in new window." The workaround is to right click on an open tab and select "Move to a new window." From there access the last entry of the Bookmark subfolder that you want to open and select "open all in tabs." For others who are using FF groups the way I have, please try my alternative using Bookmark subfolders with subfolders and provide feedback that confirms my findings. And/or launch Chrome and try this method there. Chrome will directly import FF Bookmarks. The FF team needs direction on how to fix this mess. At this point it would be much easier to remove the FF group feature and add just one Bookmark menu item.
Under OS X with some 200 tabs open and Adblock Plus the session restore time in Version 39 is 30 Minutes (!!) up from 10 Minutes under previous versions. "Don't load tabs" is enabled. Firefox spends nearly all its CPU time in js::RemoveRawValueRoot(JSContext*, JS::Value*). It is not taking advantage of Multicore CPUs. Moreover, Firefox _always_ crashes when it is quit regularly.
Could it be the parsing of previous/recovery.js that is slow? I use Firefox 39 with roughly 21 tab groups (though the issue has been around for as long as I can remember, from the Firefox 4/5/6/7 days). As can be seen in ~/.mozilla/firefox/[…]/sessionstore-backups : $ du -h recovery.js 2.4M # that's a HUGE size for a one-line javascript file… $ grep -o '"url":' recovery.js | wc -w 1354 # so roughly 1354 tabs?
And it should be noted that with those 1300+ tabs and the "don't load anything but the current active tab" setting, my Firefox takes 1 minute and 5 seconds to start. It actually shows up (the main window) within 4 seconds, then takes roughly 30-50 seconds to start showing that there are other tabs in the UI, and the rest to actually load one and unfreeze the UI. This really sounds like the process is blocked while parsing something (hence my suspicion towards the previous/recovery session js files) before anything UI-related can even happen.
Interestingly enough, closing a few tabs here and there in my groups, the recovery.js (and previous.js) filesize went down from 2.4 M to 1.0 M, and the startup time went from 1m 05s to 45s. And yet, according to my grep command in comment #20, I "only" closed 10% of the tabs (~150 URLs out of 1350*), yielding a 30% startup time improvement — it's as if the performance hit was exponential… and this is on a Solid State drive with a Core2 Quad processor. *: not 100% sure about those numbers based on the number of occurrences of '"url":' in the js files; to me it certainly doesn't feel like I closed over a hundred tabs in two minutes.
(In reply to Jeff Fortin from comment #22) > *: not 100% sure about those numbers based on the number of occurrences of > '"url":' in the js files; to me it certainly doesn't feel like I closed over > a hundred tabs in two minutes. You can always use an extension like Tab Stats (https://addons.mozilla.org/en-US/firefox/addon/tab-stats/?src=api) to check the numbers for sure. Clearing the needinfo request because I do not have that environment anymore. In case someone else has the ability to conduct the test, this was the request: > Can you start version 28 with same sessionstore.js and confirm it works great in version 28? Because originally on v28 the startup time was slow but not that slow. When v29 came around, it dramatically increased the startup time and it hasn't changed/improved since.
Flags: needinfo?(xtremesniper)
2 years later, I can confirm that I still see this bug with Firefox 48 and Nightly builds, even when starting up with all add-ons disabled, and tabs are loaded only when selected , startup time is horrible with about 300 tabs : more than 10 seconds, didn't bother to check exactly how much time it is . Any chance that this will get fixed ?
The dependent bug 906076 has been in progress for the last few months. Still a ways to go, but definitely some progress towards lowering the overheard of lots of unrestored tabs.
After loading FF on my iOS device for other reasons, I learned that FF is very responsive. The reason why is that it only loads one page at a time. FF on my desktop continues to struggle when first opening due to then need of loading all open tabs at once. Not sure why there are multiple versions of FF with different designs. Just makes it much more difficult to get an efficient and stable release. Much of this discussion centers around the fact that the FF Preference Option "update Tab only when clicked on" (which appears to have been removed) never worked as described. Why the Desktop version which updates all open tabs works differently than the mobile app needs addressing. Furthermore, due to the sluggishness caused by the database access and multiple tabs being loaded on opening, the FF developers are now rewriting FF (again) so that it will spawn more background tasks. I don't know how the developers believe this is going to increase the users cpu speed. The fact is by spawning more applications the cpu load moves from the FF application to the preemptive scheduler. For anyone that believes that a preemtive scheduler is going to save the day I would invite them to load up VMware and launch a virtual machine and then run FF. The learning point is that things like swap, IO and cache tasks will always take precedence over the user interface. More scheduler cpu will be required in exchange for less FF cpu time. How with this make FF faster? I have tuned into the heated discussions on making FF multithreaded and don't want any part of it. I can only hope that the option to turn off multithread option is permanent fixture. I have used Chrome and when opening only a few URLs the number of spawned tasks is overwhelming. If a page locks up, might as well plan on rebooting as to trace down all the tasks and hope to kill the right ones. With single threaded FF I routinely have multiple copies of FF running under different users and it is quite easy to find which FF instance needs attention if locked up.
Likely not the right place to discuss this. However I think some of the things you state in this message should be answered. (In reply to PJAL from comment #27) > After loading FF on my iOS device for other reasons, I learned that FF is > very responsive. The reason why is that it only loads one page at a time. > FF on my desktop continues to struggle when first opening due to then need > of loading all open tabs at once. Not sure why there are multiple versions > of FF with different designs. Just makes it much more difficult to get an > efficient and stable release. Firefox on iOS is completely different, because iOS forces us to do it. There are approximately zero lines of code shared between the projects. > > Much of this discussion centers around the fact that the FF Preference > Option "update Tab only when clicked on" (which appears to have been > removed) never worked as described. Why the Desktop version which updates > all open tabs works differently than the mobile app needs addressing. Firefox/Desktop doesn't load all open tabs at once nowadays. Probably the reason why the preference is not present anymore. > > Furthermore, due to the sluggishness caused by the database access and > multiple tabs being loaded on opening, the FF developers are now rewriting > FF (again) so that it will spawn more background tasks. I don't know how > the developers believe this is going to increase the users cpu speed. The > fact is by spawning more applications the cpu load moves from the FF > application to the preemptive scheduler. The reason we move to a multiprocess architecture is because computers have several CPUs. This is especially important on mobile where CPU are not individually so powerful but we have a few of them. The goal is to use all these CPUs. > > For anyone that believes that a preemtive scheduler is going to save the day > I would invite them to load up VMware and launch a virtual machine and then > run FF. The learning point is that things like swap, IO and cache tasks > will always take precedence over the user interface. More scheduler cpu > will be required in exchange for less FF cpu time. How with this make FF > faster? > > I have tuned into the heated discussions on making FF multithreaded and > don't want any part of it. I can only hope that the option to turn off > multithread option is permanent fixture. I have used Chrome and when > opening only a few URLs the number of spawned tasks is overwhelming. We don't implement it like Chrome did. > If a > page locks up, might as well plan on rebooting as to trace down all the > tasks and hope to kill the right ones. With single threaded FF I routinely > have multiple copies of FF running under different users and it is quite > easy to find which FF instance needs attention if locked up. It will still be easy to kill a content process or a firefox instance -- the additional benefit is that if you kill a content process the whole instance won't be killed. You might want to look at bug 1312373 and its dependencies. A lot of work is being done these days to make a restart faster. Yay !
Good information, thanks for the insight.
Status: NEW → RESOLVED
Closed: 7 years ago
No longer depends on: lazytabs
Resolution: --- → DUPLICATE
New insight and response to comment #28. Maybe not the right place for this discussion but it needs to be stated somewhere. These comments are based on FF 54.0 64 bit on Fedora Linux fc25. Laptop configuration is a quad AMD A8-4500M with 4 GB memory 750 GB 7200 RPM hard drive. All broswers are overloading the cpus and consuming most of the memory on desktop versions. My experience is FF consumes up to 70% of memory 3.2 GB) and 200% of cpu at times. In researching this I have learned that both Safari and Chrome users see the same thing. One account reported that Safari was using over 6 GB of memory. Browsers today are becoming memory, cpu and power hogs. When I have many tabs open sometime it requires 5-10 minutes of wait just to click a different tab and get is displayed. This is due to the OS going into a state of continuous thrashing. The learning points are this. 1) The load that a browser page may place on the cpu is not predictable since the pages themselves have inserted widgets that add a significant load. The typical webpage is not at all static. 2) The dynamic parts of today's webpage run continuously and keep a network connections alive. Even those these might be background jobs they are using cpu power and draining laptop batteries. This is why Safari on iPhone restricts developers from doing creating multiple thread as draining batteries fast is not a good selling point. 3) To emulate single webpage browsing like Safari on iPhone I have now migrated to using the FF "Library" interface (using this link in the first tab chrome://browser/content/places/places.xul) and only having 1 webpage in the next tab most of the time. This has resulted in cpu consumption for FF of 15% on average and memory use 1 GB or less. My fan runs at a low speed is quite. 4) I turned of FF History a couple of years ago and deleted all my Groups. And, I run this command "for i in $(ls *.sqlite); do sqlite3 $i vacuum; done" to reduce any load from sqlite. 5) What I am essentially doing is running FF on a laptop much like FF runs on an iPhone. The fact is to extend battery life there can only be one page running at a time. I don't care about how the FF or Chrome code is designed or multi-threaded. Background jobs with network connections eat up batteries. This is what Apple has learned and why they rewrote Safari for the iPhone. 6) The concepts of History, Groups and now Tabs ( more than a few ) are all dead. They consume too much power and cannot handle the exponential cpu load that each open webpage adds. This is the direction Apple has gone by making it easier to just use Bookmarks. FF Library does this too and will be more effective if a few bugs that I reported get fixed. Just passing these observations along. Nothing I want to debate. I just don't want to wait 10-15 minutes between Tab clicks.
You should check out https://metafluff.com/2017/07/21/i-am-a-tab-hoarder/, you'll see that the next release version of Firefox will be much better when a lot of tabs are opened, both for CPU usage and memory. I think (I hope :)) that this will make your experience more enjoyable.
See Also: → 1814596
You need to log in before you can comment on or make changes to this bug.