624409 - jank and high Memory

Status: RESOLVED DUPLICATE of bug 669603

(Firefox :: General, defect)

Description

[IU]

I have a session that causes extremely high CPU and memory consumption and makes the browser mostly unresponsive, as it churns away at who knows what -- especially with Firefox (32-bit) on Windows 7.

I have saved the session, using the Session Manager extension; however, the session contains lots of sensitive login and account information.  I cannot attach the session file, nor can I supply it to anyone; however, I am hoping one of the developers may be able to walk be through generating debugging/profiling output that I can supply, to help track down the cause of this.

Comment 1

[The 8472]

does the high CPU usage persist even when all tabs are loaded? Even if you turn off javascript (could be a some site burning CPU time with javascript)?

if the answers are yes then see https://developer.mozilla.org/en/how_to_get_a_stacktrace_with_windbg

Comment 2

[Paul O'Shannessy [:zpao] (not bugmail, email directly)]

This sounds like a general "high cpu" bug, not a session restore bug.

Look for a tab with Flash open. That's often the culprit (for me anyway). Otherwise it could be something JS related as suggest above. Keeping a copy of that saved session and then closing your open tabs 1 by 1 until CPU usage goes down would be helpful.

You set the version to trunk. Are you using a Firefox 4 nightly?

Comment 3

[IU]

Nope.  Not a JavaScript or Flash bug.

Still happens even after I have disable: JavaScript, Java, Flash, Silverlight, and Images

It could be related to one site: https://www.methodintegration.com/method/Default.aspx?...(the rest omitted for security reasons), as the presence of its tab increases the strain (though, its absence does not eliminate it).

However, it is something stored in the session file (27 MB) causing the problem, as it still happens even though I have disabled all I've indicated above.

It happens even when that session file is loaded in a new profile.  Furthermore, it does not occur if I load the above indicated URL in a profile that does not have the affected session's contents.

Comment 4

[Paul O'Shannessy [:zpao] (not bugmail, email directly)]

(In reply to comment #3)
> However, it is something stored in the session file (27 MB) causing the
> problem

Ok, so here's what we'll do. Your session file shouldn't get that large. If it does, then either you're using about:sessionrestore wrong (which is ultimately our fault for letting you) or we're doing something wrong. Bug 467409 should help but not until after Firefox 4.

When you load your session, is about:sessionrestore one of your open tabs? If so you want to get rid of it. You can close it, then reset your recently closed tabs list by setting browser.sessionstore.max_tabs_undo to 0. (you can then set it back to default, changing it will flush the data).

You sessionstore.js file should be much smaller at this point. If not then you might have something similar in your recently closed windows list.

Ultimately, starting a new session and just loading the urls you need might help a lot.

Comment 5

[IU]

(In reply to comment #4)
> When you load your session, is about:sessionrestore one of your open tabs?

No about:sessionrestore open.  I used Session Manager to save then load the bad session.  After that, it was all Firefox.

> reset your recently closed tabs list by setting browser.sessionstore.max_tabs_undo
> to 0. (you can then set it back to default, changing it will flush the data).
> You sessionstore.js file should be much smaller at this point.

Didn't help.  Tried it and it did not reduce the size.

> Ultimately, starting a new session and just loading the urls you need might
> help a lot.

Sure, but this is a nasty bug that takes time to figure out it's a sessionrestore bug, and many users aren't going to know what to do.

Comment 6

[Paul O'Shannessy [:zpao] (not bugmail, email directly)]

(In reply to comment #5)
> Sure, but this is a nasty bug that takes time to figure out it's a
> sessionrestore bug, and many users aren't going to know what to do.

Well, it's sounding like a bug with session restore (and it's possible though not super likely that it's connected to session manager). I don't believe you could make a 27MB session file without nested about:sessionrestore pages in there. But that's bug 467409. I'd like to dupe this to that, but without confirmation that's a bit hard. Can you look inside your sessionstore.js for "about:sessionrestore"?

Comment 7

[IU]

(In reply to comment #6)
> and it's possible though not super likely that it's connected to session manager

Session Manager doesn't have to be enabled to experience this bug.

> Can you look inside your sessionstore.js for "about:sessionrestore"?

Searched and there is none found.  However, there's a whole lot of {"url":"about:blank","subframe":true,"ID":... in there all over the place.

Therefore, I'm suspecting that Panorama might be ultimately responsible for this.

Comment 8

[Paul O'Shannessy [:zpao] (not bugmail, email directly)]

(In reply to comment #7)
> Searched and there is none found.  However, there's a whole lot of
> {"url":"about:blank","subframe":true,"ID":... in there all over the place.
> 
> Therefore, I'm suspecting that Panorama might be ultimately responsible for
> this.

Panorama shouldn't be messing with history. Those look to be part of the history of a page with <frameset> (though "about:blank" is a bit weird).

Would you be willing to do some more digging in that file and extract the parent object? (it represents a "tab" and should begin with {"entries": [ {objects like yours}) This would mostly just hold URLs and wouldn't contain any cookies, so should be relatively safe to put here or send to me if you don't want to do that.

Comment 9

[IU]

Did some more digging, but can't send the parent object, as it has login/session information in the URL.  It has 56 children entries with 447 about:blank entries (and 74 various occurrence of the man URL) and is 421 KB in size (430,251 characters).

It turns out all that garbage was generated by that very same https://www.methodintegration.com website.  Even though I had closed all other tabs and left only on tab, plus setting browser.sessionstore.max_tabs_undo to 0, there are thousand (3,940 to be exact) entries with children entries.

This explains why I saw the close button on for that tab flickering so badly.

Is this information helpful at all or do you still need to see the URL somehow?

Comment 10

[SpeciesX]

i don't know, if im right here, but on this site http://www.militaryphotos.net/forums/showthread.php?99988-Russian-Photos-%28updated-on-regular-basis%29/page2737, use Firefox 4 900-1000mb Memory, for same site need IE9 RC only 80mb and Opera 200mb. Why need firefox so many memory?

Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b12pre) Gecko/20110217 Firefox/4.0b12pre

Comment 11

[Randell Jesup [:jesup] (needinfo me)]

Bug 669603 has a sequence (admittedly for Linux) detailing how to "pretty-print" the sessionstore.js file so that it's readable.

a) how many windows?
b) how many tabs?
c) have you tried opening Sysinternals Process Explorer and watch it while closing windows and/or tabs to isolate the problematic tab or window?
d) have you tried this on Aurora (7.0alpha)?
e) if the MethodIntegration site doesn't have any sensitive info, can you snip and attach it?  Or Xxx out the sensitive items?
f) Do you have access to a Linux box where you could move the sessionstore.js file to, for pretty-printing, and possibly for doing a jprof-enabled build so we could get a profile of what it's doing when it's so busy?

Comment 12

[IU]

(In reply to speciesx from comment #10)
> Why need firefox so many memory?
 
All those other major browsers better handle image decoding (they do not decode images not in view) -- something Firefox badly needs.

That said, your issue is not related to this bug but is instead these: bug 660577 and bug 661304


(In reply to Randell Jesup [:jesup] from comment #11)
> d) have you tried this on Aurora (7.0alpha)?

I'm using nightly builds.  Using the problem sessionstore.js file, I still observe high memory consumption and sluggish tab switching, but CPU utilization no longer stays high for long (about 30 seconds on initial load) and only a few seconds on occasional tab switching.

> e) if the MethodIntegration site doesn't have any sensitive info, can you
> snip and attach it?  Or Xxx out the sensitive items?

Not really practical, given the size of the file and its content pattern.  

> f) Do you have access to a Linux box where you could move the
> sessionstore.js file to, for pretty-printing, and possibly for doing a
> jprof-enabled build so we could get a profile of what it's doing when it's
> so busy?

I do have access to Linux.  If there's a jprof-enabled build available, I could try getting a profile for you.

Comment 13

[Justin Lebar (not reading bugmail)]

IU wrote in bug 705597 comment 62:

> Unfortunately, this did not fix bug 624409, as I had hoped.  Does Mozilla have policies and a 
> process for handling potentially sensitive user data?  I'm thinking the only way ahead may be for me 
> to submit the sessionstore.js directly to Mozilla and maybe also mark bug 624409 a security bug.

The policies and processes I'm aware of are "send your compressed data to a Mozilla developer."  Maybe Jesup would be up for handling this data?

Comment 14

[IU]

Is he a Mozilla employee?  If not, what binds him to confidentiality?

Comment 15

[Justin Lebar (not reading bugmail)]

Yes, he is a Mozilla employee.  (I know it's hard to tell!)

Comment 16

[IU]

OK.  I'll wait for him to respond to your previous question.  Thanks.

Comment 17

[Paul O'Shannessy [:zpao] (not bugmail, email directly)]

(In reply to Justin Lebar [:jlebar] from comment #13)
> IU wrote in bug 705597 comment 62:
> 
> > Unfortunately, this did not fix bug 624409, as I had hoped.  Does Mozilla have policies and a 
> > process for handling potentially sensitive user data?  I'm thinking the only way ahead may be for me 
> > to submit the sessionstore.js directly to Mozilla and maybe also mark bug 624409 a security bug.

Bug 705597 only fixes for future saves. It won't fix all old sessions unless we are forced to recollect data for those tabs. If you don't load all tabs following restore (and I think take some action in them) then we'll use the cached data. Bug 707866 is intended to fix old sessions.

Until that's fixed, you may want to flush _closedWindows & _closedTabs (set browser.sessionstore.max_{windows|tabs}_undo to 0 & then reset). Cleaning out previous session history entries would also help. I wrote a script that does these 3 things to your current session (https://github.com/zpao/mozilla-scripts/blob/master/trim-session.js) but use it at your own risk (make backup first!)

Comment 18

[IU]

(In reply to Paul O'Shannessy [:zpao] from comment #17)
> Bug 707866 is intended to fix old sessions.

In that case, I'll wait for bug 707866 and retest then.  Thanks.

Comment 19

[creative.gajendra]

Memory consumption is too high, even when using a single tab on single window. It ranges between 280MB-330MB. I could provide a screenshot. I'm using Windows XP SP2 on Intel Dual Core with 1.5 GB RAM.

Comment 20

[Randell Jesup [:jesup] (needinfo me)]

IU:  Bug 707866 is now fixed, can you retest?  If still a problem, I can look at it.

[e-mail]: Please open a new bug, and attach the contents of about:memory and about:support when you see the excess use.  Please CC me on the bug and I'll take a look at it.  Thanks

Comment 21

[(dormant account)]

Not enough data to act on for snappy.

Comment 22

[IU]

Who do you guys want me to send the sessionstore.js to? Is it still Jesup or someone else within Mozilla?

Comment 23

[creative.gajendra]

Coming back after a gap of few months. Sadly the memory consumption by firefox has been mounting more than ever.

AVG indicated 255MB and Task Manager reported 245MB of usage! 

See screenshots.
TaskManager-gajendra-31052012.PNG
AVGNotification-gajendra-31052012.PNG

Comment 24

[creative.gajendra]

Attachment: Memory consumption screenshot

Comment 25

[creative.gajendra]

Attachment: Memory consumption screenshot

Comment 26

[Justin Lebar (not reading bugmail)]

~256MB of memory usage isn't at all high for Firefox.  Just Gmail plus Facebook can take up that much.

As Randell said in comment 20, if you think there's an actual problem here, please file a new bug with the contents of about:memory and about:support.  If you put [MemShrink] in the whiteboard, we'll take a look at it at our next meeting.

Comment 27

[Wayne Mery (:wsmwk)]

UI, 

Did https://developer.mozilla.org/en/Performance/Profiling_with_the_Built-in_Profiler help?

Is there a specific tab with extreme memory?   recent nightlies reveal memory on a per-tab basis in about:memory, verbose

Comment 28

[IU]

I've just profiled, but the output doesn't mean much to me.

Here's the link to the profile: http://people.mozilla.com/~bgirard/cleopatra/?report=069de107ff219ae9d7d59507764d2034f16cbe72

Comment 29

[Justin Lebar (not reading bugmail)]

This doesn't even appear to be spinning the CPU, as I read the profile.

Comment 30

[IU]

It pins a core for several seconds every time you launch the browser, open/load a tab or close a tab.  It was much worse when I filed the bug.

Comment 31

[IU]

Who in Mozilla do I send the profile to?

Comment 32

[(dormant account)]

(In reply to IU from comment #31)
> Who in Mozilla do I send the profile to?

try posting a more useful profile. Are you pressing ctrl+shift-o right after closing a tab?(don't click analyze button)

Comment 33

[IU]

I don't get it.  Are these the steps I need to perform?:

1. Launch Fx with profiler add-on
2. Close tab and wait for CPU util' to die down
3. Stop profiling
4. Upload profile

Or are there better steps?

Comment 34

[(dormant account)]

(In reply to IU from comment #33)
> I don't get it.  Are these the steps I need to perform?:
> 
> 1. Launch Fx with profiler add-on
> 2. Close tab and wait for CPU util' to die down
2. No. Close tab..if it is janking, then press ctrl+shift-o ASAP.


> 3. Stop profiling
> 4. Upload profile
> 
> Or are there better steps?

Comment 35

[IU]

Is this better? : http://people.mozilla.com/~bgirard/cleopatra/?report=5f38c085e44a5b001b3382aa5f0b1d0eb704c5ab

If not, may can just send the actual Firefox profile so a Mozilla employee?

Comment 36

[Wayne Mery (:wsmwk)]

(In reply to IU from comment #35)
> Is this better? :
> http://people.mozilla.com/~bgirard/cleopatra/
> ?report=5f38c085e44a5b001b3382aa5f0b1d0eb704c5ab
> 
> If not, may can just send the actual Firefox profile so a Mozilla employee?

It comes back with "Error in worker: Exception: SyntaxError: JSON.parse: expected ',' or '}' after property value in obj.."

Comment 37

[Mike Conley (:mconley) (:⚙️)]

IU,

Are you still experiencing this problem?

-Mike

Comment 38

[Wayne Mery (:wsmwk)]

As previously mentioned, there is no longer high CPU. So I don't think this is a specific URL nor high cpu from cycle collection (something I see often).

In absence of new profile, I suggest this is jank caused by large sessionstore.js (something I also see often)