1228615 - [e10s] (Win) Nightly (FF45.0a1) main process use a lot of memory in the last builds (2015-11-25)

Status: RESOLVED WORKSFORME

(Core :: Hardware Abstraction Layer (HAL), defect)

Description

[Tobias B. Besemer [:BesTo] (QA)]

Firefox.exe (Nightly, FF45.0a1) use a lot of memory on Win7 w/ 64bit & e10s in the last builds (around 2015-11-25).

Memory consumption till now ~750MB after startup and session restore.

Memory in the last days after startup and session restore: ~1.200MB

I can see in "about:memory":
117.56 MB (11.18%) ── heap-unclassified

I attach 2 screenshots from "about:memory".

Comment 1

[Tobias B. Besemer [:BesTo] (QA)]

Attachment: 2015-11-27 17_22_37-about_memory (live measurement) - Nightly (Build 20151126030226).png

Comment 2

[Tobias B. Besemer [:BesTo] (QA)]

Attachment: 2015-11-27 17_24_17-about_memory (live measurement) - Nightly (Build 20151126030226).png

Comment 3

[Timothy Nikkel (:tnikkel)]

Can you include the full memory report, not just a screen shot, one from before the regression, one from after? You can anonymize them. Then we can subtract one from the other to determine what increased.

Comment 4

[Tobias B. Besemer [:BesTo] (QA)]

(In reply to Timothy Nikkel (:tn) from comment #3)
> Can you include the full memory report, not just a screen shot, one from
> before the regression, one from after? You can anonymize them. Then we can
> subtract one from the other to determine what increased.

Sorry, forgot the mem report!

OK, does some tests ...

Report 1:
FF43.0a2 2015-11-28, started in offline mode, 8 windows, a lot of tabs, no page loaded
firefox.exe: 530MB; plugin-container.exe: 600MB

Report 2:
FF44.0a1 2015-11-27, started in offline mode, 8 windows, a lot of tabs, no page loaded
firefox.exe: 1.015MB; plugin-container.exe: 95MB

Report 3:
FF44.0a1 2015-11-27, started in offline mode, 8 windows, a lot of tabs, some pages loaded
firefox.exe: 1.245MB; plugin-container.exe: 248MB

Report 4:
FF44.0a1 2015-11-27, started in offline mode, 8 windows, a lot of tabs, this page loaded
firefox.exe: 1.315MB; plugin-container.exe: 145MB

If I would now load more pages/start surfing, the mem used by plugin-container.exe would grow rapidly.

Comment 5

[Tobias B. Besemer [:BesTo] (QA)]

Sorry!

FF43.0a2 -> FF44.0a2
FF44.0a1 -> FF45.0a1

;-)

Comment 6

[Tobias B. Besemer [:BesTo] (QA)]

Attachment: memory-report_-_FF44.0a2_-_no_page.json.gz

Comment 7

[Tobias B. Besemer [:BesTo] (QA)]

Attachment: memory-report_-_FF45.0a1_-_no_page.json.gz

Comment 8

[Tobias B. Besemer [:BesTo] (QA)]

Attachment: memory-report_-_FF45.0a1_-_some_pages.json.gz

Comment 9

[Tobias B. Besemer [:BesTo] (QA)]

Attachment: memory-report_-_FF45.0a1_-_this_bug.json.gz

Comment 10

[Tobias B. Besemer [:BesTo] (QA)]

Report 5:
FF44.0a1 2015-11-27, started in offline mode, 8 windows, a lot of tabs, 10-15 pages loaded
firefox.exe: 1.265MB; plugin-container.exe: 615MB

Comment 11

[Tobias B. Besemer [:BesTo] (QA)]

Attachment: memory-report_-_FF45.0a1_-_10-15_pages.json.gz

Comment 12

[Tobias B. Besemer [:BesTo] (QA)]

Report 6:
FF45.0a1 2015-11-27, started in offline mode, 8 windows, a lot of tabs, some more pages opened/loaded, some runtime
firefox.exe: 1.355MB; plugin-container.exe: 855MB

Comment 13

[Tobias B. Besemer [:BesTo] (QA)]

Attachment: memory-report_-_FF45.0a1_-_some_more_pages.json.gz

Comment 14

[Tobias B. Besemer [:BesTo] (QA)]

Report 7:
FF45.0a1 2015-11-27, started in offline mode, 8 windows, a lot of tabs, more pages opened/loaded, some more runtime
firefox.exe: 1.380MB; plugin-container.exe: 1.165MB

Comment 15

[Tobias B. Besemer [:BesTo] (QA)]

Attachment: memory-report_-_FF45.0a1_-_more_pages.json.gz

Comment 16

[Mike Conley (:mconley) (:⚙️)]

Hey Tobias,

I suspect this might be an expected shifting of memory around from bug 1209689. Can you test this build to see if the problem is gone:

http://archive.mozilla.org/pub/firefox/tinderbox-builds/fx-team-win32/1448039310/firefox-45.0a1.en-US.win32.zip

And then test this build to see if the problem is there?:

http://archive.mozilla.org/pub/firefox/tinderbox-builds/fx-team-win32/1448052693/firefox-45.0a1.en-US.win32.zip

If so, then this is indeed from bug 1209689, and we can probably dupe this off to bug 1227275 which will (hopefully) move the memory usage back to the content process. Alternatively, we could mark this as INVALID since it's not unexpected.

Comment 17

[Tobias B. Besemer [:BesTo] (QA)]

(In reply to Mike Conley (:mconley) - Needinfo me! from comment #16)
> Hey Tobias,
> 
> I suspect this might be an expected shifting of memory around from bug
> 1209689. Can you test this build to see if the problem is gone:
> 
> http://archive.mozilla.org/pub/firefox/tinderbox-builds/fx-team-win32/
> 1448039310/firefox-45.0a1.en-US.win32.zip
> 
> And then test this build to see if the problem is there?:
> 
> http://archive.mozilla.org/pub/firefox/tinderbox-builds/fx-team-win32/
> 1448052693/firefox-45.0a1.en-US.win32.zip
> 
> If so, then this is indeed from bug 1209689, and we can probably dupe this
> off to bug 1227275 which will (hopefully) move the memory usage back to the
> content process. Alternatively, we could mark this as INVALID since it's not
> unexpected.

Hi Mike,

you don't make it really easy for me... ;-)

OK, at first: I use normally the 64-bit builds, ... (Same results with 32-bit builds ???)
At second: The old session I used to measure was completely bookmarked by me and a new session was started... (Less tabs, less Mem...)
And the 3th is: Since the update from yesterday it looks better again. So if the patch was landed at the weekend...

So, do you really need this test by me, or should be now all fine again with this prob ???


But:
I did some other measurements yesterday and today ... run FF to (my system) limits ...
I be pretty sure, not much people at Mozilla use Windows ... so I think analyzing my measures should bring some interesting results ...

-> memory-report_1.json.gz <-
FF45.0a1, Build 2015-11-29(?), 64-bit, Win7
firefox.exe: 775MB
plugin-container.exe: 1.675MB
Open Windows: 6
Open Tabs: 4, 14, 1, 1, 1, 17 (=38)

-> memory-report_2.json.gz <-
FF45.0a1, Build 2015-11-30(?), 64-bit, Win7
firefox.exe: 855MB
plugin-container.exe: 1.985MB
Open Windows: 7
Open Tabs: 7, 17, 1, 62, 1, 1, 8 (=97) (not all loaded?)

Comment 18

[Tobias B. Besemer [:BesTo] (QA)]

Attachment: memory-report_1.json.gz

Comment 19

[Tobias B. Besemer [:BesTo] (QA)]

Attachment: memory-report_2.json.gz

Comment 20

[Mike Conley (:mconley) (:⚙️)]

Glad it has improved for you.

If you happen to see the same increase in memory again, quit the browser while saving your session, please try my two builds from comment 16 to see if there's a difference. Thanks!

Comment 21

[Tobias B. Besemer [:BesTo] (QA)]

Btw.: Who is the person by Mozilla who knew after 2h all I have ever done for the projects and that I don't be usable for the QA-Job I have applied for and send me a reject ??? Seems he is a f*cking fast worker... ;-)

Comment 22

[Tobias B. Besemer [:BesTo] (QA)]

(In reply to Mike Conley (:mconley) - Needinfo me! from comment #20)
> Glad it has improved for you.
> 
> If you happen to see the same increase in memory again, quit the browser
> while saving your session, please try my two builds from comment 16 to see
> if there's a difference. Thanks!

Try the builds only if the mem goes wild again, right?

Comment 23

[Mike Conley (:mconley) (:⚙️)]

(In reply to Tobias B. Besemer [:BesTo] (QA) from comment #22)
> (In reply to Mike Conley (:mconley) - Needinfo me! from comment #20)
> > Glad it has improved for you.
> > 
> > If you happen to see the same increase in memory again, quit the browser
> > while saving your session, please try my two builds from comment 16 to see
> > if there's a difference. Thanks!
> 
> Try the builds only if the mem goes wild again, right?

Correct! Thanks!

Comment 24

[Tobias B. Besemer [:BesTo] (QA)]

Mike, maybe you can give a feedback when you had a look at the two other reports from me in the next days? Think we can after that mark this bug as "fixed"...

Comment 25

[Tobias B. Besemer [:BesTo] (QA)]

OK, seems the mem-usage was yesterday a little bit higher again... or I had a "crazy" session... Don't know! Looks ATM good again... Also Mozilla should be aware now, so it should be intended...

But I left my FF yesterday evening in offline-mode and today (~10h later) he needed ~100MB more!
Guess there should be at least one other big mem-leak...
I attach a new log from the session (Log_3).

Also, if someone at Mozilla still don't know, whats going wrong, something to read... ;-) :-P
https://en.wikipedia.org/wiki/Usage_share_of_web_browsers
http://gs.statcounter.com/
https://www.netmarketshare.com/browser-market-share.aspx?qprid=0&qpcustomd=0
https://en.wikipedia.org/wiki/Usage_share_of_operating_systems
https://www.netmarketshare.com/operating-system-market-share.aspx?qprid=10&qpcustomd=0

And it seems I spend to much time in the web... :D (... I'm from Germany.)
https://en.wikipedia.org/wiki/File:Browser_Market_Map_June_2015.svg

Comment 26

[Tobias B. Besemer [:BesTo] (QA)]

Attachment: memory-report_3.json.gz

Comment 27

[Tobias B. Besemer [:BesTo] (QA)]

Log 4
firefox.exe: 1.000MB
plugin-container.exe: 1.650MB

Comment 28

[Tobias B. Besemer [:BesTo] (QA)]

Attachment: memory-report_4.json.gz

Comment 29

[Mike Conley (:mconley) (:⚙️)]

I'm honestly not sure what to do with these memory reports. It doesn't, from the face of it, sound like an e10s-specific bug. It's tagged with MemShrink though, so hopefully one of our helpful MemShrink team members will be able to look at your memory reports and help shine a light on what might be consuming so much memory.

Comment 30

[Mike Conley (:mconley) (:⚙️)]

Are the memory reports in here pointing to anything actionable, mccr8?

Comment 31

[Andrew McCreight [:mccr8]]

Nothing really stands out in those last two memory reports. It is hard to know what is at fault when memory increases in a long-running session, particularly if you don't have a memory report from the start. It could be that a page has a leak, or it could be something in the browser.

I do see about 18mb of memory for a worker related to translation somehow ("translation/cld-worker.js") but maybe that's normal? The -4 report also has about a dozen other workers. I'm not sure what those are doing.

Comment 32

[Andrew McCreight [:mccr8]]

I'll just mark this as worksforme, as it sounds like the initial comment was about some specific thing that has been fixed in the meanwhile.