846903 - [meta] Camera - leaving the camera app open on the viewfinder causes it and the b2g parent process to run out of memory

Status: RESOLVED FIXED

(Firefox OS Graveyard :: General, defect)

Description

[Mike Habicher [:mikeh] (high bugzilla latency)]

QC is running a camera automated test, and after taking 800+ photos with the camera, the DUT shows a white screen and the automated test stops.

A couple of things stand out in the (attached) log:

02-26 16:28:03.929   125   125 E GeckoConsole: [JavaScript Error: "[Exception... "Component returned failure code: 0x80470002 (NS_BASE_STREAM_CLOSED) [nsIOutputStream.write]"  nsresult: "0x80470002 (NS_BASE_STREAM_CLOSED)"  location: "JS frame :: chrome://global/content/devtools/dbg-transport.js :: DT_onOutputStreamReady :: line 94"  data: no]" {file: "chrome://global/content/devtools/dbg-transport.js" line: 94}]

fabrice, jgriffin: I've CCed you at cjones' suggestion that you might know if this is anything to be concerned about.

Otherwise, I see this on line 3,673,703:

02-26 13:58:58.379   125   125 I Gecko   : ###!!! [Parent][AsyncChannel] Error: Channel error: cannot send/recv

After this, PID 444 (which is the camera) never appears again in the log.  Since there are no SIGSEGV or other errors logged, this looks like the camera being shut down due to OOM conditions.

Later on, a new Camera app instance is brought up with PID 1206 and then closed around line 3,832,714.

At line 3,832,805, a new camera is started with PID 13157.

Comment 1

[Mike Habicher [:mikeh] (high bugzilla latency)]

Hmm, bugzilla can't handled a 38MiB attachment.  Link to download is here:
https://docs.google.com/file/d/0B1o1jCbrNLXodnY3ZS1DcEQ3Wk0/edit?usp=sharing

Comment 2

[Jonathan Griffin (:jgriffin)]

(In reply to Mike Habicher [:mikeh] from comment #0)
> 
> 02-26 16:28:03.929   125   125 E GeckoConsole: [JavaScript Error:
> "[Exception... "Component returned failure code: 0x80470002
> (NS_BASE_STREAM_CLOSED) [nsIOutputStream.write]"  nsresult: "0x80470002
> (NS_BASE_STREAM_CLOSED)"  location: "JS frame ::
> chrome://global/content/devtools/dbg-transport.js :: DT_onOutputStreamReady
> :: line 94"  data: no]" {file:
> "chrome://global/content/devtools/dbg-transport.js" line: 94}]
> 
> fabrice, jgriffin: I've CCed you at cjones' suggestion that you might know
> if this is anything to be concerned about.
> 
This is not anything to worry about; it's a minor flaw in the remote debugger.  This exception occurs every time Marionette closes a session.

Comment 3

[Inder]

MikeH: In the log that I shared with you earlier (comment 1) the tester had restarted the camera app manually. I have asked them to share the log for a clean automation run.
As per your suggestion they are also going to share the captured images to check if there are any corruptions.

Comment 4

[David Flanagan [:djf]]

Sounds like an OOM, doesn't it?  800 is a lot of photos, so if we're leaking memory it is a pretty slow leak.  I'd suspect the code in camera/js/filmstrip.js.  We display the newest 5 or 6 thumbnails, but maybe we're not releasing the thumbnails for older ones properly.

On the other hand, 800 46x46 thumbnails only consume some 8mb of memory, so that isn't a big enough leak to cause an OOM.

Comment 5

[Andrew Overholt [:overholt]]

ni(Inder) for the info from comment 3.

Comment 6

[Inder]

Here is the latest logcat of a clean automation run when the tester didn't restart the camera app. The same issue of white screen after around 800 pics occurred again.

Comment 7

[Inder]

https://docs.google.com/file/d/0B1o1jCbrNLXoNkg0M1pHclppTjg/edit?usp=sharing

Comment 8

[Mike Habicher [:mikeh] (high bugzilla latency)]

I see this in the new log:

03-05 20:57:05.238   128   128 I Gecko   :
03-05 20:57:05.238   128   128 I Gecko   : ###!!! [Parent][AsyncChannel] Error: Channel error: cannot send/recv
03-05 20:57:05.238   128   128 I Gecko   :
03-05 20:57:05.258   128   128 I Gecko   : -*- QCContentHelper_QC_B2G: receiveMessage: 'child-process-shutdown' arrived from content process
03-05 20:57:05.258   128   128 I Gecko   : -*- QCContentHelper_QC_B2G: Unregistered mobileconnection target: [object ChromeMessageSender]

This sure looks like an OoM process kill, but we'll need the dmesg output from the device to be sure.

Comment 9

[Mike Habicher [:mikeh] (high bugzilla latency)]

(Actually, it looks like a lot of processes are being killed in the logcat.  I really _REALLY_ wish this information was available in the logcat.  Will dig more tomorrow.)

Comment 10

[Mike Habicher [:mikeh] (high bugzilla latency)]

Inder, is it possible for you to modify your test to capture logcat and dmesg output using the following command?

logcat -v time -f /dev/kmsg | cat /proc/kmsg

(This is run on the device; adb shell "..." should do the trick as well.)

Comment 11

[Andrew Overholt [:overholt]]

ni(Inder) based on comment 10

Comment 12

[Inder]

Attachment: dmesg logs

Here is the dmesg logs for the logcat shared in comment 7.

Comment 13

[Mike Habicher [:mikeh] (high bugzilla latency)]

Thanks, Inder.

<4>[55154.270844] select 7674 (Camera), adj 2, size 6215, to kill
<4>[55154.270869] send sigkill to 7674 (Camera), adj 2, size 6215

This sure looks like an LMK.  Based on bug 842820 comment 14, it looks like the camera was only holding onto ~25MiB.  That seems low to me, compared with:

20:15:49 ➜  ~  adb shell b2g-ps
APPLICATION      USER     PID   PPID  VSIZE  RSS     WCHAN    PC         NAME
b2g              root      3927  1     185264 85280 ffffffff 400d0330 S /system/b2g/b2g
Usage            app_4003  4003  3927  75412  31444 ffffffff 400e7330 S /system/b2g/plugin-container
Homescreen       app_4015  4015  3927  99616  37068 ffffffff 400aa330 S /system/b2g/plugin-container
Camera           app_4103  4103  3927  79564  31836 ffffffff 40021330 S /system/b2g/plugin-container

I'm guessing the camera is being killed because another process is soaking up available memory.

Would it be possible to periodically run b2g-ps on the DUT and capture the output?

I just tried this quickly (took about 30 photos) and when taking photos at a leisurely pace, the Camera RSS fluctuates between ~35 and 39MiB; if I take photos as fast as the UI will let me, I can push it up to ~42MiB, but once it gets there it stays there, and even starts to drop.

Comment 14

[Andrew Overholt [:overholt]]

Any progress here?

Comment 15

[Mike Habicher [:mikeh] (high bugzilla latency)]

I'm going to try to reproduce this test on our end, to see if I can reproduce the crash.  My best guess right now is this is a slow memory leak, but we'll probably need to dig to identify where it's happening.

jlebar: any advice?

Comment 16

[Justin Lebar (not reading bugmail)]

> Based on bug 842820 comment 14, it looks like the camera was only holding onto ~25MiB.

I think it says that right in the dmesg.  "size 6215" is in pages, I think.  6215 * 4kb = 24.2mb.

One question is, which process's memory usage is causing the camera app to be killed?  Is it the main process, perhaps?  We're also killing the homescreen due to low-memory.

Comment 17

[Justin Lebar (not reading bugmail)]

Anyway, I think you should run this test and run get_about_memory.py in a loop.  Then you should be able to see the process's memory usage right before the crash, and we can see which process is using so much memory, and why.

Comment 18

[Mike Habicher [:mikeh] (high bugzilla latency)]

I just ran a simple test to take 1000 photos about as fast as the camera can, and I was _unable_ to reproduce this OOM condition.  When the test completed, the camera was still running.

This was with:
- gecko inbound-src:b7f53f9b0205
- gaia: 2b594f6587dd9c4a7cca8c9cd59325c4eb2fedf1

Comment 19

[Michael Vines [:m1] [:evilmachines]]

We should test on the v1.0.1 codebase please.

Comment 20

[Mike Habicher [:mikeh] (high bugzilla latency)]

Unable to reproduce with:
- gecko b2g18_v1.0.1:36d35b08e68a
- gaia: 2b594f6587dd9c4a7cca8c9cd59325c4eb2fedf1

Will try again with BRANCH=v1.0.1 for gaia.

Comment 21

[Mike Habicher [:mikeh] (high bugzilla latency)]

Still unable to reproduce with:
- gecko: b2g18_v1.0.1:36d35b08e68a
- gaia: v1.0.1:d6cdc2a82d05a63cf5720f57428141bec66fc745

In each case I can successfully take 1000 photos in a row without problems.

If you'd like to try reproducing my test, you need to install gaia-ui-tests:
# git clone git://github.com/mozilla/gaia-ui-tests.git
# cd gaia-ui-tests
# sudo python setup.py develop
# adb forward tcp:2828 tcp:2828
# gaiatest --address localhost:2828 gaiatest/tests/camera/test_camera_capture_1000_photos.py

(I will attach the .py file to this bug as it doesn't exist in the repo (yet).)

Comment 22

[Mike Habicher [:mikeh] (high bugzilla latency)]

Attachment: script to take 1000 photos

Save as $GAIA-UI-TESTS/gaiatest/tests/camera/test_camera_capture_1000_photos.py

Comment 23

[Inder]

Attachment: Periodic b2g-ps output

As per the request in Comment 13. Here is the periodic b2g-ps output tester provided.

Comment 24

[Justin Lebar (not reading bugmail)]

Inder, we really need the info I requested in comment 17.  b2g-ps is not particularly helpful for understanding what's going on.

Comment 25

[Mike Habicher [:mikeh] (high bugzilla latency)]

b2g-ps does help give a mile-high view of where to look.  Early iterations show:

-------Iteration 13----------
APPLICATION       OOM_ADJ  OOM_SCORE  OOM_SCORE_ADJ  USER     PID   PPID  VSIZE  RSS
b2g                  0        283         0          root      128   1     196404 58836
Dogfood              6        501         400        app_410   410   128   58372  19100
Usage                6        518         400        app_427   427   128   60828  22216
Homescreen           4        401         267        app_446   446   128   66692  25172
QcSettings           6        511         400        app_520   520   128   59508  20916
Camera               2        267         134        app_951   951   128   92472  25044
(Preallocated a      6        477         400        root      2060  128   58372  20228

At iteration 643, 'QcSettings' and 'Usage' are no longer running:

-------Iteration 643----------
APPLICATION       OOM_ADJ  OOM_SCORE  OOM_SCORE_ADJ  USER     PID   PPID  VSIZE  RSS
b2g                  0        349         0          root      128   1     188184 71176
Dogfood              6        460         400        app_410   410   128   58372  11336
Homescreen           4        353         267        app_446   446   128   66692  16084
Camera               2        374         134        app_951   951   128   127944 45088
(Preallocated a      6        428         400        root      2060  128   57348  10972

Camera VSIZE/RSS has increased from 92472/25044 to 127944/45088.

At iteration 752, we lose 'Dogfood':

-------Iteration 752----------
APPLICATION       OOM_ADJ  OOM_SCORE  OOM_SCORE_ADJ  USER     PID   PPID  VSIZE  RSS
b2g                  0        368         0          root      128   1     192280 74720
Homescreen           4        351         267        app_446   446   128   66692  15864
Camera               2        403         134        app_951   951   128   133064 50628
(Preallocated a      6        427         400        root      2060  128   57348  10784

Camera VSIZE/RSS has increased to 133064/50628.

At iteration 829, we lose the preallocated process:

-------Iteration 829----------
APPLICATION       OOM_ADJ  OOM_SCORE  OOM_SCORE_ADJ  USER     PID   PPID  VSIZE  RSS
b2g                  0        386         0          root      128   1     195480 78224
Homescreen           4        351         267        app_446   446   128   66692  15836
Camera               2        425         134        app_951   951   128   137160 54668

Camera VSIZE/RSS has increased to 137160/54668.

At iteration 878, Camera VSIZE/RSS has grown to 139208/57256, and on iteration 879, 'Camera' is gone.

What's strange is that by iteration 1471, a new 'Camera' instance has launched, with VSIZE/RSS numbers comparable to those in iteration 13 (above); but the b2g process has ballooned to 236376/113112:

-------Iteration 1471----------
APPLICATION       OOM_ADJ  OOM_SCORE  OOM_SCORE_ADJ  USER     PID   PPID  VSIZE  RSS
b2g                  0        572         0          root      128   1     236376 113112
Homescreen           4        360         267        app_31861 31861 128   66692  17552
Camera               2        273         134        app_31975 31975 128   98536  26012
(Preallocated a      6        436         400        root      32081 128   57348  12324

As the test progresses, 'b2g' continues to use more memory, until 'Camera' is killed again, and there is not enough memory for even the 'Homescreen':

-------Iteration 3845----------
APPLICATION       OOM_ADJ  OOM_SCORE  OOM_SCORE_ADJ  USER     PID   PPID  VSIZE  RSS
b2g                  0        767         0          root      128   1     279860 149820

So, two questions:
1. Why does the camera memory footprint increase as it does?  (Note that I don't see this behaviour running the test attached above.)
2. Why is the b2g parent process expanding too?

Comment 26

[Inder]

(In reply to Justin Lebar [:jlebar] from comment #24)
> Inder, we really need the info I requested in comment 17.  b2g-ps is not
> particularly helpful for understanding what's going on.

Justin: sorry, missed the earlier request. I have requested test guys to run get_about_memory.py script and get back with the memory logs. Will let you guys know  once i get it.

Comment 27

[Mike Habicher [:mikeh] (high bugzilla latency)]

I'm running my test with get_about_memory.py called after every iteration.  (I'm about halfway done, and will post the results here when I get them.)

Comment 28

[Mike Habicher [:mikeh] (high bugzilla latency)]

Attachment: get_about_memory.py output over 1000 photos

This archive contains the first 10 memory reports, and the last 10.  (The entire set compresses to 65MiB, which is too big for Bugzilla--let me know if you need more detail.)

Comment 29

[Mike Habicher [:mikeh] (high bugzilla latency)]

Attachment: Python script to scrape about-memory-# reports and aggregate Camera "resident" fields

This is the script I used to generate the upcoming chart.

Comment 30

[Mike Habicher [:mikeh] (high bugzilla latency)]

Attachment: Plot of Camera RSS by picture

From here it's easy to see the gradually increasing memory requirements of the Camera app.  There's some up and down over the course of the first 200 photos, but from then on it's uphill the whole way.

jlebar, the about:memory report for the Camera shows quite a bit of "heap-unclassified" memory--would addressing bug 842604 help?

Comment 31

[Justin Lebar (not reading bugmail)]

> jlebar, the about:memory report for the Camera shows quite a bit of "heap-unclassified" 
> memory--would addressing bug 842604 help?

A DMD dump would probably be more useful, if this appears to be a heap-unclassified issue.

https://wiki.mozilla.org/Performance/MemShrink/DMD

Comment 32

[Justin Lebar (not reading bugmail)]

Looking through the memory reports, the only aberrant thing I see is the heap-unclassified.  So the next step is running DMD.

Sorry to make you run the analysis again.  I guess I should be asking for memory reports + DMD by default, given past history.

Comment 33

[Mike Habicher [:mikeh] (high bugzilla latency)]

Thanks, jlebar.  I'm (re)running the tests with DMD enabled now.

Comment 34

[Mike Habicher [:mikeh] (high bugzilla latency)]

Attachment: Sample DMD log for the camera (from photo #5)

Does this contain any useful information?

Comment 35

[Justin Lebar (not reading bugmail)]

> Does this contain any useful information?

Not really, because addr2line is not working, as you noticed.  But there also doesn't seem to be a lot of heap-unclassified there:

> Unreported:       ~1,822,060 bytes ( 19.67%) in    ~301 blocks ( 29.63%)

Nor are the b2g and camera processes taking up a ton of memory.

> #    APPLICATION        PID      Vss      Rss      Pss      Uss  cmdline
> #    b2g              21684   95444K   83384K   72231K   67480K  /system/b2g/b2g
> #    Camera           25721   56972K   38500K   27118K   22196K  /system/b2g/plugin-container

Comment 36

[Mike Habicher [:mikeh] (high bugzilla latency)]

This is only after 5 photos.  My script has another 995 to go.  :)

This is a build with B2G_DEBUG=1 set, so I'm not sure why addr2line isn't working.  Would setting NOOPT=1 help?

Comment 37

[Mike Habicher [:mikeh] (high bugzilla latency)]

Actually, I wonder if addr2line is failing because I'm running tools/get_about_memory.py from outside of the $B2G folder.

Comment 38

[Justin Lebar (not reading bugmail)]

> I wonder if addr2line is failing because I'm running tools/get_about_memory.py from outside of the 
> $B2G folder.

I don't think that's it; the script uses __file__ to figure out $B2G.

You need to pass --gecko-objdir to the script if you're not building into $B2G/objdir-gecko.  Maybe that's your problem.  We should try to detect that particular footgun somehow...

Comment 39

[Mike Habicher [:mikeh] (high bugzilla latency)]

Attachment: Plot of Camera RSS by picture, with DMD enabled

In a build with DMD enabled, the result is considerably different: the RSS footprint starts higher, but doesn't increase over the course of taking a few thousand photos.

Comment 40

[Justin Lebar (not reading bugmail)]

What about the main process's memory usage?  I'm a lot more concerned about that, because we don't restart the main process when its memory usage is too high.

Comment 41

[Mike Habicher [:mikeh] (high bugzilla latency)]

Attachment: Plot of Main Process RSS by picture, DMD enabled

Comment 42

[Justin Lebar (not reading bugmail)]

(In reply to Mike Habicher [:mikeh] from comment #41)
> Created attachment 730174 [details]
> Plot of Main Process RSS by picture, DMD enabled

That looks quite bad to me.

Comment 43

[Mike Habicher [:mikeh] (high bugzilla latency)]

Attachment: First get_about_memory report w/ DMD

The entire log set is too big to post to bugzilla; hopefully the first and last (to be posted soon) can shed some light on what's going on.

Comment 44

[Mike Habicher [:mikeh] (high bugzilla latency)]

Attachment: Last get_about_memory report w/ DMD

Last log.

Comment 45

[Mike Habicher [:mikeh] (high bugzilla latency)]

(Possibly related to bug 850893?)

Comment 46

[Justin Lebar (not reading bugmail)]

> (Possibly related to bug 850893?)

Could be; without any data there, it's pretty hard to say.

Comment 47

[Justin Lebar (not reading bugmail)]

> Main Process (pid 107)
>
> Explicit Allocations
> 116.31 MB (100.0%) -- explicit
> ├───57.53 MB (49.46%) ── heap-unclassified

That looks bad.

The heap-unclassified in the main process is all OGL memory.

We're leaking the majority of our memory (tens of mb) from mozilla::gl::GLLibraryEGL::fCreateImage.  We're also leaking a few mb from mozilla::gl::GLLibraryEGL::fCreateContext.  There's also 600kb of unreported memory from mozilla::layers::Layer::SetAnimations(), which may or may not be related.

Comment 48

[Justin Lebar (not reading bugmail)]

I wonder why the heck the camera app causes us to allocate so much memory in the main process.  The fact that we're leaking it is bad, but -- knowing nothing about how this actually works -- it seems pretty broken to me that we allocate it at all.

Comment 49

[Mike Habicher [:mikeh] (high bugzilla latency)]

Thinking out loud:

I doubt it's a leak in the ImageBridge; we'd run into that one pretty quickly just leaving the camera open with the viewfinder active if it were.  (Will verify, for sanity's sake.)

Could it be a leak in device storage?  It seems to correlate with taking (and saving) photos.  (Though that doesn't explain the OGL memory.)

Another possibility is the filmstrip possibly leaking images in the parent process.

Comment 50

[Justin Lebar (not reading bugmail)]

roc, I'm not sure who owns ShadowLayers and so on anymore.  Can you help us find the right person to loop in here?

Comment 51

[Mike Habicher [:mikeh] (high bugzilla latency)]

Attachment: Periodic DMD logs of the Camera app sitting on the viewfinder

RSS values from the command:

watch -n600 /home/mikeh/dev/mozilla/btg019/tools/get_about_memory.py --product unagi --no-auto-open --gecko-objdir /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd

(tl;dr: capture a DMD log every 10 minutes, b2g process memory seems to be inflating.)

As noted in the graph, 1 X-axis unit = DMD log = 10 minutes.

Comment 52

[Mike Habicher [:mikeh] (high bugzilla latency)]

Just to be clear, the graph attached in comment 51 is for the main b2g process, NOT the camera app process.  The camera was simply left running with the viewfinder active, with no photos being taken.

Comment 53

[Mike Habicher [:mikeh] (high bugzilla latency)]

Attachment: Initial DMD log from leaving the Camera app idling

Comment 54

[Mike Habicher [:mikeh] (high bugzilla latency)]

Attachment: Log 35 (Camera app open but idle for ~350 minutes)

Comment 55

[Justin Lebar (not reading bugmail)]

This DMD report does look pretty similar to the last one.  I have to say I'm surprised by that.

Comment 56

[Robert O'Callahan (:roc) (email my personal email if necessary)]

(In reply to Justin Lebar [:jlebar] from comment #50)
> roc, I'm not sure who owns ShadowLayers and so on anymore.  Can you help us
> find the right person to loop in here?

The two Nicks probably

Comment 57

[Mike Habicher [:mikeh] (high bugzilla latency)]

Attachment: 2.5h Camera viewfinder, b2g-ps output

Output of:

# watch -n600 ./frob.sh

Where frob.sh:

#!/bin/sh
echo >> b2gps.log
date >> b2gps.log
adb shell b2g-ps >> b2gps.log

Over the course of this test, b2g RSS rises from 87MB to 103MB.  (I ran this simple test to make sure that marionette wasn't interfering with the result.)

Comment 58

[Mike Habicher [:mikeh] (high bugzilla latency)]

Updating the bug summary to more accurately reflect observations.  Taking pictures is not required for the Camera app process and the b2g parent process to gradually increase in memory use until one or both of them are killed.

Comment 59

[Mike Habicher [:mikeh] (high bugzilla latency)]

m1, okay to un-qc-conf this bug, since it seems anyone can reproduce it?

Comment 60

[Mike Habicher [:mikeh] (high bugzilla latency)]

Attachment: 2.5h Camera viewfinder, b2g-ps output on b2g18 w/ DMD disabled

This log contains output from the script described in comment 57, but with the following differences:
- built with b2g18 instead of inbound-src
- built with MOZ_DMD=0

In this log, the RSS of both the b2g parent process and the Camera app increase over 2h40m: from 104288 to 114984, and 50204 to 57960, respectively.

Will repeat the test using b2g18 with MOZ_DMD=1.

Comment 61

[Mike Habicher [:mikeh] (high bugzilla latency)]

Attachment: 2.5h Camera viewfinder, b2g-ps output on b2g18 w/ DMD enabled

The same test with a b2g18 build as carried out in comment 60, except with MOZ_DMD=1.

With this test, RSS in the b2g parent and Camera processes also increases gradually over time.

- gecko: b2g18:f9fd0e46a631
- gaia: 1c38c91bb16f2bf0d5066c4787d2249463f61bb3

Comment 62

[Justin Lebar (not reading bugmail)]

Mike, you've now seen how to analyze a few DMD reports.  Would you mind looking at this one and seeing what you can see?

Comment 63

[Mike Habicher [:mikeh] (high bugzilla latency)]

Attachment: 2.5h Camera viewfinder, b2g-ps output on b2g18 w/ DMD enabled (run #2)

[A re-run of the test in comment 61.]

The same test with a b2g18 build as carried out in comment 60, except with MOZ_DMD=1.

With this test, RSS in the b2g parent and Camera processes also increases gradually over time (from 90 to 118, and 34 to 51MB, respectively) until the camera is killed about 4h30m later.  At that point, the b2g parent process ticks along at ~116MB indefinitely.

- gecko: b2g18:f9fd0e46a631
- gaia: 1c38c91bb16f2bf0d5066c4787d2249463f61bb3

(Will attach post-mortem DMD report next.)

Comment 64

[Mike Habicher [:mikeh] (high bugzilla latency)]

Attachment: 2.5h Camera viewfinder, post-mortem DMD report on b2g18 w/ DMD enabled (run #2)

Post-mortem DMD report (parallels attachment 733307 [details]).

Highlights:

------------------------------------------------------------------
Unreported stack trace records
------------------------------------------------------------------

Unreported: ~2,471 blocks in stack trace record 1 of 566
 ~10,113,803 bytes (~10,113,803 requested / ~0 slop)
 14.89% of the heap (14.89% cumulative);  19.93% of unreported (19.93% cumulative)
 Allocated at
   realloc /home/mikeh/dev/mozilla/m-c/inbound-src/memory/build/replace_malloc.c:190 (0x4015f376 libmozglue.so+0x4376)
   os_calloc_ext (0x42a920e4 libgsl.so+0x40e4) (no addr2line)
   eglSetYUVAttributes (0x42ddeac4 libEGL_adreno200.so+0x19ac4) (no addr2line)
   qeglDrvAPI_eglCreateImageKHR (0x42dd11f0 libEGL_adreno200.so+0xc1f0) (no addr2line)
   eglCreateImageKHR (0x42dca6f4 libEGL_adreno200.so+0x56f4) (no addr2line)
   eglCreateImageKHR /home/mikeh/dev/mozilla/btg019/frameworks/base/opengl/libs/EGL/eglApi.cpp:1276 (0x402d3fc8 libEGL.so+0xcfc8)
   mozilla::dom::HTMLAnchorElementBinding::get_hreflang(JSContext*, JS::Handle<JSObject*>, mozilla::dom::HTMLAnchorElement*, JS::Value*) /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/HTMLAnchorElementBinding.cpp:317 (0x41a9ca82 libxul.so+0x10f7a82)
   MOZ_ReportAssertionFailure /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dist/include/mozilla/Assertions.h:204 (0x41a9cf86 libxul.so+0x10f7f86)
   MOZ_ReportAssertionFailure /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dist/include/mozilla/Assertions.h:204 (0x41a73526 libxul.so+0x10ce526)
   ValueToPrimitive<unsigned int, (mozilla::dom::ConversionBehavior)0u> /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/DocumentBinding.cpp:949 (0x41a70b12 libxul.so+0x10cbb12)
   ValueToPrimitive<unsigned int, (mozilla::dom::ConversionBehavior)0u> /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/DocumentBinding.cpp:949 (0x41a70b12 libxul.so+0x10cbb12)
   WrapNewBindingNonWrapperCachedObject<mozilla::dom::TreeWalker> /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dist/include/mozilla/dom/BindingUtils.h:651 (0x41a70706 libxul.so+0x10cb706)
   ValueToPrimitive<unsigned int, (mozilla::dom::ConversionBehavior)0u> /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/DocumentBinding.cpp:949 (0x41a70b12 libxul.so+0x10cbb12)
   ValueToPrimitive<unsigned int, (mozilla::dom::ConversionBehavior)0u> /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/DocumentBinding.cpp:949 (0x41a70b12 libxul.so+0x10cbb12)
   ValueToPrimitive<unsigned int, (mozilla::dom::ConversionBehavior)0u> /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/DocumentBinding.cpp:949 (0x41a70b12 libxul.so+0x10cbb12)
   ValueToPrimitive<unsigned int, (mozilla::dom::ConversionBehavior)0u> /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/DocumentBinding.cpp:949 (0x41a70b12 libxul.so+0x10cbb12)
   JSVAL_TO_OBJECT /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dist/include/js/Value.h:1702 (0x41a769ea libxul.so+0x10d19ea)
   SetReservedSlot /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dist/include/jsfriendapi.h:505 (0x41a77060 libxul.so+0x10d2060)
   GetConstructorObject /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/DocumentBinding.h:96 (0x41a74520 libxul.so+0x10cf520)
   mozilla::dom::ElementBinding::getElementsByTagNameNS(JSContext*, JS::Handle<JSObject*>, mozilla::dom::Element*, unsigned int, JS::Value*) /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/ElementBinding.cpp:552 (0x41a892a2 libxul.so+0x10e42a2)
   nsCOMPtr<nsIFileOutputStream>::Assert_NoQueryNeeded() /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dist/include/nsCOMPtr.h:505 (0x40fb85c0 libxul.so+0xe375c0)
   mozilla::net::PFTPChannelChild::SendResume() /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/ipc/ipdl/PFTPChannelChild.cpp:222 (0x411dfa56 libxul.so+0x105ea56)
   operator delete /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dist/include/mozilla/mozalloc.h:225 (0x411e0280 libxul.so+0x105f280)
   mozilla::net::PHttpChannelChild::Write(mozilla::ipc::StandardURLParams const&, IPC::Message*) /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/ipc/ipdl/PHttpChannelChild.cpp:1181 (0x411e0fd6 libxul.so+0x105ffd6)

Unreported: ~2,458 blocks in stack trace record 2 of 566
 ~10,060,594 bytes (~10,060,594 requested / ~0 slop)
 14.81% of the heap (29.70% cumulative);  19.83% of unreported (39.76% cumulative)
 Allocated at
   posix_memalign /home/mikeh/dev/mozilla/m-c/inbound-src/memory/build/replace_malloc.c:160 (0x4015f42c libmozglue.so+0x442c)
   os_malloc_ext (0x42a92134 libgsl.so+0x4134) (no addr2line)
   qeglDrvAPI_eglCreateImageKHR (0x42dd0d10 libEGL_adreno200.so+0xbd10) (no addr2line)
   eglCreateImageKHR (0x42dca6f4 libEGL_adreno200.so+0x56f4) (no addr2line)
   eglCreateImageKHR /home/mikeh/dev/mozilla/btg019/frameworks/base/opengl/libs/EGL/eglApi.cpp:1276 (0x402d3fc8 libEGL.so+0xcfc8)
   mozilla::dom::HTMLAnchorElementBinding::get_hreflang(JSContext*, JS::Handle<JSObject*>, mozilla::dom::HTMLAnchorElement*, JS::Value*) /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/HTMLAnchorElementBinding.cpp:317 (0x41a9ca82 libxul.so+0x10f7a82)
   MOZ_ReportAssertionFailure /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dist/include/mozilla/Assertions.h:204 (0x41a9cf86 libxul.so+0x10f7f86)
   MOZ_ReportAssertionFailure /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dist/include/mozilla/Assertions.h:204 (0x41a73526 libxul.so+0x10ce526)
   ValueToPrimitive<unsigned int, (mozilla::dom::ConversionBehavior)0u> /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/DocumentBinding.cpp:949 (0x41a70b12 libxul.so+0x10cbb12)
   ValueToPrimitive<unsigned int, (mozilla::dom::ConversionBehavior)0u> /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/DocumentBinding.cpp:949 (0x41a70b12 libxul.so+0x10cbb12)
   WrapNewBindingNonWrapperCachedObject<mozilla::dom::TreeWalker> /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dist/include/mozilla/dom/BindingUtils.h:651 (0x41a70706 libxul.so+0x10cb706)
   ValueToPrimitive<unsigned int, (mozilla::dom::ConversionBehavior)0u> /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/DocumentBinding.cpp:949 (0x41a70b12 libxul.so+0x10cbb12)
   ValueToPrimitive<unsigned int, (mozilla::dom::ConversionBehavior)0u> /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/DocumentBinding.cpp:949 (0x41a70b12 libxul.so+0x10cbb12)
   ValueToPrimitive<unsigned int, (mozilla::dom::ConversionBehavior)0u> /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/DocumentBinding.cpp:949 (0x41a70b12 libxul.so+0x10cbb12)
   ValueToPrimitive<unsigned int, (mozilla::dom::ConversionBehavior)0u> /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/DocumentBinding.cpp:949 (0x41a70b12 libxul.so+0x10cbb12)
   JSVAL_TO_OBJECT /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dist/include/js/Value.h:1702 (0x41a769ea libxul.so+0x10d19ea)
   SetReservedSlot /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dist/include/jsfriendapi.h:505 (0x41a77060 libxul.so+0x10d2060)
   GetConstructorObject /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/DocumentBinding.h:96 (0x41a74520 libxul.so+0x10cf520)
   mozilla::dom::ElementBinding::getElementsByTagNameNS(JSContext*, JS::Handle<JSObject*>, mozilla::dom::Element*, unsigned int, JS::Value*) /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/ElementBinding.cpp:552 (0x41a892a2 libxul.so+0x10e42a2)
   nsCOMPtr<nsIFileOutputStream>::Assert_NoQueryNeeded() /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dist/include/nsCOMPtr.h:505 (0x40fb85c0 libxul.so+0xe375c0)
   mozilla::net::PFTPChannelChild::SendResume() /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/ipc/ipdl/PFTPChannelChild.cpp:222 (0x411dfa56 libxul.so+0x105ea56)
   operator delete /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dist/include/mozilla/mozalloc.h:225 (0x411e0280 libxul.so+0x105f280)
   mozilla::net::PHttpChannelChild::Write(mozilla::ipc::StandardURLParams const&, IPC::Message*) /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/ipc/ipdl/PHttpChannelChild.cpp:1181 (0x411e0fd6 libxul.so+0x105ffd6)
   mozilla::net::PHttpChannelChild::Read(mozilla::dom::PBlobChild**, IPC::Message const*, void**, bool) /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/ipc/ipdl/PHttpChannelChild.cpp:1810 (0x41a05352 libxul.so+0x1060352)

Unreported: ~2,448 blocks in stack trace record 3 of 566
 ~10,019,664 bytes (~10,019,664 requested / ~0 slop)
 14.75% of the heap (44.46% cumulative);  19.75% of unreported (59.51% cumulative)
 Allocated at
   realloc /home/mikeh/dev/mozilla/m-c/inbound-src/memory/build/replace_malloc.c:190 (0x4015f376 libmozglue.so+0x4376)
   os_calloc_ext (0x42a920e4 libgsl.so+0x40e4) (no addr2line)
   eglSetYUVAttributes (0x42ddeab4 libEGL_adreno200.so+0x19ab4) (no addr2line)
   qeglDrvAPI_eglCreateImageKHR (0x42dd11f0 libEGL_adreno200.so+0xc1f0) (no addr2line)
   eglCreateImageKHR (0x42dca6f4 libEGL_adreno200.so+0x56f4) (no addr2line)
   eglCreateImageKHR /home/mikeh/dev/mozilla/btg019/frameworks/base/opengl/libs/EGL/eglApi.cpp:1276 (0x402d3fc8 libEGL.so+0xcfc8)
   mozilla::dom::HTMLAnchorElementBinding::get_hreflang(JSContext*, JS::Handle<JSObject*>, mozilla::dom::HTMLAnchorElement*, JS::Value*) /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/HTMLAnchorElementBinding.cpp:317 (0x41a9ca82 libxul.so+0x10f7a82)
   MOZ_ReportAssertionFailure /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dist/include/mozilla/Assertions.h:204 (0x41a9cf86 libxul.so+0x10f7f86)
   MOZ_ReportAssertionFailure /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dist/include/mozilla/Assertions.h:204 (0x41a73526 libxul.so+0x10ce526)
   ValueToPrimitive<unsigned int, (mozilla::dom::ConversionBehavior)0u> /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/DocumentBinding.cpp:949 (0x41a70b12 libxul.so+0x10cbb12)
   ValueToPrimitive<unsigned int, (mozilla::dom::ConversionBehavior)0u> /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/DocumentBinding.cpp:949 (0x41a70b12 libxul.so+0x10cbb12)
   WrapNewBindingNonWrapperCachedObject<mozilla::dom::TreeWalker> /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dist/include/mozilla/dom/BindingUtils.h:651 (0x41a70706 libxul.so+0x10cb706)
   ValueToPrimitive<unsigned int, (mozilla::dom::ConversionBehavior)0u> /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/DocumentBinding.cpp:949 (0x41a70b12 libxul.so+0x10cbb12)
   ValueToPrimitive<unsigned int, (mozilla::dom::ConversionBehavior)0u> /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/DocumentBinding.cpp:949 (0x41a70b12 libxul.so+0x10cbb12)
   ValueToPrimitive<unsigned int, (mozilla::dom::ConversionBehavior)0u> /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/DocumentBinding.cpp:949 (0x41a70b12 libxul.so+0x10cbb12)
   ValueToPrimitive<unsigned int, (mozilla::dom::ConversionBehavior)0u> /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/DocumentBinding.cpp:949 (0x41a70b12 libxul.so+0x10cbb12)
   JSVAL_TO_OBJECT /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dist/include/js/Value.h:1702 (0x41a769ea libxul.so+0x10d19ea)
   SetReservedSlot /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dist/include/jsfriendapi.h:505 (0x41a77060 libxul.so+0x10d2060)
   GetConstructorObject /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/DocumentBinding.h:96 (0x41a74520 libxul.so+0x10cf520)
   mozilla::dom::ElementBinding::getElementsByTagNameNS(JSContext*, JS::Handle<JSObject*>, mozilla::dom::Element*, unsigned int, JS::Value*) /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/ElementBinding.cpp:552 (0x41a892a2 libxul.so+0x10e42a2)
   nsCOMPtr<nsIFileOutputStream>::Assert_NoQueryNeeded() /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dist/include/nsCOMPtr.h:505 (0x40fb85c0 libxul.so+0xe375c0)
   mozilla::net::PFTPChannelChild::SendResume() /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/ipc/ipdl/PFTPChannelChild.cpp:222 (0x411dfa56 libxul.so+0x105ea56)
   operator delete /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dist/include/mozilla/mozalloc.h:225 (0x411e0280 libxul.so+0x105f280)
   mozilla::net::PHttpChannelChild::Write(mozilla::ipc::StandardURLParams const&, IPC::Message*) /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/ipc/ipdl/PHttpChannelChild.cpp:1181 (0x411e0fd6 libxul.so+0x105ffd6)

Unreported: ~2,402 blocks in stack trace record 4 of 566
 ~9,831,386 bytes (~9,831,386 requested / ~0 slop)
 14.48% of the heap (58.93% cumulative);  19.38% of unreported (78.89% cumulative)
 Allocated at
   realloc /home/mikeh/dev/mozilla/m-c/inbound-src/memory/build/replace_malloc.c:190 (0x4015f376 libmozglue.so+0x4376)
   os_calloc_ext (0x42a920e4 libgsl.so+0x40e4) (no addr2line)
   eglSetYUVAttributes (0x42ddeaa4 libEGL_adreno200.so+0x19aa4) (no addr2line)
   qeglDrvAPI_eglCreateImageKHR (0x42dd11f0 libEGL_adreno200.so+0xc1f0) (no addr2line)
   eglCreateImageKHR (0x42dca6f4 libEGL_adreno200.so+0x56f4) (no addr2line)
   eglCreateImageKHR /home/mikeh/dev/mozilla/btg019/frameworks/base/opengl/libs/EGL/eglApi.cpp:1276 (0x402d3fc8 libEGL.so+0xcfc8)
   mozilla::dom::HTMLAnchorElementBinding::get_hreflang(JSContext*, JS::Handle<JSObject*>, mozilla::dom::HTMLAnchorElement*, JS::Value*) /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/HTMLAnchorElementBinding.cpp:317 (0x41a9ca82 libxul.so+0x10f7a82)
   MOZ_ReportAssertionFailure /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dist/include/mozilla/Assertions.h:204 (0x41a9cf86 libxul.so+0x10f7f86)
   MOZ_ReportAssertionFailure /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dist/include/mozilla/Assertions.h:204 (0x41a73526 libxul.so+0x10ce526)
   ValueToPrimitive<unsigned int, (mozilla::dom::ConversionBehavior)0u> /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/DocumentBinding.cpp:949 (0x41a70b12 libxul.so+0x10cbb12)
   ValueToPrimitive<unsigned int, (mozilla::dom::ConversionBehavior)0u> /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/DocumentBinding.cpp:949 (0x41a70b12 libxul.so+0x10cbb12)
   WrapNewBindingNonWrapperCachedObject<mozilla::dom::TreeWalker> /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dist/include/mozilla/dom/BindingUtils.h:651 (0x41a70706 libxul.so+0x10cb706)
   ValueToPrimitive<unsigned int, (mozilla::dom::ConversionBehavior)0u> /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/DocumentBinding.cpp:949 (0x41a70b12 libxul.so+0x10cbb12)
   ValueToPrimitive<unsigned int, (mozilla::dom::ConversionBehavior)0u> /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/DocumentBinding.cpp:949 (0x41a70b12 libxul.so+0x10cbb12)
   ValueToPrimitive<unsigned int, (mozilla::dom::ConversionBehavior)0u> /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/DocumentBinding.cpp:949 (0x41a70b12 libxul.so+0x10cbb12)
   ValueToPrimitive<unsigned int, (mozilla::dom::ConversionBehavior)0u> /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/DocumentBinding.cpp:949 (0x41a70b12 libxul.so+0x10cbb12)
   JSVAL_TO_OBJECT /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dist/include/js/Value.h:1702 (0x41a769ea libxul.so+0x10d19ea)
   SetReservedSlot /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dist/include/jsfriendapi.h:505 (0x41a77060 libxul.so+0x10d2060)
   GetConstructorObject /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/DocumentBinding.h:96 (0x41a74520 libxul.so+0x10cf520)
   mozilla::dom::ElementBinding::getElementsByTagNameNS(JSContext*, JS::Handle<JSObject*>, mozilla::dom::Element*, unsigned int, JS::Value*) /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dom/bindings/ElementBinding.cpp:552 (0x41a892a2 libxul.so+0x10e42a2)
   nsCOMPtr<nsIFileOutputStream>::Assert_NoQueryNeeded() /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dist/include/nsCOMPtr.h:505 (0x40fb85c0 libxul.so+0xe375c0)
   mozilla::net::PFTPChannelChild::SendResume() /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/ipc/ipdl/PFTPChannelChild.cpp:222 (0x411dfa56 libxul.so+0x105ea56)
   operator delete /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/dist/include/mozilla/mozalloc.h:225 (0x411e0280 libxul.so+0x105f280)
   mozilla::net::PHttpChannelChild::Write(mozilla::ipc::StandardURLParams const&, IPC::Message*) /home/mikeh/dev/mozilla/btg019/objdir-gecko-inbound-debug-dmd/ipc/ipdl/PHttpChannelChild.cpp:1181 (0x411e0fd6 libxul.so+0x105ffd6)

Comment 65

[Nicolas Silva [:nical]]

You should try the patch attached to bug 856080

It fixes a memory leak that happens on pages that have css animations with OMTC + animate-transform prefed on.

I haven't tried with many css animation demos, but on this one https://developer.mozilla.org/fr/demos/detail/rofox-css3-animation-by-anthony-calzadilla/launch we keep leaking continuously after the animation is over.

Comment 66

[Justin Lebar (not reading bugmail)]

I'd really appreciate if we could get a gfx person to look at this bug.  It appears to be pretty serious, and I don't think it's getting attention commensurate with that.

Comment 67

[Milan Sreckovic [:milan] (needinfo for best results)]

Justin, has somebody tried the patch from Comment 65?

Comment 68

[Justin Lebar (not reading bugmail)]

As I explained in bug 856080 (sorry, I should have commented here, too), it seems extremely unlikely to me that that change would fix the OpenGL leaks that are giving us trouble here.  The leaked object in that bug holds no references that I can see, so I don't think cleaning it up could clean up the OGL objects.

Comment 69

[Milan Sreckovic [:milan] (needinfo for best results)]

Got it - missed that comment in bug 856080.  Jeff, Nick, any bandwidth this week before we land bug 852734?

Comment 70

[Mike Habicher [:mikeh] (high bugzilla latency)]

Just confirmed that on a build with the bug 856080 patch included, b2g parent and camera app RSS continue to increase as before.

Comment 71

[Mike Habicher [:mikeh] (high bugzilla latency)]

Attachment: b2g18-dmd get_about_memory logs

Obsoleting all of the other logs since this one shows much more _sane_ stack traces.

From the camera:
Unreported: ~2,541 blocks in stack trace record 1 of 218
 ~10,400,313 bytes (~10,400,313 requested / ~0 slop)
 64.99% of the heap (64.99% cumulative);  87.68% of unreported (87.68% cumulative)
 Allocated at
   malloc /home/mikeh/dev/mozilla/m-c/b2g18/memory/build/replace_malloc.c:152
   moz_xmalloc /home/mikeh/dev/mozilla/m-c/b2g18/memory/mozalloc/mozalloc.cpp:55
   mozilla::layers::ImageContainerChild::CreateSharedImageFromData(mozilla::layers::Image*) /home/mikeh/dev/mozilla/m-c/b2g18/gfx/layers/ipc/ImageContainerChild.cpp:234
   mozilla::layers::ImageContainerChild::ImageToSharedImage(mozilla::layers::Image*) /home/mikeh/dev/mozilla/m-c/b2g18/gfx/layers/ipc/ImageContainerChild.cpp:356
   mozilla::layers::ImageBridgeCopyAndSendTask::Run() /home/mikeh/dev/mozilla/m-c/b2g18/gfx/layers/ipc/ImageContainerChild.cpp:328
   MessageLoop::RunTask(Task*) /home/mikeh/dev/mozilla/m-c/b2g18/ipc/chromium/src/base/message_loop.cc:335
   MessageLoop::DeferOrRunPendingTask(MessageLoop::PendingTask const&) /home/mikeh/dev/mozilla/m-c/b2g18/ipc/chromium/src/base/message_loop.cc:345
   MessageLoop::DoWork() /home/mikeh/dev/mozilla/m-c/b2g18/ipc/chromium/src/base/message_loop.cc:442
   base::MessagePumpDefault::Run(base::MessagePump::Delegate*) /home/mikeh/dev/mozilla/m-c/b2g18/ipc/chromium/src/base/message_pump_default.cc:24
   MessageLoop::RunInternal() /home/mikeh/dev/mozilla/m-c/b2g18/ipc/chromium/src/base/message_loop.cc:217
   ~AutoRunState /home/mikeh/dev/mozilla/m-c/b2g18/ipc/chromium/src/base/message_loop.cc:503
   base::Thread::ThreadMain() /home/mikeh/dev/mozilla/m-c/b2g18/ipc/chromium/src/base/thread.cc:159
   ThreadFunc /home/mikeh/dev/mozilla/m-c/b2g18/ipc/chromium/src/base/platform_thread_posix.cc:41
   __thread_entry /home/mikeh/dev/mozilla/btg019/bionic/libc/bionic/pthread.c:218
   pthread_create /home/mikeh/dev/mozilla/btg019/bionic/libc/bionic/pthread.c:362

From the b2g parent process:
Unreported: ~691 blocks in stack trace record 1 of 544
 ~2,828,263 bytes (~2,828,263 requested / ~0 slop)
 7.93% of the heap (7.93% cumulative);  13.24% of unreported (13.24% cumulative)
 Allocated at
   calloc /home/mikeh/dev/mozilla/m-c/b2g18/memory/build/replace_malloc.c:182
   os_calloc_ext (0x42a920e4 libgsl.so+0x40e4) (no addr2line)
   eglSetYUVAttributes (0x42ddeac4 libEGL_adreno200.so+0x19ac4) (no addr2line)
   qeglDrvAPI_eglCreateImageKHR (0x42dd11f0 libEGL_adreno200.so+0xc1f0) (no addr2line)
   eglCreateImageKHR (0x42dca6f4 libEGL_adreno200.so+0x56f4) (no addr2line)
   eglCreateImageKHR /home/mikeh/dev/mozilla/btg019/frameworks/base/opengl/libs/EGL/eglApi.cpp:1276
   mozilla::gl::GLLibraryEGL::fCreateImage(void*, void*, unsigned int, void*, int const*) /home/mikeh/dev/mozilla/m-c/b2g18/gfx/gl/GLLibraryEGL.h:346
   mozilla::gl::GLContextEGL::BindExternalBuffer(unsigned int, void*) /home/mikeh/dev/mozilla/m-c/b2g18/gfx/gl/GLContextProviderEGL.cpp:439
   mozilla::layers::ShadowImageLayerOGL::RenderLayer(int, nsIntPoint const&) /home/mikeh/dev/mozilla/m-c/b2g18/gfx/layers/opengl/ImageLayerOGL.cpp:969
   ContainerRender<mozilla::layers::ShadowContainerLayerOGL> /home/mikeh/dev/mozilla/m-c/b2g18/gfx/layers/opengl/ContainerLayerOGL.cpp:264
   ContainerRender<mozilla::layers::ShadowContainerLayerOGL> /home/mikeh/dev/mozilla/m-c/b2g18/gfx/layers/opengl/ContainerLayerOGL.cpp:264
   ContainerRender<mozilla::layers::ShadowRefLayerOGL> /home/mikeh/dev/mozilla/m-c/b2g18/gfx/layers/opengl/ContainerLayerOGL.cpp:264
   ContainerRender<mozilla::layers::ShadowContainerLayerOGL> /home/mikeh/dev/mozilla/m-c/b2g18/gfx/layers/opengl/ContainerLayerOGL.cpp:264
   ContainerRender<mozilla::layers::ShadowContainerLayerOGL> /home/mikeh/dev/mozilla/m-c/b2g18/gfx/layers/opengl/ContainerLayerOGL.cpp:264
   ContainerRender<mozilla::layers::ShadowContainerLayerOGL> /home/mikeh/dev/mozilla/m-c/b2g18/gfx/layers/opengl/ContainerLayerOGL.cpp:264
   ContainerRender<mozilla::layers::ShadowContainerLayerOGL> /home/mikeh/dev/mozilla/m-c/b2g18/gfx/layers/opengl/ContainerLayerOGL.cpp:264
   mozilla::layers::LayerManagerOGL::Render() /home/mikeh/dev/mozilla/m-c/b2g18/gfx/layers/opengl/LayerManagerOGL.cpp:1031
   mozilla::layers::LayerManagerOGL::EndTransaction(void (*)(mozilla::layers::ThebesLayer*, gfxContext*, nsIntRegion const&, nsIntRegion const&, void*), void*, mozilla::layers::LayerManager::EndTransactionFlags) /home/mikeh/dev/mozilla/m-c/b2g18/gfx/layers/opengl/LayerManagerOGL.cpp:700
   mozilla::layers::LayerManagerOGL::EndEmptyTransaction(mozilla::layers::LayerManager::EndTransactionFlags) /home/mikeh/dev/mozilla/m-c/b2g18/gfx/layers/opengl/LayerManagerOGL.cpp:642
   ~AutoResolveRefLayers /home/mikeh/dev/mozilla/m-c/b2g18/gfx/layers/ipc/CompositorParent.cpp:458
   RunnableMethod<mozilla::layers::ImageContainerChild, void (mozilla::layers::ImageContainerChild::*)(), Tuple0>::Run() /home/mikeh/dev/mozilla/m-c/b2g18/ipc/chromium/src/base/task.h:308
   MessageLoop::RunTask(Task*) /home/mikeh/dev/mozilla/m-c/b2g18/ipc/chromium/src/base/message_loop.cc:335
   MessageLoop::DeferOrRunPendingTask(MessageLoop::PendingTask const&) /home/mikeh/dev/mozilla/m-c/b2g18/ipc/chromium/src/base/message_loop.cc:345
   MessageLoop::DoWork() /home/mikeh/dev/mozilla/m-c/b2g18/ipc/chromium/src/base/message_loop.cc:442

Comment 72

[Mike Habicher [:mikeh] (high bugzilla latency)]

One thing I noticed from the new logs: the call to ImageContainerChild::CreateSharedImageFromData() hits line 234[1] which is the handler for GRALLOC_PLANAR_YCBCR.  Assuming this is the viewfinder preview, I'm pretty sure this should be using GONK_IO_SURFACE.

1. http://hg.mozilla.org/releases/mozilla-b2g18/file/d2d8ae22d88a/gfx/layers/ipc/ImageContainerChild.cpp#l234

Comment 73

[Mike Habicher [:mikeh] (high bugzilla latency)]

Something else I noticed in ImageContainerChild.cpp:

In ImageContainerChild::SendImageAsync():
  if (InImageBridgeChildThread()) {
    SharedImage *img = ImageToSharedImage(aImage);
    if (img) {
      SendPublishImage(*img);
    } else {
      NS_WARNING("Failed to create a shared image!");
    }
    delete img;
    return;
  }

If we're not InImageBridgeChildThread(), we dispatch a ImageBridgeCopyAndSendTask() to the image bridge child thread, and Run():
  void Run()
  { 
    SharedImage* img = mChild->ImageToSharedImage(mImage.get());
    if (img) {
      mChild->SendPublishImage(*img);
    }
  }

Should there be a 'delete img' statement here as well?

Comment 74

[Sotaro Ikeda [:sotaro]]

(In reply to Mike Habicher [:mikeh] from comment #73)
> Something else I noticed in ImageContainerChild.cpp:
> 
> In ImageContainerChild::SendImageAsync():
>   if (InImageBridgeChildThread()) {
>     SharedImage *img = ImageToSharedImage(aImage);
>     if (img) {
>       SendPublishImage(*img);
>     } else {
>       NS_WARNING("Failed to create a shared image!");
>     }
>     delete img;
>     return;
>   }
> 
> If we're not InImageBridgeChildThread(), we dispatch a
> ImageBridgeCopyAndSendTask() to the image bridge child thread, and Run():
>   void Run()
>   { 
>     SharedImage* img = mChild->ImageToSharedImage(mImage.get());
>     if (img) {
>       mChild->SendPublishImage(*img);
>     }
>   }
> 
> Should there be a 'delete img' statement here as well?

:nical, can you comment to it?

Comment 75

[Mike Habicher [:mikeh] (high bugzilla latency)]

(In reply to Sotaro Ikeda [:sotaro] from comment #74)
>
> :nical, can you comment to it?

I added the missing 'delete img' and the memory leak seems to be gone (and the camera still works).

For what it's worth, ImageContainerChild.cpp doesn't even exist anymore in the latest version of mozilla-inbound.

Comment 76

[Mike Habicher [:mikeh] (high bugzilla latency)]

Attachment: delete SharedImage object in ImageBridgeCopyAndSendTask::Run() to prevent leak

This patch is b2g18-specific.  As of FIREFOX_AURORA_19_BASE[1], this code was re-factored into SendImageNow(), which always calls 'delete img'[2].  (And as of ee5ca214e87c[3] the ImageContainerChild.cpp file doesn't exist anymore.)

1. http://hg.mozilla.org/mozilla-central/file/cf8750abee06/gfx/layers/ipc/ImageContainerChild.cpp#l341
2. http://hg.mozilla.org/mozilla-central/file/cf8750abee06/gfx/layers/ipc/ImageContainerChild.cpp#l327
3. http://hg.mozilla.org/mozilla-central/rev/ee5ca214e87c

Inder, please retry your stress test with this patch.

Comment 77

[Inder]

MikeH: Sure, let me share it will the test team to run their stress tests.

Comment 78

[Mike Habicher [:mikeh] (high bugzilla latency)]

Inder, my apologies--I may have spoken too soon.  The memory leak in the Camera app is gone, but it looks like the b2g parent process RSS is still creeping up, albeit _very_ slowly.  Your test team shouldn't run into any new problems if they apply the above patch, but it might not solve this issue.  I'll keep you posted.

Comment 79

[Justin Lebar (not reading bugmail)]

> The memory leak in the Camera app is gone, but it looks like the b2g parent process RSS is still 
> creeping up, albeit _very_ slowly.

Would you mind splitting this patch off into a new bug that blocks this one, then?

We've had a lot of confusion in the past when we tried to use a single bug to track two separate fixes.  Worst case, this slow leak you're seeing in the main process isn't actually there, and we created an extra bug for nothing.

Comment 80

[Nicolas Silva [:nical]]

Comment on attachment 735850 [details] [diff] [review]
delete SharedImage object in ImageBridgeCopyAndSendTask::Run() to prevent leak

Review of attachment 735850 [details] [diff] [review]:
-----------------------------------------------------------------

This is indeed a leak. mozilla-central is not affected, since this code has been rewritten as part of the layers refactoring.

Comment 81

[Mike Habicher [:mikeh] (high bugzilla latency)]

Comment on attachment 735850 [details] [diff] [review]
delete SharedImage object in ImageBridgeCopyAndSendTask::Run() to prevent leak

Marking obselete--patch moved to specific bug 860395.  Will carry r+ over to the copy of the patch there.

Comment 82

[Nicolas Silva [:nical]]

(In reply to Mike Habicher [:mikeh] from comment #75)
> For what it's worth, ImageContainerChild.cpp doesn't even exist anymore in
> the latest version of mozilla-inbound.

The equivalent of ImageContainerChild in mozilla-central would be ImageClient (except that ImageClient is used for both async and in-transaction texture transfer), and the corresponding method would be ImageClient::UpdateImage in which we place the shared image into a stack allocated SurfaceDescriptor rather than creating one on the heap and forgetting to delete it (see SharedPlanarYCbCrImage::ToSurfaceDescriptor).

Comment 83

[Nicholas Nethercote [inactive]]

Can someone clarify the status?  Bug 860395 was fixed, does that fix this?  Is there anything more to do here?

Comment 84

[Mike Habicher [:mikeh] (high bugzilla latency)]

Nicholas, bug 860395 addressed a leak on the client side (in the Camera app process); bug 860483 contains a patch to resolve a leak on the parent side, but at this point, it looks like we may also need an additional blocked-on bug to address an issue in the 3rd-party graphics library.

Comment 85

[Mike Habicher [:mikeh] (high bugzilla latency)]

With bug 860395 fixed, the graphics driver problem only exists on Unagi; unable to reproduce on Inari.

Comment 86

[Cody Roesch [:croesch]]

Marking as QARegressExclude. No access to Automation.