memory leak when running a test on repeat

RESOLVED INVALID

Status

Firefox OS
General
RESOLVED INVALID
5 years ago
4 years ago

People

(Reporter: mdas, Unassigned)

Tracking

unspecified
Other
Gonk (Firefox OS)
Dependency tree / graph

Firefox Tracking Flags

(blocking-b2g:-, b2g18+)

Details

(Whiteboard: [MemShrink:P2])

Attachments

(3 attachments)

Created attachment 713693 [details]
about:memory at startup

We've been seeing a lot of weird issues in our CI lately related to Bug 838786, and one of them looked like a memory issue since we've been seeing a lot of:

"E/memalloc(  109): /dev/pmem: No more pmem available"

in the logs, and usually right before our error.

I ran a single one of these tests on repeat to see if it reproduced our issue. It did, and I can see that the memory consumption increased after the suite was ran, despite killing the app it launches. If I run ./get_about_memory.py --minimize, a little of the memory is freed up.

The test (https://github.com/mozilla/gaia-ui-tests/blob/master/gaiatest/tests/test_wallpaper.py) kills all running apps, runs the settings app, sets a new wallpaper, sets it back to the default wallpaper, and quits. If I run it at most 70 times on a freshly booted phone, it will eventually hit a socket.error, and the memory consumption is really high.

I have the about:memory dumps from after startup, and from the point our test fails. I noticed that the b2g process' VSIZE grew by 50%, even after terminating the test and closing the settings app manually. 

It's not clear to me *where* the memory leak is. I see b2g's process' VSIZE is huge, and I see in the logs that the Main Process' memory at least doubles. Can anyone give some insight?

For STR, plug in your unagi, and, assuming you have python virtualenv installed on your machine:
git clone git@github.com:mozilla/gaia-ui-tests.git
cd gaia-ui-tests
virtualenv venv
. venv/bin/activate
pip install marionette_client
python setup.py develop
adb forward tcp:2828 tcp:2828
gaiatest --address localhost:2828 --repeat=70 gaiatest/tests/test_wallpaper.py
deactivate

After this runs, check the about:memory usage.
Created attachment 713694 [details]
about:memory after failure occurs and after killing settings app
Created attachment 713696 [details]
about:memory after --minimize is run
blocking-b2g: --- → shira?

Comment 3

5 years ago
Memory leak after running an uncommon user scenario 70 times back to back, not a blocker.
blocking-b2g: shira? → -
tracking-b2g18: --- → +
Hey jlebar, can we borrow your eyes to spot anything odd in the logs?
Flags: needinfo?(justin.lebar+bug)
RSS, which is the most meaningful measure of size, goes from 52.02mb to 56.56mb (after --minimize) in the main process.  The homescreen goes from 27.88mb to 21.29mb.

So neither of these seems to be the problem.

> "E/memalloc(  109): /dev/pmem: No more pmem available"

pmem isn't "real" memory; it's a type of graphics memory, I think.  Perhaps we're leaking that, and perhaps it's allocated in a way which doesn't show up here.
Flags: needinfo?(justin.lebar+bug)
(In reply to Justin Lebar [:jlebar] from comment #5)
> RSS, which is the most meaningful measure of size, goes from 52.02mb to
> 56.56mb (after --minimize) in the main process.  The homescreen goes from
> 27.88mb to 21.29mb.
> 
> So neither of these seems to be the problem.
> 
> > "E/memalloc(  109): /dev/pmem: No more pmem available"
> 
> pmem isn't "real" memory; it's a type of graphics memory, I think.  Perhaps
> we're leaking that, and perhaps it's allocated in a way which doesn't show
> up here.

Ah, thanks. One thing I forgot to comment about was that when we hit the error point, the UI freezes, and it always freezes after we launch the app and the app is in view but with the dark overlay, ie: it freezesright before it transitions to being a usable app. You can't use anything on the screen, but, if you press the Home button, you will be able to access the Homescreen and use the phone again. Not sure if this helps with the issue.
Important update to the STR:
Replace:

git clone git@github.com:mozilla/gaia-ui-tests.git

with:

git clone git://github.com/malini/gaia-ui-tests-1.git gaia-ui-tests
git checkout remove_sp

The mozilla/gaia-ui-tests are using SpecialPowers which could add extra memory leakage. The branch I have in my github repo removes the usage of SpecialPowers. I generated the about memory logs using the tests I have in my repository.
Whiteboard: [MemShrink] → [MemShrink:P2]
Does this still happen?
No, this is no longer reproducible.
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → INVALID
You need to log in before you can comment on or make changes to this bug.