~1.5mb regression in resident memory usage from mapping more libxul.so

RESOLVED INVALID

Status

()

Firefox for Android
General
RESOLVED INVALID
6 years ago
6 years ago

People

(Reporter: kats, Unassigned)

Tracking

(Depends on: 1 bug)

Trunk
ARM
Android
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

Unfortunately, switching the ARMv6 android builds to NDK r8c caused a regression in resident memory usage. Clicking on the Jan 2 data point on the "start" line at http://people.mozilla.com/~kgupta/awsy/ shows this in the detail view:

Timestamp       resident mem    changeset
--------------------------------------------
20130102071439	60751872	a5fee86dec9e
20130102072236	62283776	8e186f3b8f25

which indicates an increase of ~1.5 mb in resident memory on startup. Similar bumps can be seen on the other lines in the resident memory graph.

I looked at the full about:memory dumps for the before/after and this increase can be entirely attributed to mapping more of libxul.so into memory (this is also why a corresponding increase doesn't show up on the explicit memory graph).

Before:

├──17.13 MB (27.52%) -- shared-libraries
│  ├──11.55 MB (18.57%) -- shared-libraries-mozilla
│  │  ├───9.08 MB (14.59%) ── libxul.so [r-xs] [2]
│  │  ├───1.65 MB (02.65%) ── libxul.so [rwxs]
│  │  └───0.83 MB (01.33%) ++ (20 tiny)
│  └───5.57 MB (08.96%) ++ shared-libraries-other


and after:

├──18.60 MB (29.20%) -- shared-libraries
│  ├──13.03 MB (20.45%) -- shared-libraries-mozilla
│  │  ├──10.55 MB (16.56%) ── libxul.so [r-xs]
│  │  ├───1.65 MB (02.59%) ── libxul.so [rwxs]
│  │  └───0.82 MB (01.29%) ++ (20 tiny)
│  └───5.57 MB (08.75%) ++ shared-libraries-other

I also checked the sizes of the APKs and libxul.so files before and after.

          | before       | after
APK size  | 19,798,820   | 18,931,488
libxul.so | 27,491,328   | 26,589,716

So despite libxul.so getting *smaller* (which was the expected outcome), the amount we map into memory *increased* and resulted in a resident memory usage regression. Presumably doing PGO builds on android will win this back, and more.
Can you show the output of readelf -l for both libxul.so's ?
Created attachment 700068 [details]
readelf -l libxul.so outputs

attached; the first one is from the "before" libxul and the second is from the "after" libxul
Did we switch to gold with the new NDK?
Yup, the 4.6 toolchain in NDK r8c uses gold by default.
That would explain the difference. I suspect switching to GNU ld would reduce the regression, if not remove it entirely.
That being said, I also suspect these RSS numbers are a lie from the kernel.
(In reply to Mike Hommey [:glandium] from comment #6)
> That being said, I also suspect these RSS numbers are a lie from the kernel.

Can you elaborate on this?

I can try a build with ld and see how that fares. But would the PGO build also solve this problem?
glandium explained this to me on IRC. Basically if we map a tmpfs/ashemem file, the entire file is in memory anyway, and the RSS/Private_Dirty of the map will reflect how much of the file was read/written through that map. If we set up two mappings, use one extensively, and then unmap it and start using the second, this could result in under-reporting of the actual memory usage.

So what seems to be happening here is that the kernel was under-reporting more with ld than it is with gold because of how we end up reading/writing because of the new layout. This shows up as a RSS regression even though it's actually an improvement in actual memory usage because libxul.so is smaller (and is entirely in memory).
Status: NEW → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → INVALID
You need to log in before you can comment on or make changes to this bug.