828650 - ~1.5mb regression in resident memory usage from mapping more libxul.so

Status: RESOLVED INVALID

(Firefox for Android Graveyard :: General, defect)

Description

[Kartikaya Gupta (email:[e-mail])]

Unfortunately, switching the ARMv6 android builds to NDK r8c caused a regression in resident memory usage. Clicking on the Jan 2 data point on the "start" line at http://people.mozilla.com/~kgupta/awsy/ shows this in the detail view:

Timestamp       resident mem    changeset
--------------------------------------------
20130102071439	60751872	a5fee86dec9e
20130102072236	62283776	8e186f3b8f25

which indicates an increase of ~1.5 mb in resident memory on startup. Similar bumps can be seen on the other lines in the resident memory graph.

I looked at the full about:memory dumps for the before/after and this increase can be entirely attributed to mapping more of libxul.so into memory (this is also why a corresponding increase doesn't show up on the explicit memory graph).

Before:

├──17.13 MB (27.52%) -- shared-libraries
│  ├──11.55 MB (18.57%) -- shared-libraries-mozilla
│  │  ├───9.08 MB (14.59%) ── libxul.so [r-xs] [2]
│  │  ├───1.65 MB (02.65%) ── libxul.so [rwxs]
│  │  └───0.83 MB (01.33%) ++ (20 tiny)
│  └───5.57 MB (08.96%) ++ shared-libraries-other


and after:

├──18.60 MB (29.20%) -- shared-libraries
│  ├──13.03 MB (20.45%) -- shared-libraries-mozilla
│  │  ├──10.55 MB (16.56%) ── libxul.so [r-xs]
│  │  ├───1.65 MB (02.59%) ── libxul.so [rwxs]
│  │  └───0.82 MB (01.29%) ++ (20 tiny)
│  └───5.57 MB (08.75%) ++ shared-libraries-other

I also checked the sizes of the APKs and libxul.so files before and after.

          | before       | after
APK size  | 19,798,820   | 18,931,488
libxul.so | 27,491,328   | 26,589,716

So despite libxul.so getting *smaller* (which was the expected outcome), the amount we map into memory *increased* and resulted in a resident memory usage regression. Presumably doing PGO builds on android will win this back, and more.

Comment 1

[Mike Hommey [:glandium]]

Can you show the output of readelf -l for both libxul.so's ?

Comment 2

[Kartikaya Gupta (email:[e-mail])]

Attachment: readelf -l libxul.so outputs

attached; the first one is from the "before" libxul and the second is from the "after" libxul

Comment 3

[Mike Hommey [:glandium]]

Did we switch to gold with the new NDK?

Comment 4

[Kartikaya Gupta (email:[e-mail])]

Yup, the 4.6 toolchain in NDK r8c uses gold by default.

Comment 5

[Mike Hommey [:glandium]]

That would explain the difference. I suspect switching to GNU ld would reduce the regression, if not remove it entirely.

Comment 6

[Mike Hommey [:glandium]]

That being said, I also suspect these RSS numbers are a lie from the kernel.

Comment 7

[Kartikaya Gupta (email:[e-mail])]

(In reply to Mike Hommey [:glandium] from comment #6)
> That being said, I also suspect these RSS numbers are a lie from the kernel.

Can you elaborate on this?

I can try a build with ld and see how that fares. But would the PGO build also solve this problem?

Comment 8

[Kartikaya Gupta (email:[e-mail])]

glandium explained this to me on IRC. Basically if we map a tmpfs/ashemem file, the entire file is in memory anyway, and the RSS/Private_Dirty of the map will reflect how much of the file was read/written through that map. If we set up two mappings, use one extensively, and then unmap it and start using the second, this could result in under-reporting of the actual memory usage.

So what seems to be happening here is that the kernel was under-reporting more with ld than it is with gold because of how we end up reading/writing because of the new layout. This shows up as a RSS regression even though it's actually an improvement in actual memory usage because libxul.so is smaller (and is entirely in memory).