libxul.so gets loaded at different addresses for each copy of plugin-container

RESOLVED INVALID

Status

Firefox OS
General
RESOLVED INVALID
5 years ago
5 years ago

People

(Reporter: dhylands, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

5 years ago
I'm running a debug build of otoro-ics

So I noticed that if I ran Calculater, Dialer, and Calendar, and then say run Dialer again, then Dialer will get killed due to OOM.

If I change the order to Calculator, Calendar, Dialer, Calendar, then Calendar will get killed due to OOM.

So then I just launched those 3 so that nothing crashed and took a look at the memory maps (/proc/PID/maps) and observed the following for libxul.so

4078b000-41f2b000 r-xp 0006b000 1f:05 274        /system/b2g/libxul.so (b2g main process)
403f7000-41b97000 r-xp 0006b000 1f:05 274        /system/b2g/libxul.so (Dialer)
40467000-41c07000 r-xp 0006b000 1f:05 274        /system/b2g/libxul.so (Calculator)
403fb000-41b9b000 r-xp 0006b000 1f:05 274        /system/b2g/libxul.so (Calendar)

This represents the text area of libxul.so, and in each case it takes up 0x17a0000 or about 24 Mb of memory.

However, since libxul.so is being linked to a different base address, there is no possible way that the physical pages can be shared between the processes.

The memory is demand paged, so not all 25 Mb will be present (i.e. it how much is actually present will depend on the exact code paths in each process).

We really need to ensure that the libraries are linked and loaded in the same order (or to the same addresses) so that we can maximize code page sharing between processes.
mwu, do you know what's required to sic the prelinker on b2g and plugin-container?

Comment 2

5 years ago
Nice find.

Comment 3

5 years ago
According to http://source.android.com/tech/security/index.html#memory-management-security-enhancements ICS added ASR, which explains why we get randomized into oblivion here.

Comment 4

5 years ago
Pages are only nonshared if they have text relocations, which we very much would like to avoid in general.

(In reply to Dave Hylands [:dhylands] from comment #0)
> However, since libxul.so is being linked to a different base address, there
> is no possible way that the physical pages can be shared between the
> processes.
> 

In general, Linux systems tend to use PIC which eliminate this issue. See http://eli.thegreenplace.net/2011/11/03/position-independent-code-pic-in-shared-libraries/

Comment 5

5 years ago
Are we compiling libxul as pic?

Comment 6

5 years ago
Yeah we are. There is occasionally a bit of assembly code that isn't relocatable, but in general most of the relocations shouldn't be text relocations.

Comment 7

5 years ago
dhylands, this should be a wontfix then according to comment 6. Do you agree?
(Reporter)

Comment 8

5 years ago
OK - I'm satisfied that the text areas actually look identical (I pickled a couple of random 4K chuncks and compared them).

I'd still like to verify that they map to the same physical pages. I'll see if there's an easy way to do that.
$ adb shell cat /proc/242/smaps | grep -A14 libxul

tells me that most of the +rx pages are shared, and the PSS for the mappings in plugin-container with just that and b2g running are 1/2 the shared size.

So I think this is a false alarm.  Would have been a nice win though!
(In reply to Michael Wu [:mwu] from comment #6)
> Yeah we are. There is occasionally a bit of assembly code that isn't
> relocatable, but in general most of the relocations shouldn't be text
> relocations.

We got rid of all remaining text relocations a while ago.

Note that even without aslr, relocated pages would still not be shared, because what happens is that the linker loads the file, and then applies relocations. In each process. So there's no page sharing for relocated pages. And there are an awful lot of them, actually, because of vtables, mostly. There are 269k relocations. Accounting for the fact that not all of them are contiguous, and thus spread on more pages, that's 1.6MB of unshared memory*. Then there's also static initializers, that, in many cases, store the same values in every process. That's not shared either.

The android custom linker is planned to gain memory sharing abilities, that will lift this. It's a Q3 goal fwiw. Prelinking would help in the meanwhile. As well as KSM, if the kernel were compiled with it (which it isn't on otoro): http://kernelnewbies.org/Linux_2_6_32#head-d3f32e41df508090810388a57efce73f52660ccb

* expr $(readelf -r objdir-gecko/dist/bin/libxul.so | awk '/^0/{print int(strtonum("0x" $1) / 4096)}' | sort -u | wc -l) \* 4096
(Reporter)

Comment 11

5 years ago
So I took a look at the rw-p sections (writable data).

The Dialer instance of plugin container has a total of 27 Mb of writable data, and the Calculator instance has about 20 Mb.

The Dialer app had 3 extra sections of size 0xff000 (4K less than 1 Mb), 1 extra section of 1 Mb, and a section that was 6 Mb, where the calculator was 4Mb.

All of the sections that were "added" were dynamic (not associated with an .so)

For calculator, the sum of the data associated with a .so file was about 2Mb. The sum of non-associated data was 18.6 Mb.

Anyways, the size of the writable data in comparison to the amount that MemFree drops when launching an app suggests that the code areas are in fact being shared between the apps.
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → INVALID
You need to log in before you can comment on or make changes to this bug.