1357225 - Nightly Parent process leaks readwrite segments.

Status: RESOLVED DUPLICATE of bug 1347909

(Core :: IPC, defect)

Description

[Danial Horton]

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:55.0) Gecko/20100101 Firefox/55.0
Build ID: 20170414163021

Steps to reproduce:

Open nightly, either a clean profile or one with a few tabs and extensions.
do not click to load any of the restored tabs

wait.

Alternate
Open About:memory
Open about:performance

in a fresh profile


Actual results:

The parent process memory usage grows with no interaction occurring.

Differing the memory reports shows the only major difference is in readwrite segments

Other Measurements

0.00 MB (100.0%) -- address-space
├──-3,086.86 MB (100.0%) ── free(segments=457) [-]
├──2,231.70 MB (100.0%) ── free(segments=416) [+]
├──807.43 MB (100.0%) -- commit
│  ├──818.14 MB (100.0%) -- private
│  │  ├──1,032.35 MB (100.0%) ── readwrite(segments=2633) [+]
│  │  ├──-213.48 MB (100.0%) ── readwrite(segments=1222) [-]
│  │  ├───-2.90 MB (100.0%) ── readwrite+stack(segments=174) [-]
│  │  ├────2.50 MB (100.0%) ── readwrite+stack(segments=156) [+]
│  │  ├───-1.33 MB (100.0%) ── readwrite+guard(segments=174) [-]
│  │  ├────1.21 MB (100.0%) ── readwrite+guard(segments=156) [+]
│  │  ├───-1.19 MB (100.0%) ── execute-read(segments=12) [-]
│  │  ├────0.95 MB (100.0%) ── execute-read(segments=10) [+]
│  │  └────0.04 MB (100.0%) ── execute-readwrite(segments=5)
│  ├────0.45 MB (100.0%) -- mapped
│  │    ├──-121.48 MB (100.0%) ── readonly(segments=98) [-]
│  │    ├──117.80 MB (100.0%) ── readonly(segments=64) [+]
│  │    ├──66.72 MB (100.0%) ── readwrite(segments=37) [+]
│  │    └──-62.59 MB (100.0%) ── readwrite(segments=33) [-]
│  └──-11.16 MB (100.0%) -- image
│     ├──-103.86 MB (100.0%) ── execute-read(segments=217) [-]
│     ├───99.34 MB (100.0%) ── execute-read(segments=198) [+]
│     ├──-69.41 MB (100.0%) ── readonly(segments=488) [-]
│     ├───64.10 MB (100.0%) ── readonly(segments=437) [+]
│     ├───-3.08 MB (100.0%) ── writecopy(segments=122) [-]
│     ├────2.53 MB (100.0%) ── writecopy(segments=97) [+]
│     ├───-2.28 MB (100.0%) ── readwrite(segments=261) [-]
│     ├────1.99 MB (100.0%) ── readwrite(segments=223) [+]
│     ├───-0.60 MB (100.0%) ── execute-readwrite(segments=1) [-]
│     └────0.12 MB (100.0%) ── execute-readwrite(segments=2) [+]
└──47.72 MB (100.0%) -- reserved
   ├──449.41 MB (100.0%) ── private(segments=1742) [+]
   ├──-401.79 MB (100.0%) ── private(segments=1078) [-]
   └───0.10 MB (100.0%) ── mapped(segments=5)


Expected results:

It probably shouldn't consume more and more ram while idle.

Comment 1

[Danial Horton]

Attachment: memory-report_first.json.gz

saved from a fresh start

Comment 2

[Danial Horton]

Attachment: memory-report_second.json.gz

saved after a few days running.

this session has had no interaction at all other then to click force gc/cc and minimize memory usage.

only about:memory and about:performance is loaded.


I tried doing this in a clean profile, but for some reason about:performance would end up consuming 100% of a core and spike to 2GB at random.

Comment 3

[Steve Fink [:sfink] [:s:]]

I'm seeing what looks to be the same thing on linux. They're all shm segments named eg /dev/shm/org.chromium.5aHrz4. Some IPC thing?

Comment 4

[Steve Fink [:sfink] [:s:]]

Attachment: lsof output showing lots and lots of shm stuff

Comment 5

[Steve Fink [:sfink] [:s:]]

Attachment: corresponding /proc/(pid)/maps file

Comment 6

[Andrew McCreight [:mccr8]]

Jed, is this something you could look at?

Bill said one possible approach would be to give these shm things more specific names, though it wouldn't be easy.

Comment 7

[Jed Davis [:jld] ⟨⏰|UTC-7⟩ ⟦he/him⟧]

There might be two different bugs here.

:sfink's lsof file shows 445 shared memory segments of size 40.  I think this is CrossProcessSemaphore — in the copy of glibc I have, sizeof(sem_t) == 32, so adding two more uint32_t's would be 40.  (Constrast CrossProcessMutex; sizeof(pthread_mutex_t) == 40, here, so with the one added uint32_t in that case it would be larger.)

The one caller of CrossProcessSemaphore::Create is in gfx/layers/client/TextureClient.cpp; it's wrapped in a refcounted object and TextureClient holds one via RefPtr.  But it's possible that CrossProcessSemaphore is buggy and not correctly cleaning up the shared memory on destruction.


But the original report in comment #0: Why do we think that has anything to do with shared memory?  I don't know Windows internals very well, but I'd tend to assume that `address-space/commit/private/readwrite` is just normal heap memory.

Comment 8

[Steve Fink [:sfink] [:s:]]

Oops, sorry, I just kind of assumed that it was the same thing. And I'm not even sure my report should be taken that seriously -- I've been messing around with my Optimus laptop and juggling multiple X servers and things, so it's possible that this is purely a system misconfiguration problem on my part. (And my current firefox only has 100 of them.)

Comment 9

[Danial Horton]

Heap memory shouldn't just grow and grow and grow with no interaction from the user.

I scoured over the diffs and it is the only value that changes when this growth is occurring.

The linux issue could be similar but without having finer details in that about:memory field its hard to say

There was once an issue with a certain video game emulator where cache invalidation for textures was not purging them, so the process kept loading the same textures over and over till it ran out of address space.

im pretty sure it was pcsx2 and one of the Gran Turismo games.

Comment 10

[Danial Horton]

it looks like bug 1357225 is already trying to find out what i reported here.


335.80 MB (100.0%) ++ explicit

Other Measurements

4,095.94 MB (100.0%) ++ address-space

45.28 MB (100.0%) ++ decommitted

2,921 (100.0%) ++ event-counts

1 (100.0%) ++ file-blob-urls

205.48 MB (100.0%) ++ heap-committed

4.99 MB (100.0%) ++ images

225.49 MB (100.0%) ++ js-main-runtime

464 (100.0%) ++ js-main-runtime-compartments

105.99 MB (100.0%) ++ js-main-runtime-gc-heap-committed

0 (100.0%) ++ low-memory-events

918 (100.0%) ++ message-manager

1,293 (100.0%) ++ observer-service

433 (100.0%) ++ observer-service-suspect

794 (100.0%) ++ preference-service

0 (100.0%) ++ queued-ipc-messages

6.50 MB (100.0%) ++ window-objects

    0.00 MB ── gfx-d2d-vram-draw-target
    0.00 MB ── gfx-d2d-vram-source-surface
    0.99 MB ── gfx-surface-win32
    0.00 MB ── gfx-textures
    0.00 MB ── gfx-textures-peak
    0.00 MB ── gfx-tiles-waste
          0 ── ghost-windows
    0.00 MB ── gpu-committed
    0.00 MB ── gpu-dedicated
    0.00 MB ── gpu-shared
  188.97 MB ── heap-allocated
    1.00 MB ── heap-chunksize
  241.00 MB ── heap-mapped
          1 ── host-object-urls
    1.72 MB ── imagelib-surface-cache-estimated-locked
    1.97 MB ── imagelib-surface-cache-estimated-total
          0 ── imagelib-surface-cache-overflow-count
    1.21 MB ── js-main-runtime-temporary-peak
  654.51 MB ── private
  720.90 MB ── resident
  628.65 MB ── resident-unique
   21.32 MB ── shmem-allocated
   21.41 MB ── shmem-mapped
    3.80 MB ── system-heap-allocated
    0.00 MB ── tracelogger
          0 ── unresolved-ipc-promises
1,396.93 MB ── vsize
1,805.94 MB ── vsize-max-contiguous

And is suggesting that VMM Free's are failing for some reason.

Comment 11

[Danial Horton]

woops, linked back to this bug instead of bug 1298905

Ben appears to be looking into this with bug 1347909 so perhaps marking duplicate.