752339 - JS uses MADV_DONTNEED to decommit on mac, but should use MADV_FREE

Status: RESOLVED FIXED

(Core :: JavaScript Engine, defect)

Description

[Justin Lebar (not reading bugmail)]

js/src/gc/Memory.cpp uses MADV_DONTNEED to decommit memory on Mac and Linux.

But we use MADV_FREE on Mac for this purpose, in jemalloc.

Paul and I did a lot of tests to arrive at this, but I don't immediately recall what the effect of MADV_DONTNEED was.  It may have been that MADV_DONTNEED pages are not collected by the OS when the OS is under memory pressure.  If so, that would completely defeat the purpose of decommitting memory.

There's an additional problem with the code as written: There's no equivalent of jemalloc's "double-purge".  In jemalloc, before we read the process's RSS (for telemetry or about:memory), we go through and actually munmap all pages which have been MADV_FREE'd.  This is necessary for our RSS to match reality, and is quite important for our measurements.

Comment 1

[Justin Lebar (not reading bugmail)]

I just tested on my 10.7 machine, and MADV_DONTNEED and MADV_FREE seem to do the exact same thing.  That's not what I remember!

Can someone please test on 10.6 and 10.5?

Steps to test:

 * Run the following test program (substituting MADV_DONTNEED for MADV_FREE as appropriate):

#include <sys/mman.h>
#include <stdio.h>
#include <unistd.h>

#define SIZE (512 * 1024 * 1024)

int main()
{
  char *x = mmap(0, SIZE, PROT_READ | PROT_WRITE, MAP_ANON | MAP_PRIVATE, -1, 0);
  int i;
  for (i = 0; i < SIZE; i += 1024) {
    x[i] = i;
  }
  madvise(x, SIZE / 2, MADV_FREE);
  fprintf(stderr, "Sleeping.  Now check my RSS.\n", SIZE / (2 * 1024 * 1024));
  sleep(1024);
  return 0;
}

  * Open activity monitor and watch this process's memory usage.

  * Now run the following program, which will eat up all your RAM:

#include <stdio.h>
#include <stdlib.h>

int main()
{
  while (1) {
    char* buf = malloc(1024 * 1024);
    if (!buf) {
      break;
    }
    for (int i = 0; i < 1024 * 1024; i += 1024) {
      buf[i] = i;
    }
  }
  fprintf(stderr, "Done allocating a bunch of memory.\n");
  return 0;
}


And report whether |test|'s memory usage drops down to 256mb.

Comment 2

[Justin Lebar (not reading bugmail)]

> There's an additional problem with the code as written: There's no equivalent of 
> jemalloc's "double-purge".

I filed this as bug 752342.

Comment 3

[d.a.]

I ran the above test and with MADV_FREE it went down to 256MB as the machine ran out of memory. When I replaced it with MADV_DONTNEED it went down to 271.8 MB over a few seconds in steps of 80 MB or so. I had to run the second program twice for the test program to go down to 256MB.

Comment 4

[d.a.]

For clarification, I ran the above on my 10.6 machine.

Comment 5

[Justin Lebar (not reading bugmail)]

Mike, do you remember why we use MADV_FREE on Mac instead of DONTNEED?

Comment 6

[Mike Hommey [:glandium]]

(In reply to Justin Lebar [:jlebar] from comment #5)
> Mike, do you remember why we use MADV_FREE on Mac instead of DONTNEED?

AFAIK, it's because MADV_DONTNEED does even less than MADV_FREE on FreeBSD/OSX.

(In reply to Justin Lebar [:jlebar] from comment #0)
> we go through and actually munmap all pages which have been MADV_FREE'd.

Note that depending how memory is reused, it may be safer to use mmap(MAP_FIXED).

Comment 7

[Justin Lebar (not reading bugmail)]

> AFAIK, it's because MADV_DONTNEED does even less than MADV_FREE on FreeBSD/OSX.

In the sense that DONTNEED'ed data is freed less aggressively than FREE'd memory?

> we go through and actually munmap all pages which have been MADV_FREE'd.
>
> Note that depending how memory is reused, it may be safer to use mmap(MAP_FIXED).

Are there circumstances in which mmap followed by munmap is safer than MAP_FIXED?  The former is obviously racy and I'm pretty embarrassed I put it in our jemalloc.  (It's unlikely to cause problems because we hold the arena lock while unmapping stuff; we can only race with code which doesn't call malloc.  But it's still bad!)

Comment 8

[Mike Hommey [:glandium]]

(In reply to Justin Lebar [:jlebar] from comment #7)
> > AFAIK, it's because MADV_DONTNEED does even less than MADV_FREE on FreeBSD/OSX.
> 
> In the sense that DONTNEED'ed data is freed less aggressively than FREE'd
> memory?

Reportedly, it's not freed. Comment 3 suggests otherwise, though.

> > Note that depending how memory is reused, it may be safer to use mmap(MAP_FIXED).
> 
> Are there circumstances in which mmap followed by munmap is safer than
> MAP_FIXED?

I guess you mean munmap followed by mmap, but no, there aren't any. MAP_FIXED is always safer. Unfortunately, that doesn't exist on Windows.

Comment 9

[Paul Bone [:pbone]]

Bug 1567366 changed this to MADV_FREE_REUSABLE