Closed Bug 532965 Opened 10 years ago Closed 2 years ago

Our Ts cold startup on the Mac is not cold enough

Categories

(Testing :: General, defect)

x86
macOS
defect
Not set

Tracking

(Not tracked)

RESOLVED WONTFIX

People

(Reporter: joelr, Unassigned)

References

Details

purge on Mac OSX does not purge the pages of the code mmap-ed into memory by the dynamic linker. Purging the code is what we care about and there 3 ways of accomplishing it

1) Requesting 2x physical memory from the virtual memory pager and then touching every page, e.g.

#!/usr/bin/env python

import mmap

SIZE = 8589934592L # 8Gb

map = mmap.mmap(-1, SIZE)
map[::4096] = '\x01' * (SIZE // 4096)

or 

#include <sys/mman.h>
#include <fcntl.h>
#include <stdio.h>

static long SIZE = 8589934592L;

int main(int argc, char**argv)
{
  char *result = mmap(0, SIZE, PROT_READ|PROT_WRITE, MAP_ANON|MAP_PRIVATE, -1, 0); 
  if (result != MAP_FAILED)
  {
    unsigned long i;
    
    for (i = 0; i < SIZE; i += 4096)
      result[i] = 1;
    return 0;
  }
  return 1;
}

or 

2) Rebooting and waiting about a minute for the kernel to stop streaming whatever it streams from disk, or

3) Unmounting the file system where our code lives, which clears the Unified Buffer Cache (UBC).

so

#1 takes 5-10 minutes for 8Gb of memory on my laptop (4Gb physical)

#2 may not be an option

#3 is the option I recommend

I have a USB HD plugged in and a shell script that takes care of mounting and unmounting 

#!/bin/sh

FF=/tmp/firefox

mkdir $FF > /dev/null 2>&1
umount $FF > /dev/null 2>&1
mount_hfs /dev/disk1s2 $FF

$FF/Minefield$*.app/Contents/MacOS/firefox-bin -no-remote -foreground file://$FF/startup.html#`python -c 'import time; print int(time.time() * 1000);'`

As suggested by Taras, we can also mount and unmount a file system. This can be a partition (results most truly matching real life) or a disk image. 

I'm not sure what kind of an overhead going to a disk image on a real file system adds. It's certainly a level of indirection which may skew results not in our favor but it does not require re-partitioning the disks in our test machines.
Here's a simpler and more secure way to unmount and remount the file system.

#!/bin/sh

FF=/Volumes/Fujitsu80Gb

diskutil unmount force $FF
diskutil mount /dev/disk1s2

$FF/Minefield$*.app/Contents/MacOS/firefox-bin -no-remote -foreground -P clean file://$FF/startup.html#`python -c 'import time; print int(time.time() * 1000);'`
In your case the app is on a USB HD and I'm presuming that's what you're un/mounting here?

I don't see how that can work on the talos boxes, where we don't have any unmountable storage, and the code is not on there anyways.

I'll dig around for another way to clear UBC.
As Taras suggested, create a disk image with Firefox and mount and unmount that.

You can also create a separate file system on the talos boxes and I would prefer that option since running from a disk image will likely slow things down a bit. 

Creating a separate file system on the talos boxes will require formatting the drives to create a separate partition, though.
Dietrich,

You can also clear the UBC using the approach in my original post. Just allocate twice the physical memory then touch every page. Use my C or Python code. Takes about 5 minutes for 4Gb of physical memory.
Blocks: 501563
Mass closing bugs with no activity in 2+ years. If this bug is important to you, please re-open.
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.