1139243 - Reduce size of Relooper's asm.js buffer

Status: RESOLVED INCOMPLETE

(Firefox Graveyard :: Shumway, defect)

Description

[Till Schneidereit [:till]]

Currently, the Relooper allocates a 4MB buffer. For 10 arbitrarily-chosen banners, the entire player compartment was 12-15MB, so the Relooper buffer is 27%-33% of the entire player's memory usage. (Or ~16-19% of total memory usage.)

I don't know how small we can make the buffer. We don't store the compiled source in it, so it should mostly depend on baseline usage + max-number-of-blocks * size-of-block.

We should probably look into ensuring that we can't blow the memory budget in any case and abort compilation, falling back to the interpreter, if we would.

Comment 1

[Marco Castelluccio [:marco]]

We should probably investigate for j2me.js as well.

Comment 2

[Eric Rahm [:erahm]]

Are there any hooks in ASM.js to figure out how much of that buffer is actually used? It would be nice to add a memory reporter to make determining what the buffer size should be easier.

Comment 3

[Till Schneidereit [:till]]

(In reply to Eric Rahm [:erahm] from comment #2)
> Are there any hooks in ASM.js to figure out how much of that buffer is
> actually used? It would be nice to add a memory reporter to make determining
> what the buffer size should be easier.

That sounds like a great idea to me.

Luke, is this something we can do somewhat easily? Might it even be a useful devtool, perhaps?

Comment 4

[Luke Wagner [:luke]]

Emscripten has tools for visualizing the usage of the heap dynamically.  I don't know what they are, though, so forwarding to Jukka.  The JS VM doesn't really have any idea, though.

Comment 5

[Jukka Jylänki]

Nick Desaulniers is currently working on getting the Emscripten heap visualization up to latest - there are a few updates that need to be made to the tool to make it work reproducibly. The initial PR for that is being polished in https://github.com/kripken/emscripten/pull/3178 , following the tracing tool itself.

Comment 6

[Jukka Jylänki]

Oh, and also, in an Emscripten module, one can examine the global DYNAMICTOP variable to see how much of the Emscripten HEAP has been allocated by sbrk and how much is unused. This does not give a total used memory size though, but gives a high watermark/upper bound of how much memory has been used at most by the module.

Comment 7

[Nicholas Nethercote [inactive]]

This may be less of a win than it first seems, because any unused part of the buffer will hopefully not be in RSS. (I certainly hope not, though I'm not certain.)

Having said that, the buffer still takes up address space, and on Win32 address space exhaustion does happen in practice.

Comment 8

[Eric Rahm [:erahm]]

(In reply to Nicholas Nethercote [:njn] from comment #7)
> This may be less of a win than it first seems, because any unused part of
> the buffer will hopefully not be in RSS. (I certainly hope not, though I'm
> not certain.)

It really depends how the memory is used. Each untouched page would not be in RSS, but once the pages are touched they definitely are. So lets say the heap is used in a circular buffer manner then all of the memory would eventually be in RSS. And if we do something like calloc then you lose off the bat.