1003124 - verify that CLD's memory usage doesn't grow infinitely

Status: NEW

(Firefox :: General, defect)

Description

[Florian Quèze [:florian]]

My initial testing showed that the memory usage grows from 16MB to 18MB after detecting the language of a few pages:

(Florian Quèze from bug 971047 comment #81)

> About memory usage:

> Our cld-worker.js is compiled with a newer version of emscripten that
> outputs asm.js code. It takes 16MB of memory when loaded, and when I tried
> using it on real web pages, I [saw] the memory usage grow to 18MB.

(Florian Quèze from bug 971047 comment #88)
> For all these values, I'm looking at the size indicated by about:memory for
> the relevant worker.

> The 16MB figure is the memory usage just after our worker is loaded, before
> it's actually used. All of it is in "non-heap/elements/asm.js".
> 
> The 18MB figure is our worker, after it is has been used for a while. Here
> is how I see it in about:memory :
> │   ├──18.38 MB (06.90%) --
> worker(resource:///modules/translation/cld-worker.js, 0x119fc8400)
> │   │  ├──17.11 MB (06.42%) -- zone(0x12507d800)
> │   │  │  ├──16.89 MB (06.34%) -- compartment(web-worker)
> │   │  │  │  ├──16.84 MB (06.32%) -- classes
> │   │  │  │  │  ├──16.00 MB (06.00%) -- class(ArrayBuffer)
> │   │  │  │  │  │  ├──16.00 MB (06.00%) -- objects
> │   │  │  │  │  │  │  ├──16.00 MB (06.00%) ── non-heap/elements/asm.js
> │   │  │  │  │  │  │  └───0.00 MB (00.00%) ++ (2 tiny)
> │   │  │  │  │  │  └───0.00 MB (00.00%) ++ shapes/gc-heap
> │   │  │  │  │  └───0.84 MB (00.32%) ++ (6 tiny)
> │   │  │  │  └───0.05 MB (00.02%) ++ (3 tiny)
> │   │  │  └───0.21 MB (00.08%) ++ (6 tiny)
> │   │  └───1.27 MB (00.48%) ++ (4 tiny)


We need to verify that the memory usage doesn't continue to grow if we detect language on thousands of pages.

Comment 1

[Nicholas Nethercote [inactive]]

The 16 MiB start-up cost is a bigger problem. I guess that's the emscripten heap? Grr. Is this something that's on by default, or do you have to enable the ability to translate?

Comment 2

[Florian Quèze [:florian]]

(In reply to Nicholas Nethercote [:njn] from comment #1)
> The 16 MiB start-up cost is a bigger problem. I guess that's the emscripten
> heap?

Let's discuss that part in bug 1003120.

> Grr. Is this something that's on by default, or do you have to enable
> the ability to translate?

Currently it's not built at all (you need to add a line in a moz.build file to have it included in the build), and then it's pref'ed off by default.

The goal is to have it on by default (in the Firefox 32 cycle if I understood correctly).