1106662 - Memory leak on piratepad.net

Status: NEW

(Core :: JavaScript Engine, defect)

Description

[Timvde]

Attachment: memory-report.json.gz

I had piratepad.net open in an app tab for a few days. In the meantime, I suspended my laptop, reconnected to other wifi networks, and it disconnected from the server. (I'm not 100% sure if it's the direct cause, but let's mention it anyway)

This caused the website to consume over 2GB of RAM, with the strongest offender being one string of length 368b of which there were 1 331 699 copies.

I do have quite a few add-ons installed (17 active). If needed, I can provide the full list. I haven't tried to reproduce this on a clean profile yet, but I will try soon and report results.

An anonymized about:memory dump is attached. If needed, I can also provide the full report through e-mail.

Comment 1

[Jan de Mooij [:jandem]]

(In reply to Timvde from comment #0)
> This caused the website to consume over 2GB of RAM, with the strongest
> offender being one string of length 368b of which there were 1 331 699
> copies.

I loaded the attached report in about:memory but I can't find this string. Do you happen to know what string it was? It could tell us something. Was this Firefox 33 or Nightly?

njn, all numbers below seem really big? Any thoughts?

6,114,841,478 B (100.0%) -- explicit
├──3,082,718,120 B (50.41%) ++ window-objects
├──2,007,573,528 B (32.83%) ++ js-non-window
├────420,324,560 B (06.87%) ++ add-ons
├────322,146,168 B (05.27%) ── heap-unclassified

Comment 2

[Timvde]

I have sent you an e-mail with the more detailed report. Feel free to share it with other developers, or paste anonymized, relevant pieces in this bug.

At this moment, I'm using the release branch of Firefox, since Nightly seems to be unusably slow ever since the e10s shim was activated. A bug report about that issue will follow as soon as I have more time to find the culprit (probably an add-on).

smaug mentioned on IRC that it might not be Firefox's fault, but that piratepad.net is just leaky. I am trying to reproduce the issue (but it'll take a few days for memory to fill up) and see if it gets properly released afterwards. Sadly, I forgot to test.

Comment 3

[Jan de Mooij [:jandem]]

Thanks Tim! I also forwarded the full report to njn.

(In reply to Timvde from comment #0)
> This caused the website to consume over 2GB of RAM, with the strongest
> offender being one string of length 368b of which there were 1 331 699
> copies.

Actually there are many strings with 1331699 copies IIUC... Here are some examples:

│  │  │  │  ├─────95,882,328 B (01.57%) -- string(length=44, copies=1331699, "connect: set, hiccup: set, singleHiccup: set")
│  │  │  │  │     ├──63,921,552 B (01.05%) ── malloc-heap/latin1
│  │  │  │  │     └──31,960,776 B (00.52%) ── gc-heap/latin1

│  │  │  │  ├─────42,614,464 B (00.70%) ── string(length=13, copies=1331702, "{ post: set }")/gc-heap/latin1

│  │  │  │  ├─────31,960,776 B (00.52%) ── string(length=6, copies=1331699, "881301")/gc-heap/latin1

Comment 4

[Nicholas Nethercote [inactive]]

Here's another one:

> │  │  │  │  ├────518.16 MB (08.91%) -- string(length=368, copies=1331699, "shortpolling: { isConnected: true, isClosed: true, timeouts: { connectAttempt: set, initialConnect: set }, request: not set }, longpolling: { isConnected: true, isClosed: true, timeouts: { connectAttempt: set, initialConnect: set }, request: not set }, streaming: { isConnected: false, isClosed: true, timeouts: { initialConnect: unset }, request: not set, cursor: 0 }")

And there are various others with contents related to network/socket stuff.

And they're all from within the piratepad.net zone, so it's coming from the site. Judging from the contents, my guess is that the site "leaks" a string every time it tries and fails to connect. (This is not a true leak, as the string is still being held onto.)

Having said that, there's also this:

> ├──1,925.17 MB (33.11%) -- js-non-window
> │  ├──1,743.68 MB (29.99%) -- zones
> │  │  ├──1,400.42 MB (24.08%) -- zone(0x7f7977f52000)
> │  │  │  ├────901.10 MB (15.50%) ── unused-gc-things

That's bad fragmentation in the GC heap. Hopefully compacting GC (bug 650161) will help with that.

Comment 5

[Nicholas Nethercote [inactive]]

Have you tried this in another browser, such as Chrome?

Comment 6

[Andrew McCreight [:mccr8]]

This sounds like it is probably a leak in the site itself.

Comment 7

[Timvde]

Sorry for my late answer, I had a busy time at university (end of semester project deadlines + exams).

Meanwhile, it seems impossible to reproduce on Nightly. I know that I had this a few times before (that's why I originally stopped using the website as app tab), but Firefox froze too hard and I needed to use my browser, so this was the first time that I waited 10 minutes or so for the report to generate.

I did stop using Ghostery (and added the EasyPrivacy list to ABP) because I switched back from release to Nightly. That is (apart from the actual switch to Nightly) the only change in my set-up that I can think of. I can't test Nightly + Ghostery, because even if I disable e10s, the shim is making everything extremely slow.

As it is now, this bug is probably INCOMPLETE. Would it help if I try to reproduce it again on release? Maybe with some custom build which adds extra memory debugging?

Comment 8

[Nicholas Nethercote [inactive]]

> Would it help if I try to reproduce it again on release?
> Maybe with some custom build which adds extra memory debugging?

It might help. You shouldn't need anything custom because about:memory gave good data earlier. Thank you.

Comment 9

[Timvde]

After a long time, I could reproduce it again today. I'm on Nightly again with Ghostery disabled (I switched back after not being able to reproduce the bug for over a week), so that's not the culprit. It seems like I have to get the website in some specific state to get the leak. I have not idea how to reproduce that state, though.

This time, I found out about the leak way before I actually ran out of memory, so Firefox was still snappy and I could look around a bit. The profiler reports most of the busy time in req.onreadystatechange of the following code:

function doAction(method, uri, async, headers, body, cb) {
  try {
    document.domain = oldDomain;
  } catch (e) { }
  var req = createRequestObject();
  req.open(method, '//'+host+uri, async);
  for (var i in headers) {
    req.setRequestHeader(i, headers[i]);
  }
  req.onreadystatechange = function() {
    if (req.readyState == 4) {
      try {
        document.domain = newDomain;
        cb(req.status, req.responseText);
      } catch (e) {
        // yikes. well, hopefully a timeout will notice this error.
      }
    }
  }
  req.send(body);
}

In one of the code paths leading there, HTTPS Everywhere appeared, although with a lot less processing time. Most of the time spent was "self time" in the above function, so I'm not sure if it's actually related.

about:memory looks exactly the same as before (except that the number of copies of each string is lower). What might be interesting, is that after closing the tab and clicking "Minimize memory usage" in about:memory, a lot of the memory seemed to have gone into "heap-unclassified". piratepad.net also still appears a few times in the report, although not with any high values. I'll send a detailed report again via e-mail, from before and after closing the tab, to make sure values can be compared.

While I was at it, I also saved gc and cc logs, and the full profiler details. I'll add them to the mail too.

Comment 10

[Wayne Mery (:wsmwk)]

(In reply to Timvde from comment #9)
> ...
> While I was at it, I also saved gc and cc logs, and the full profiler
> details. I'll add them to the mail too.

Did this prove to be useful?

Comment 11

[Nicholas Nethercote [inactive]]

> Did this prove to be useful?

I don't think I received an email about this. mccr8 is normally the one who asks for GC and CC logs...

Comment 12

[Timvde]

njn: I sent an e-mail on 2015-02-18 to jandem and you. Maybe it got blocked by your spamfilter for some reason? I can send it again if you want, if you still think it's relevant. But I guess the leak was (is? I haven't used the site in a while) on the website itself, and is not Firefox's fault.