memory leak when using PAC file (regex ?)

RESOLVED FIXED

Status

()

RESOLVED FIXED
8 years ago
3 years ago

People

(Reporter: jo.hermans, Unassigned)

Tracking

Trunk
x86
Windows XP
Points:
---

Firefox Tracking Flags

(blocking2.0 final+)

Details

(Whiteboard: [softblocker])

Attachments

(1 attachment)

31.20 KB, text/plain
Details
(Reporter)

Description

8 years ago
Since bug 605662 (enable per-compartment GC) is checked in, I noticed a serious memory leak in the nightly builds when I'm using a PAC file. Since a PAC file uses JavaScript, I'm thinking that its compartment is never gc'ed or something similar.

If I'm at home, I don't have access to the proxy-server, and I'm using direct connections. My memory usage (win32/workingset is about:memory) is more or less stable around 200-240 MB, and I can see memory being released all the time.

But when I'm at work, I have to use a proxy server thru the PAC-file (which is pretty large).  My memory usage is growing rapidly, and can reach 500 MB after 20 minutes or so. It never goes down. The malloc/allocated counter doesn't change much, it's mostly the win32/* and js/* counters.

I know it's pretty vague, but I tried to eliminate all other possibilities. I've run in safe mode, I've created a new profile, etc ...  I'm using the exact same browser and profile (and hardware of course) in both environments. The only difference is that I'm using a PAC file at the office. Note that I'm not changing my preferences at all, at home I'm not able to load the PAC-file, and thus I'm falling back to direct connections.

Since there is no way currently to see better memory reports detailed per compartment or tab (I've seen the bugs), what do you suggest that I should use to debug this further?

Comment 1

8 years ago
We should figure out whats going on here.
blocking2.0: --- → final+
Whiteboard: [softblocker]
(Reporter)

Comment 2

8 years ago
I'll ask some help from the Nightly-testers mailinglist, to see if someone else can reproduce. I'll also attach the PAC file in a moment.
(Reporter)

Comment 3

8 years ago
Created attachment 503618 [details]
PAC file
Not sure exactly what those functions do, but it looks like this could be creating a zillion regexps and compiling them to native code. It could be a leak, but maybe we just need a discard algorithm like we have for methodjit code.
(In reply to comment #4)
> Not sure exactly what those functions do, but it looks like this could be
> creating a zillion regexps and compiling them to native code. It could be a
> leak, but maybe we just need a discard algorithm like we have for methodjit
> code.

I suppose if it started only after compartment GC landed, then it more likely is a leak.
Does it help if you set javascript.options.mem.gc_per_compartment in about:config to false?
(Reporter)

Comment 7

8 years ago
No.

I'm testing with a group of 15 URLS that I load as a group.

               gc_per_compartment  gc_per_compartment
               == true             == false
PAC file       427.515.904         464.564.22
without PAC    277.647.36          307.945.47

As you can see that PAC-file uses a lot more memory, but changing javascript.options.mem.gc_per_compartment had not much influence (interesting, setting it to true seems to use less memory).
(Reporter)

Comment 8

8 years ago
(In reply to comment #5)
> (In reply to comment #4)
> > Not sure exactly what those functions do, but it looks like this could be
> > creating a zillion regexps and compiling them to native code. It could be a
> > leak, but maybe we just need a discard algorithm like we have for methodjit
> > code.
> 
> I suppose if it started only after compartment GC landed, then it more likely
> is a leak.

maybe bug 587288 ?
(In reply to comment #8)
> (In reply to comment #5)
> > I suppose if it started only after compartment GC landed, then it more likely
> > is a leak.
> 
> maybe bug 587288 ?

That one landed to TM, so you could try the latest nightly and see if the problem is still there (unless you're already on it).
(Reporter)

Comment 10

8 years ago
(In reply to comment #9)
> That one landed to TM, so you could try the latest nightly and see if the
> problem is still there (unless you're already on it).

I am running the latest nightlies. I was thinking that the problem was *caused* by bug 587288, because it also appeared in the same build as I first saw it. And it mentions both regex and compartments. But that is just a wild guess. I'm starting to think it's relates to one of the YARR/regex changes. Apparently, my PAC file uses a lot of regex (I didn't wrote that monstrosity), which make the problem easy to detect.

I'll try to repeat the problem without a PAC file, just a manually configured proxy server, but it will take some time, as this setup doesn't work through the VPN at home. I'll have to go to the office.
(Reporter)

Comment 11

8 years ago
Bingo. When  I configure a manual proxy (which is the same one as the bottom rule of attachment 503618 [details], and all test URLS were using that one anyway), I fall back to the same memory usage as without a proxy server. It seems indeed that it's caused by the regular expressions.

Chris, do you have any idea ?

               gc_per_compartment  gc_per_compartment
               == true             == false
PAC file       436,064,256         445,788,160
without PAC    274,132,99          269,496,320
manual proxy   272,568,320         272,441,344


Note that I repeated the tests several times and took an average of 3 requests requests, because the results can vary a lot (+- 30 MB easily), due to the large amount of simultaneous requests. 

PS : I forgot to mention that I didn't see this behavior in earlier nightlies, or in Firefox 3.6.*. It is a regression.
Depends on: 626118
(Reporter)

Comment 13

8 years ago
The memory leak seems to be gone, now that bug 626118 is fixed.

Running with a PAC file still consumes a lot more memory than without one (due to the complexity of attachment 503618 [details]). But during normal surfing (not loading all at once), it doesn't hurt anymore. And the memory is reclaimed, which was this bug.
Status: NEW → RESOLVED
Last Resolved: 8 years ago
Resolution: --- → FIXED
Summary: memory leak when using PAC file since comparent-gc is enabled → memory leak when using PAC file (regex ?)

Comment 14

3 years ago
This is available in 43-44 version of firefox. Without regexps for some load browser consume 500M, with pac-file with 400 regexps with same load it consume 1.6G

Comment 15

3 years ago
Veryfy/reopen please
You need to log in before you can comment on or make changes to this bug.