703474 - Setting gczeal to 4 and back to 0 causes everything to leak

Status: RESOLVED FIXED

Description

[Jesse Ruderman]

Created attachment 575350 [details]
browser testcase (requires extension)

To reproduce in a debug browser build:

  1. Install https://www.squarefree.com/extensions/domFuzzLite3.xpi 
  2. XPCOM_MEM_LEAK_LOG=2 ./firefox v.html

  --> trace-refcnt reports that nsGlobalWindow and more leak

To reproduce in the shell using valgrind:

  1. Build a spidermonkey shell:
    a. Create a new directory
    b. js/src/configure --enable-debug --disable-optimize --enable-valgrind
    c. make
  2. valgrind --leak-check=full --dsymutil=yes ./js -e "gczeal(4); gczeal(0);"

  --> valgrind reports a significant leak rooted in js::gc::StartVerifyBarriers

Comment 1

[Jesse Ruderman]

Created attachment 575351 [details]
shell valgrind output

Comment 2

[Jesse Ruderman]

It worries me that turning on GC debugging makes the GC stop working ;)

Comment 3

[Bill McCloskey (:billm)]

Created attachment 608552 [details] [diff] [review]
fix

I remember fixing the shell problem a while ago, although I can't find the bug. It no longer reproduces.

The problem in the browser case was that, even though we set the gczeal back to 0, we were still calling Start/EndVerifyBarriers repeatedly. I had thought that the verifier would stop at the next GC. However, if a GC happens during a verification, it starts a new one after the GC is done.

The current patch adds a few places where we EndVerifyBarriers if the zeal is 0. I considered just calling EndVerifyBarriers from SetGCZeal, but I'd rather have this happen from places where we already know that GC is safe.

Also, I made sure not to start a verification if we do a CC_FORCED GC, since it relies on the mark bits being correct afterwards.

Comment 4

[Igor Bukanov]

Comment on attachment 608552 [details] [diff] [review]
fix

Review of attachment 608552 [details] [diff] [review]:
-----------------------------------------------------------------

::: js/src/jsgc.cpp
@@ +3693,5 @@
>          }
> +        ~AutoVerifyBarriers() {
> +            if (inVerify && reason != gcreason::CC_FORCED &&
> +                cx->runtime->gcZeal() == ZealVerifierValue)
> +            {

There is no reason to store inVerify and reason as they stay the same. Instead store the result of inVerify && reason != gcreason::CC_FORCED in a flag. Also comment here why it is necessary to protect StartVerifyBarriers(cx) with the extra checks.

Comment 5

[Bill McCloskey (:billm)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/ca74029146c2

Comment 6

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/ca74029146c2