920200 - Firefox segfaults at startup (even in safe mode)

Status: RESOLVED WORKSFORME

(Core :: JavaScript Engine, defect)

Description

[Pierre Lejeune]

Attachment: GDB output

When trying to start firefox (in either normal or safe mode), it makes a segfault.

Compiled on Linux 3.11.0 x86_64.

Comment 1

[Pierre Lejeune]

Attachment: Strace output

Comment 2

[Benjamin Smedberg]

Iona, please actually look at the stack traces before moving things into the wrong component.

Comment 3

[Benjamin Smedberg]

Looking at just the stack, this is likely a duplicate of bug 851015, but that was filed based solely on crash-stats. Since it seems you can reproduce this, hopefully we can come up with a fix.

Pierre, what version of the code are you compiling?

Naveed, can you find an owner for this and bug 851015?

Comment 4

[Pierre Lejeune]

Hello Benjamin,

I'm trying to compile the latest stable version, i.e. 24.0.

Comment 5

[Pierre Lejeune]

Are there any news about this bug? I can't use my favourite browser. :-(

Comment 6

[mjh563]

Pierre, is there any reason you're trying to compile Firefox yourself instead of using Mozilla's official binaries?

If you just want to run Firefox 24, you can download a pre-built Linux x86_64 version from here:
http://ftp.mozilla.org/pub/mozilla.org/firefox/releases/24.0

Comment 7

[Pierre Lejeune]

I use a source-based distro...

Comment 8

[Pierre Lejeune]

Any news?

Comment 9

[Bobby Holley (:bholley)]

Arg, désolé Pierre. If you can reproduce this crash and are willing to help, there's no excuse for us not to fix it. ;-)

So, let's get to the bottom of this. Here are some things that would be helpful:

(1) Are you running a debug build or an opt build?

(2) What address does |obj| point to in JS_ObjectToOuterObject? When you hit the crash in gdb, you can do |p obj_| to answer that question.

(3) If you can reproduce this in a debug build, what happens when you do |obj_->dump()|?

(4) What is the value of obj_->getClass()? Is it garbage data?

(5) What happens if you do |p obj_->getClass()->name|? Does it give you a char*? If so, what is the value of the char*? You can do |x/10bs obj_->getClass()->name|.

Comment 10

[Bobby Holley (:bholley)]

(also, do you have any extensions installed?)

Comment 11

[Pierre Lejeune]

Hello Bobby,

Thanks for helping. :-)

Here are the answers:

1) I'm running an opt build but debugging symbols are around.
2) When I |p obj_|, gdb outputs "No symbol "obj_" in current context."
So, I can't answer to points 3, 4 & 5.

Same results with a clean profile.

Comment 12

[Bobby Holley (:bholley)]

From what I see in the gdb stack, you need to do |f 1| or |up| in order to execute the print in the context of the right frame.

Can you try doing a debug build and seeing if the problem is still there? Debug builds are (surprise) easier to debug ;-)

Comment 13

[Pierre Lejeune]

Here are the results after a debug build:

2) "p obj" (without underscore): $1 = {<js::HandleBase<JSObject*>> = {<No data fields>}, ptr = 0x7fffdc5b1f70}
3) "p obj->dump()": $2 = void
4) "p obj->getClass()": $3 = (js::Class *) 0x7f0d4eca1520 <js::FunctionProxyClass>
5) "p obj->getClass()->name": $4 = 0x7f0d4eaa3428 "Proxy"

Comment 14

[Bobby Holley (:bholley)]

Can you link me to your crash report? See about:crashes

> 3) "p obj->dump()": $2 = void

This should print a dump of the object in the same terminal as the console spew from Firefox. Are you running gdb in a separate terminal? If so, can you paste the dump spew from the other one?

Comment 15

[Bobby Holley (:bholley)]

You can also try catching me in #content on irc.mozilla.org - my nick is bholley.

Comment 16

[Pierre Lejeune]

Well, in a debug build (--enable-debug), firefox waits 5min for gdb to attach (from another terminal).
I got some other values from debugger (but it describes the same object obviously).
Maybe I didn't type "up" enough. ;-)

Frame #7  0x00007f5f5747fe3f in JS_ObjectToOuterObject (cx=0x7f5f4f1743e0, obj_=0x7f5f4a83a6a0) at /var/tmp/paludis/build/dev-libs-xulrunner-24.0/work/mozilla-release/js/src/jsobj.cpp:91
2) p obj_ (notice the underscore): $1 = (JSObject *) 0x7f5f4a83a6a0
3) p obj_->dump(): $2 = void
From launcher terminal:

object 0x7f5f4a83a6a0
class 0x7f5f57aa1520 Proxy
flags:
proto <Proxy object at 0x7f5f4a847d40>
parent <Window object at 0x7f5f4a834060>
not native
slots:
   0 (reserved) = <unnamed function (:0) at 0x7f5f4a837040>
   1 (reserved) = 3,45963e-310
   2 (reserved) = undefined
   3 (reserved) = undefined
   4 (reserved) = undefined
   5 (reserved) = undefined

4) $3 = (js::Class *) 0x7f5f57aa1520 <js::FunctionProxyClass>
5) $4 = 0x7f5f578a3428 "Proxy"

How do you access about:crashes without being able to open the browser (even in safe mode)?
I guess there's no crash report with a startup segfault. :-(

Comment 17

[Bobby Holley (:bholley)]

Ok. A few more data points:

(1) What is the value of obj_->getClass()->ext.outerObject?

(2) What happens when you perform the same ->dump on js::UncheckedUnwrap(obj_, 0, 0)?

(3) What happens when you do |x/wa GetProxyHandler(obj_)|?

Comment 18

[Pierre Lejeune]

Here are the results:

1) $1 = (JSObjectOp) 0x7fb2ad561410

2) $2 = (JSObject *) 0x7fb2bb837040
object 0x7fb2bb837040
class 0x7fb2c899fe40 Function
flags: delegate
proto <Object at 0x7fb2bb836020>
parent <Window object at 0x7fb2bb834060>
properties:
    ((JSShape *) 0x7fb2bb8339e8) JSString* (0x7fb2bb924340) = jschar * (0x7fb2bb924350) = "constructor"
: slot 0 = <function Function at 0x7fb2bb8370c0>
    ((JSShape *) 0x7fb2bb833a88) JSString* (0x7fb2bb925080) = jschar * (0x7fb2bb925090) = "toSource"
: slot 1 = <function toSource at 0x7fb2bb837840>
    ((JSShape *) 0x7fb2bb833b00) JSString* (0x7fb2bb9250c0) = jschar * (0x7fb2bb9250d0) = "toString"
: slot 2 = <function toString at 0x7fb2bb837880>
    ((JSShape *) 0x7fb2bb833b28) JSString* (0x7fb2bb923800) = jschar * (0x7fb2bb923810) = "apply"
: slot 3 = <function apply at 0x7fb2bb8378c0>
    ((JSShape *) 0x7fb2bb833b50) JSString* (0x7fb2bb923860) = jschar * (0x7fb2bb923870) = "call"
: slot 4 = <function call at 0x7fb2bb837900>
    ((JSShape *) 0x7fb2bb833b78) JSString* (0x7fb2bb926320) = jschar * (0x7fb2bb926330) = "bind"
: slot 5 = <function bind at 0x7fb2bb837940>
    ((JSShape *) 0x7fb2bb833ba0) JSString* (0x7fb2bb9259c0) = jschar * (0x7fb2bb9259d0) = "isGenerator"
: slot 6 = <function isGenerator at 0x7fb2bb837980>

3) 0x7fb2c77a2ba0 <_ZN3xpc10XrayWaiverE>:	0xffffffffc89ad790

Comment 19

[Bobby Holley (:bholley)]

(In reply to Pierre Lejeune from comment #18)
> Here are the results:
> 
> 1) $1 = (JSObjectOp) 0x7fb2ad561410

Hm! That's not expected. The value should be 0x0. Can you try doing:

p *(obj_->getClass())

Comment 20

[Pierre Lejeune]

Here is the (lengthy) output:
$1 = {name = 0x7fbf4c6a3428 "Proxy", flags = 263712, addProperty = 0x7fbf4c196429 <JS_PropertyStub(JSContext*, JS::Handle<JSObject*>, JS::Handle<jsid>, JS::MutableHandle<JS::Value>)>, 
  delProperty = 0x7fbf4c196462 <JS_DeletePropertyStub(JSContext*, JS::Handle<JSObject*>, JS::Handle<jsid>, int*)>, 
  getProperty = 0x7fbf4c196429 <JS_PropertyStub(JSContext*, JS::Handle<JSObject*>, JS::Handle<jsid>, JS::MutableHandle<JS::Value>)>, 
  setProperty = 0x7fbf4c196444 <JS_StrictPropertyStub(JSContext*, JS::Handle<JSObject*>, JS::Handle<jsid>, int, JS::MutableHandle<JS::Value>)>, 
  enumerate = 0x7fbf4c196487 <JS_EnumerateStub(JSContext*, JS::Handle<JSObject*>)>, resolve = 0x7fbf4c19649a <JS_ResolveStub(JSContext*, JS::Handle<JSObject*>, JS::Handle<jsid>)>, 
  convert = 0x7fbf4c1964b1 <JS_ConvertStub(JSContext*, JS::Handle<JSObject*>, JSType, JS::MutableHandle<JS::Value>)>, finalize = 0x7fbf4c2cbc36 <proxy_Finalize(js::FreeOp*, JSObject*)>, checkAccess = 0x0, 
  call = 0x7fbf4c2cbd04 <proxy_Call(JSContext*, unsigned int, JS::Value*)>, hasInstance = 0x7fbf4c2cbcb2 <proxy_HasInstance(JSContext*, JS::HandleObject, JS::MutableHandleValue, JSBool*)>, 
  construct = 0x7fbf4c2cbdfb <proxy_Construct(JSContext*, unsigned int, JS::Value*)>, trace = 0x7fbf4c2cbad0 <proxy_TraceFunction(JSTracer*, JSObject*)>, ext = {outerObject = 0x7fbf31b61410, innerObject = 0x0, 
    iteratorObject = 0x0, isWrappedNative = false, weakmapKeyDelegateOp = 0x7fbf4c2cbb40 <proxy_WeakmapKeyDelegate(JSObject*)>}, ops = {
    lookupGeneric = 0x7fbf4c2ca64e <proxy_LookupGeneric(JSContext*, JS::HandleObject, JS::HandleId, JS::MutableHandleObject, js::MutableHandleShape)>, 
    lookupProperty = 0x7fbf4c2ca6ed <proxy_LookupProperty(JSContext*, JS::HandleObject, js::HandlePropertyName, JS::MutableHandleObject, js::MutableHandleShape)>, 
    lookupElement = 0x7fbf4c2ca79e <proxy_LookupElement(JSContext*, JS::HandleObject, uint32_t, JS::MutableHandleObject, js::MutableHandleShape)>, 
    lookupSpecial = 0x7fbf4c2ca86d <proxy_LookupSpecial(JSContext*, JS::HandleObject, js::HandleSpecialId, JS::MutableHandleObject, js::MutableHandleShape)>, 
    defineGeneric = 0x7fbf4c2ca91b <proxy_DefineGeneric(JSContext*, JS::HandleObject, JS::HandleId, JS::HandleValue, js::PropertyOp, js::StrictPropertyOp, unsigned int)>, 
    defineProperty = 0x7fbf4c2ca9db <proxy_DefineProperty(JSContext*, JS::HandleObject, js::HandlePropertyName, JS::HandleValue, js::PropertyOp, js::StrictPropertyOp, unsigned int)>, 
    defineElement = 0x7fbf4c2caa97 <proxy_DefineElement(JSContext*, JS::HandleObject, uint32_t, JS::HandleValue, js::PropertyOp, js::StrictPropertyOp, unsigned int)>, 
    defineSpecial = 0x7fbf4c2cab77 <proxy_DefineSpecial(JSContext*, JS::HandleObject, js::HandleSpecialId, JS::HandleValue, js::PropertyOp, js::StrictPropertyOp, unsigned int)>, 
    getGeneric = 0x7fbf4c2cac30 <proxy_GetGeneric(JSContext*, JS::HandleObject, JS::HandleObject, JS::HandleId, JS::MutableHandleValue)>, 
    getProperty = 0x7fbf4c2cac70 <proxy_GetProperty(JSContext*, JS::HandleObject, JS::HandleObject, js::HandlePropertyName, JS::MutableHandleValue)>, 
    getElement = 0x7fbf4c2cad21 <proxy_GetElement(JSContext*, JS::HandleObject, JS::HandleObject, uint32_t, JS::MutableHandleValue)>, 
    getElementIfPresent = 0x7fbf4c2cadf0 <proxy_GetElementIfPresent(JSContext*, JS::HandleObject, JS::HandleObject, uint32_t, JS::MutableHandleValue, bool*)>, 
    getSpecial = 0x7fbf4c2cae39 <proxy_GetSpecial(JSContext*, JS::HandleObject, JS::HandleObject, js::HandleSpecialId, JS::MutableHandleValue)>, 
    setGeneric = 0x7fbf4c2caee7 <proxy_SetGeneric(JSContext*, JS::HandleObject, JS::HandleId, JS::MutableHandleValue, JSBool)>, 
    setProperty = 0x7fbf4c2caf34 <proxy_SetProperty(JSContext*, JS::HandleObject, js::HandlePropertyName, JS::MutableHandleValue, JSBool)>, 
    setElement = 0x7fbf4c2cafe4 <proxy_SetElement(JSContext*, JS::HandleObject, uint32_t, JS::MutableHandleValue, JSBool)>, 
    setSpecial = 0x7fbf4c2cb0b2 <proxy_SetSpecial(JSContext*, JS::HandleObject, js::HandleSpecialId, JS::MutableHandleValue, JSBool)>, 
    getGenericAttributes = 0x7fbf4c2cb15f <proxy_GetGenericAttributes(JSContext*, JS::HandleObject, JS::HandleId, unsigned int*)>, 
    getPropertyAttributes = 0x7fbf4c2cb1de <proxy_GetPropertyAttributes(JSContext*, JS::HandleObject, js::HandlePropertyName, unsigned int*)>, 
    getElementAttributes = 0x7fbf4c2cb284 <proxy_GetElementAttributes(JSContext*, JS::HandleObject, uint32_t, unsigned int*)>, 
    getSpecialAttributes = 0x7fbf4c2cb348 <proxy_GetSpecialAttributes(JSContext*, JS::HandleObject, js::HandleSpecialId, unsigned int*)>, 
    setGenericAttributes = 0x7fbf4c2cb3eb <proxy_SetGenericAttributes(JSContext*, JS::HandleObject, JS::HandleId, unsigned int*)>, 
    setPropertyAttributes = 0x7fbf4c2cb487 <proxy_SetPropertyAttributes(JSContext*, JS::HandleObject, js::HandlePropertyName, unsigned int*)>, 
    setElementAttributes = 0x7fbf4c2cb52d <proxy_SetElementAttributes(JSContext*, JS::HandleObject, uint32_t, unsigned int*)>, 
    setSpecialAttributes = 0x7fbf4c2cb5f1 <proxy_SetSpecialAttributes(JSContext*, JS::HandleObject, js::HandleSpecialId, unsigned int*)>, 
    deleteProperty = 0x7fbf4c2cb703 <proxy_DeleteProperty(JSContext*, JS::HandleObject, js::HandlePropertyName, JSBool*)>, 
    deleteElement = 0x7fbf4c2cb7a9 <proxy_DeleteElement(JSContext*, JS::HandleObject, uint32_t, JSBool*)>, deleteSpecial = 0x7fbf4c2cb86d <proxy_DeleteSpecial(JSContext*, JS::HandleObject, js::HandleSpecialId, JSBool*)>, 
    enumerate = 0x0, thisObject = 0x0}, pad = '\000' <repeats 39 times>, static NON_NATIVE = 262144}

Comment 21

[Bobby Holley (:bholley)]

This is getting weirder and weirder.

At the point of crash, verify that:

p js::FunctionProxyClass.ext.outerObject

Gives you a non-zero answer. This value should be initialized to zero (see the definition in js/src/jsproxy.cpp), so presumably the memory is being clobbered (or your compiler is bonkers).

First, let's break really early in startup to see what the value is. If you launch firefox in a debugger:

gdb --args ./firefox -P myProfile

then type:

b xpcModuleCtor

It will warn you that the symbol is unknown (because the libraries haven't been loaded yet). This is fine. Go ahead and continue, and run the program. It should resolve the breakpoint.

Very early in startup (before your crash), you should hit the xpcModuleCtor breakpoint. At this point, examine the data again:

p js::FunctionProxyClass.ext.outerObject

This should be zero. Is it? If it is, it's becoming non-zero between this moment and the moment when your browser crashes (if not, something is seriously messed up, probably your compiler). To figure out when it becomes nonzero, you can use a watchpoint:

watch js::FunctionProxyClass.ext.outerObject

Then continue the program. The watchpoint should trigger, and that will tell you who's clobbering the memory.

Comment 22

[Pierre Lejeune]

p js::FunctionProxyClass.ext.outerObject gives $2 = (JSObjectOp) 0x7fbf31b61410, non-zero answer
At breakpoint, p js::FunctionProxyClass.ext.outerObject gives $1 = (JSObjectOp) 0x0, zero

At watchpoint, I get:
Hardware watchpoint 2: js::FunctionProxyClass.ext.outerObject

Old value = (JSObjectOp) 0x0
New value = (JSObjectOp) 0x7fffdb361410 <fun_hasInstance(JSContext*, JSObject*, js::Value const*, int*)>
0x00007fffdb3f39c0 in _GLOBAL__sub_I_jsproxy.cpp () from /usr/lib64/../lib64/libmozjs185.so.1.0

Looks like firefox is compiled against spidermonkey (in one way or another). Weird.

Comment 23

[Bobby Holley (:bholley)]

And what is the backtrace when you hit the watchpoint?

Comment 24

[Bobby Holley (:bholley)]

This seems eerily related to bug 870423 - almost as if some third-party code were trying to monkey-patch a fix for that bug, but hitting the wrong memory offset...

Comment 25

[Pierre Lejeune]

Attachment: GDB backtrace at watchpoint

Reply to comment #23

Comment 26

[Mike Hommey [:glandium]]

So, you're hitting something similar to bug 809430, but that was fixed in Firefox 23.
It would be useful if you could test a mozilla.org Firefox 24 build to see whether this happens with it or not. If it doesn't, that would suggest some toolchain glitch, or a build system problem.
If it does, then the fix for bug 809430 might not have been enough, or something else is fishy.

Now, seeing the strace output, I think I know what's going on: xulrunner looks like it is built --with-shared-js, which builds the js engine as a shared library. So xulrunner loads that shared js engine, which has symbol versions for that engine version, and then gtk loads a module that i guess loads libproxy (like in bug 809430), which in turn loads the system libmozjs185, which has symbol versions for that engine version... except iirc, libmozjs185 is the engine from Firefox 24, so both shared libraries would have the same version. So the result is the same as if there were no symbol versioning: symbol clashes.

But that's only a guess.

Comment 27

[Pierre Lejeune]

Hello Mike,

Problem looks the same with mozilla.org build.

Here is the backtrace from gdb:
#0  0x00007fffdcf61410 in ?? ()
#1  0x00007ffff46337f5 in JS_ObjectToOuterObject(JSContext*, JSObject*) () from /home/pierre/firefox/libxul.so
#2  0x00007ffff3aeee5c in ?? () from /home/pierre/firefox/libxul.so
#3  0x00007ffff45f1716 in ?? () from /home/pierre/firefox/libxul.so
#4  0x00007ffff45f1eb3 in ?? () from /home/pierre/firefox/libxul.so
#5  0x00007ffff45d004d in JS_WrapObject(JSContext*, JSObject**) () from /home/pierre/firefox/libxul.so
#6  0x00007ffff3aef50c in ?? () from /home/pierre/firefox/libxul.so
#7  0x00007ffff3aef797 in ?? () from /home/pierre/firefox/libxul.so
#8  0x00007ffff3aef809 in ?? () from /home/pierre/firefox/libxul.so
#9  0x00007ffff465271d in ?? () from /home/pierre/firefox/libxul.so
#10 0x00007ffff4652909 in ?? () from /home/pierre/firefox/libxul.so
#11 0x00007ffff4588922 in ?? () from /home/pierre/firefox/libxul.so
#12 0x00007ffff4590234 in ?? () from /home/pierre/firefox/libxul.so
#13 0x00007ffff45859fc in ?? () from /home/pierre/firefox/libxul.so
#14 0x00007ffff4590793 in ?? () from /home/pierre/firefox/libxul.so
#15 0x00007ffff45d40b4 in JS_CallFunctionValue(JSContext*, JSObject*, JS::Value, unsigned int, JS::Value*, JS::Value*) () from /home/pierre/firefox/libxul.so
#16 0x00007ffff3ac658f in ?? () from /home/pierre/firefox/libxul.so
#17 0x00007ffff3ac2eeb in ?? () from /home/pierre/firefox/libxul.so
#18 0x00007ffff4064519 in ?? () from /home/pierre/firefox/libxul.so
#19 0x00007ffff4063a43 in ?? () from /home/pierre/firefox/libxul.so
#20 0x00007fffdcf8e0c0 in ?? ()
#21 0x00007fffdd3e6ee0 in ?? ()
#22 0x00007fffdce74e88 in ?? ()
#23 0x00007fffdd3e6ee0 in ?? ()
#24 0x00007fffdd262d40 in ?? ()
#25 0x00007fffffff8260 in ?? ()
#26 0x0000000000000000 in ?? ()

Comment 28

[Bobby Holley (:bholley)]

Ok, so for Pierre, a decent workaround is just to try compiling Beta (25), which should have different symbol versions.

glandium, how do we make sure this doesn't happen again?

Comment 29

[Mike Hommey [:glandium]]

(In reply to Bobby Holley (:bholley) from comment #28)
> Ok, so for Pierre, a decent workaround is just to try compiling Beta (25),
> which should have different symbol versions.

Another workaround is not to build with --with-shared-js.

> glandium, how do we make sure this doesn't happen again?

We make non standalone shared library js builds have different symbol versions. Which would need to be done after bug 927073, since that touches that too.

Comment 30

[Mike Hommey [:glandium]]

(In reply to Mike Hommey [:glandium] from comment #29)
> We make non standalone shared library js builds have different symbol
> versions. Which would need to be done after bug 927073, since that touches
> that too.

... which is already the case...

And now i realize i was half right... libmozjs185 is from 17, not 24... Which make things a little worse as for the conflict. But it shouldn't really happen.

Pierre, could you attach the output of running the mozilla.org firefox with the LD_DEBUG environment variable set to "bindings"?

Comment 31

[Pierre Lejeune]

Attachment: firefox with LD_DEBUG="bindings"

Reply to comment #30

Comment 32

[Wayne Mery (:wsmwk)]

(In reply to Pierre Lejeune from comment #31)
> Created attachment 824785 [details]
> firefox with LD_DEBUG="bindings"
> 
> Reply to comment #30

Comment 33

[Wayne Mery (:wsmwk)]

Pierre wrote he no longer crashes