629959 - Missing symbols generate bad signatures for crashes at [@ xul.dll@0xccd20] [@ xul.dll@0xccd28 ]

Status: RESOLVED DUPLICATE of bug 628731

(Firefox :: General, defect)

Description

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

Stacks are garbage, virtually no symbols.  These started on the 1/29 nightlies.  I've hit this crash several times today.

Example report: https://crash-stats.mozilla.com/report/index/e4a16e37-7817-46ca-b663-099ea2110129

Comment 1

[[e-mail]]

I am having the same crashes a lot with the same nigtlies on Windows. I can get it to crash 100% of the time by visiting <http://www.roku.com/roku-products>. It seems to be JavaScript related since it only crashes when I allow scripts on that page in NoScript.

Comment 2

[(mostly gone) XtC4UaLL [:xtc4uall]]

A Windbg Log would be nice i guess.
https://developer.mozilla.org/en/How_to_get_a_stacktrace_with_WinDbg

Comment 3

[[e-mail]]

Attachment: WinDbg log

Created with 20110130 nightly on Windows 7 x64

Comment 4

[chris hofmann]

(In reply to comment #0)
> Stacks are garbage, virtually no symbols.  These started on the 1/29 nightlies.
>  I've hit this crash several times today.
> 
> Example report:
> https://crash-stats.mozilla.com/report/index/e4a16e37-7817-46ca-b663-099ea2110129

do we think this is a new crash, or just missing symbols as the result of a build process failure.  Kyle did you have any crashes with jemalloc and nspr on the stack in similar locations?   looks like atleast mozcrt19.dll and nspr.dll symbols are making to the socorro server and are gettting resolved in processing.  probably need build team folks looking at this too.

0 	xul.dll 	xul.dll@0xccd28 	
1 		@0x31002f 	
2 	xul.dll 	xul.dll@0x50bf42 	
3 	xul.dll 	xul.dll@0x5568d8 	
4 	xul.dll 	xul.dll@0xeaf53 	
5 	mozcrt19.dll 	arena_dalloc_small 	obj-firefox/memory/jemalloc/crtsrc/jemalloc.c:4209
6 	xul.dll 	xul.dll@0x4be45c 	
7 	xul.dll 	xul.dll@0x10d0a5 	
8 	xul.dll 	xul.dll@0x5a3f13 	
9 	xul.dll 	xul.dll@0x4be44f 	
10 	xul.dll 	xul.dll@0x99937a 	
11 	xul.dll 	xul.dll@0x9994f9 	
12 	xul.dll 	xul.dll@0x99937a 	
13 	xul.dll 	xul.dll@0xa90b7 	
14 	xul.dll 	xul.dll@0xcd8bb7 	
15 	xul.dll 	xul.dll@0xcd8bb7 	
16 	xul.dll 	xul.dll@0x89c33 	
17 	xul.dll 	xul.dll@0x94651 	
18 	xul.dll 	xul.dll@0xe924c 	
19 	xul.dll 	xul.dll@0x94824 	
20 	nspr4.dll 	PR_ExitMonitor 	nsprpub/pr/src/threads/prmon.c:132
21 	xul.dll 	xul.dll@0x943bb 	
22 	xul.dll 	xul.dll@0xd15f2 	
23 	xul.dll 	xul.dll@0xa5d8d 	
24 	xul.dll 	xul.dll@0xb20a43 	
25 	xul.dll 	xul.dll@0xb29ddb 	
26 	xul.dll 	xul.dll@0x278e5c 	
27 	xul.dll 	xul.dll@0x278e45 	
28 	mozcrt19.dll 	_VEC_memzero 	
29 	xul.dll 	xul.dll@0x354f3d 	
30 	firefox.exe 	firefox.exe@0x1bb7 	
31 	ntdll.dll 	WinSqmSetIfMaxDWORD 	
32 	ntdll.dll 	_RtlUserThreadStart 	
33 	firefox.exe 	firefox.exe@0x186f 	
34 	firefox.exe 	firefox.exe@0x186f

Comment 5

[chris hofmann]

in addtion to possible build process problems I wonder if the processor logs could tell us of problems on the socorro side.  this is #3 crash on mozilla-central now so we need to know if this is a new problem, or just an old problem masked by lack of symbol resolution.

Comment 6

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

Based on comment 3 this looks like Bug 629912, but the lack of symbols here is a bug in its own right.

Comment 7

[[e-mail]]

I am sorry. Th Ruku page crash I mentioned in comment 1 is actually a different crash than this one. Please ignore it. The log provided is comment 3 IS for this one. It has the same signature.

Comment 8

[(mostly gone) XtC4UaLL [:xtc4uall]]

(In reply to comment #3)
Per that Stacktrace the Crashes point to Bug 629912.

So this should be left opened about the missing Symbols, right?

Comment 9

[chris hofmann]

(In reply to comment #8)
> (In reply to comment #3)
> Per that Stacktrace the Crashes point to Bug 629912.
> 
> So this should be left opened about the missing Symbols, right?

sounds like a plan.  if any of the missing symbol problems are hiding new crashes lets try and get new bugs on file for those.

changed the title for this bug.

Comment 10

[chris hofmann]

john, can you assign some one to take a look at this?  if the builds are failing to upload symbols we need to know why.  

if symbols are uploading ok, then reassign to laura so the socorro team can investigate possible problems in the processor.

one way or another we need to get this fixed soon or open up risk for shipping crash regressions in firefox 4 betas.

Comment 11

[Carsten Book [:Tomcat]]

cc'ing ted

Comment 12

[(not currently active) Ted Mielczarek]

This is probably fallout from the Socorro migration. See bug 628731. Symbols are still being uploaded to a server in SJC, and rsync'ed to PHX, which takes a while.

Comment 13

[chris hofmann]

yeah that would explain it.  duping to that bug for now, and we can un-dup if there are other reasons that turn up.

Comment 14

[chris hofmann]

it sounds like good progress getting sybmols to PHX in a more timely manner.

but, reopening and assigning back to john,

we might have symbol problem on a specific build as well.

for just incoming reports processed yesterday it looks like no symbols were around for builds at 2011 01 21 16:13:58

nearly all the top crashes and a lot of the long tail mentions that buildid specifically.

do we still have symbols around for that build, and can they get uploads and sync'ed to phx?

count release  buildid     sig

  21 4.0b10 20110121161358 xul.dll@0x35dfad
  19 4.0b10 20110121161358 xul.dll@0x2b0e3
  19 4.0b10 20110121161358 xul.dll@0x11e07c
  11 4.0b10 20110121161358 xul.dll@0x178fed
   9 4.0b10 20110121161358 xul.dll@0x4879ec
   8 4.0b10 20110121161358 xul.dll@0x2d1baa
   6 4.0b10 20110121161358 xul.dll@0x354c3a
   5 4.0b10 20110121161358 xul.dll@0xcdc2a0
   5 4.0b10 20110121161358 PL_DHashTableOperate | xul.dll@0xb2090f
   4 4.0b10 20110121161358 xul.dll@0xb67fe0
   4 4.0b10 20110121161358 xul.dll@0xa33630
   3 4.0b10 20110121161358 xul.dll@0xcdbeec
   3 4.0b10 20110121161358 xul.dll@0xc33b88
   3 4.0b10 20110121161358 xul.dll@0x36093e
   3 4.0b10 20110121161358 xul.dll@0x3559cd
   3 4.0b10 20110121161358 nsCOMPtr_base::assign_from_qi(nsQueryInterface, nsID const&) | xul.dll@0xb40d7b
   3 4.0b10 20110121161358 nsCOMPtr_base::assign_from_qi(nsQueryInterface, nsID const&) | xul.dll@0xb2f9f7
   2 4.0b10pre 20110111030357 xul.dll@0xea755
   2 4.0b10 20110121161358 xul.dll@0xe069d
   2 4.0b10 20110121161358 xul.dll@0xcd7c10
   2 4.0b10 20110121161358 xul.dll@0xcd7c0f
   2 4.0b10 20110121161358 xul.dll@0xc94074
   2 4.0b10 20110121161358 xul.dll@0xc33db7
   2 4.0b10 20110121161358 xul.dll@0xc33b8a
   2 4.0b10 20110121161358 xul.dll@0xbd0503
   2 4.0b10 20110121161358 xul.dll@0xbb9ccc
   2 4.0b10 20110121161358 xul.dll@0xba368c
   2 4.0b10 20110121161358 nsCOMPtr_base::assign_from_qi(nsQueryInterface, nsID const&) | xul.dll@0xbc4153
   2 4.0b10 20110121161358 nsCOMPtr_base::assign_from_qi(nsQueryInterface, nsID const&) | xul.dll@0xb3d547
   2 4.0b10 20110121161358 nsCOMPtr_base::assign_from_qi(nsQueryInterface, nsID const&) | xul.dll@0xb3c133
   2 4.0b10 20110121161358 nsCOMPtr_base::assign_from_qi(nsQueryInterface, nsID const&) | xul.dll@0xb2fa07
   2 4.0b10 20110121161358 nsCOMPtr_base::assign_from_qi(nsQueryInterface, nsID const&) | xul.dll@0xb29b5b

Comment 15

[Chris AtLee [:catlee]]

So that's the official b10 build for windows...tracking down where the symbols ended up.

Comment 16

[chris hofmann]

ok, that makes sense. that list is just b10 crashes.

so this may be more just a case that symbols are there, but processing was unable to resolve on a set of bad reports.  here are some reports as examples out of the list from above.

   1 4.0b10 20110121161358 xul.dll@0xbfe32a 
http://crash-stats.mozilla.com/report/index/d2954b5f-cf18-4acf-9119-fec382110130

   1 4.0b10 20110121161358 xul.dll@0xbd0503 
http://crash-stats.mozilla.com/report/index/f11eb65b-52f4-4e0d-b82d-05ed32110130

   1 4.0b10 20110121161358 xul.dll@0xbd0503 
http://crash-stats.mozilla.com/report/index/84892833-7496-4e9f-b0c6-6fa7f2110130

   1 4.0b10 20110121161358 xul.dll@0xbc0144 
http://crash-stats.mozilla.com/report/index/8c846874-9c4a-4eb4-a266-8fd412110130

   1 4.0b10 20110121161358 xul.dll@0xbb9ccc 
http://crash-stats.mozilla.com/report/index/f25965c9-a9ff-4a45-bf65-e459d2110130

   1 4.0b10 20110121161358 xul.dll@0xbb9ccc 
http://crash-stats.mozilla.com/report/index/b45d4a7d-8ca2-425b-9071-917752110130

   1 4.0b10 20110121161358 xul.dll@0xba4500 
http://crash-stats.mozilla.com/report/index/712174bd-28dc-4259-a2c1-cf23f2110130

   1 4.0b10 20110121161358 xul.dll@0xba368c 
http://crash-stats.mozilla.com/report/index/fb398958-7738-415f-a98f-85db72110130

   1 4.0b10 20110121161358 xul.dll@0xba368c 
http://crash-stats.mozilla.com/report/index/4bbcb6b2-41d9-48ea-8346-89eec2110130

   1 4.0b10 20110121161358 xul.dll@0xb982c5 
http://crash-stats.mozilla.com/report/index/9202aea7-c2b9-4429-a097-d2d552110130

   1 4.0b10 20110121161358 xul.dll@0xb931c5 
http://crash-stats.mozilla.com/report/index/bbc2f393-f775-4f16-8759-42e8d2110130

   1 4.0b10 20110121161358 xul.dll@0xb93120 
http://crash-stats.mozilla.com/report/index/3cb5a6a6-20cd-46e2-8a40-228c12110130

   1 4.0b10 20110121161358 xul.dll@0xb8cd47 
http://crash-stats.mozilla.com/report/index/f1832cb6-bebc-411e-9649-5711d2110130

Comment 17

[Aki Sasaki (not active)]

Attachment: 4.0b10 win32 upload symbols log

Might be helpful ?

Comment 18

[(not currently active) Ted Mielczarek]

choffman: if you look at those crash reports they very clearly have symbols! It's just the top frame is crashing at a weird offset in xul.dll. I don't know if they're all the same crash, but please file a separate bug.