1147556 - Missing an operator new symbol for msvcr120.dll

Status: NEW

(Toolkit :: Crash Reporting, defect)

Description

[Jeff Muizelaar [:jrmuizel]]

This crash report is missing symbols:
https://crash-stats.mozilla.com/report/index/77264948-764b-44dd-9363-573bf2150325

Comment 1

[(not currently active) Ted Mielczarek]

It's not missing symbols, the stack frame would be colored red if it was, and you can see frame 1 has symbols. I'm not sure why that frame is not symbolized, but if you look at the raw dump you can see that it has "trust": "cfi", so we didn't get there by accident. I'd say look at the CRT source and see what calls there?

Comment 2

[Robert Kaiser]

FWIW, we have a similar issue with bug 1102612.

Comment 3

[(not currently active) Ted Mielczarek]

So looking at the CRT source, frame 3 is:
return (_Allocate(_Count, (pointer)0));

where _Allocate is an inline function that calls ::operator new and if that fails calls _XBad_alloc (presumably where we're winding up). _XBad_alloc calls _THROW_NCEE(_XSTD bad_alloc, _EMPTY_ARGUMENT), which just expands out to "throw std::bad_alloc()", which is how we get to frame 2.

Comment 4

[(not currently active) Ted Mielczarek]

Loading this in WinDBG gives me:
0099ba80 6bd44388 msvcr120!operator new+0x50
for that frame.

Running dump_syms on the PDB and grepping for "operator new" shows me:
PUBLIC ee11 0 operator new(unsigned int)
PUBLIC 11683 0 operator new[](unsigned int)
PUBLIC a6179 0 operator new(unsigned int,int,char const *,int)
PUBLIC a6187 0 operator new[](unsigned int,int,char const *,int)

None of which are even close to that address.

Asking WinDBG about operator new symbols, it tells me:
0:000> x /v msvcr120!operator new
pub func   6c36ee11             0 msvcr120!operator new (<no parameter info>)
pub func   6c406179             0 msvcr120!operator new (<no parameter info>)

Which are the two "operator new" PUBLIC lines that dump_syms finds (ee11 and a6179). It doesn't list the one it tells me this frame is in!

If I look at that frame:
0:000> .frame /r 2
02 0099ba80 6bd44388 msvcr120!operator new+0x50
eax=0099b9c8 ebx=00000001 ecx=00000003 edx=00000000 esi=00369f10 edi=779547d0
eip=6c3bda6a esp=0099ba68 ebp=0099ba80 iopl=0         nv up ei pl nz ac po nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00200212
msvcr120!operator new+0x50:

and then ask it what's at the instruction pointer:
0:000> dt -v 0x6c3bda6a
Got symbol operator new from address.
operator new
Symbol  not found.

...it finds it *somewhere*, but I don't know where. Maybe dmajor can apply his WinDBG skills to figure something out here.

Comment 5

[(Away)]

So apparently this -bda6a address is actually part of the -6ee11 function (run "uf" on the 6ee11 address to see). That failed-allocation handler is hopefully rare, so I wouldn't be surprised if PGO moved it to some faraway cold page. The only weird thing is that it gets given the name "operator new+0x50" which isn't true at all. Would that throw off our symbol code somehow?

Comment 6

[(Away)]

It looks like the offsets are screwy even in the non-displaced parts of that function:

msvcr120!operator new:
6c36ee11 55              push    ebp
6c36ee12 8bec            mov     ebp,esp
6c36ee14 83ec10          sub     esp,10h

msvcr120!operator new+0x15:
6c36ee17 ff7508          push    dword ptr [ebp+8]

By my count that oughta be +0x6 not +0x15!

Comment 7

[(not currently active) Ted Mielczarek]

The counting is weird there, but the only thing Breakpad does is take the instruction pointer, figure out which module contains it, subtract out the base address, and look for a FUNC or PUBLIC record that covers that offset, so it's looking for msvcr120 + 0x5da69 and not finding anything there.

Breakpad's dump_syms does have some code that tries to find the PGOed cold blocks and annotate them properly (I wrote it a while ago), maybe that's not working properly?
https://code.google.com/p/google-breakpad/source/browse/trunk/src/common/windows/pdb_source_line_writer.cc#443

Comment 8

[(not currently active) Ted Mielczarek]

Also, this is inside the CRT, so I would be kind of surprised to find that it would be compiled with PGO!

Comment 9

[(not currently active) Ted Mielczarek]

I'd just like to figure out how WinDBG figures this out and make Breakpad able to do that.