963496 - runtime crashes being reported in Firefox product on Breakpad

Status: RESOLVED INVALID

(Firefox Graveyard :: Webapp Runtime, defect)

Description

[Myk Melez [:myk] [@mykmelez]]

bsmedberg thinks this crash report is for a runtime crash:

https://crash-stats.mozilla.com/report/index/266dbe5e-2d91-4d69-9a82-48ccb2140120

But it was reported in the Firefox product on Breakpad instead of the WebappRuntime product:

https://crash-stats.mozilla.com/home/products/WebappRuntime

The last nightly crash in the WebappRuntime product on Breakpad is this one from 2013-12-05:

https://crash-stats.mozilla.com/report/index/db37c12d-66bc-4999-96ae-ba6ed2131205

So it seems like runtime crashes are now being reported in the wrong place.  It isn't clear whether the problem is in the runtime or in Breakpad, so I'm filing this in the runtime component pending further investigation and discovery.

Comment 1

[Robert Kaiser]

It's pretty surely not in Breakpad. The bug is either the runtime sending the wrong product name, or Socorro not doing a rewrite to put it into a different product bucket. I somewhat expect the former, and I also think that the ADI pings might not be sent with the correct data as well.

Comment 2

[Benjamin Smedberg]

Further investigation shows that the crash in comment 0 is not a runtime crash.

However, I tried to make the runtime crash and I couldn't using any of my injection tools. It appeared to silently eat the crash and the app kept running. So we definitely need to make sure that it works and have automated testing for it.

Kairo, I'm not sure what you mean by ADI data. Since these are webapprt, they aren't going to have separate ADI, just Firefox.

Comment 3

[Myk Melez [:myk] [@mykmelez]]

(In reply to Benjamin Smedberg  [:bsmedberg] from comment #2)
> However, I tried to make the runtime crash and I couldn't using any of my
> injection tools. It appeared to silently eat the crash and the app kept
> running. So we definitely need to make sure that it works and have automated
> testing for it.

fzzzy was having this problem over in bug 759505.

Comment 4

[Marco Castelluccio [:marco]]

I've made the runtime crash. Minidump files are created, but the crash reporter isn't executed. I'll investigate further tomorrow.

Comment 5

[Marco Castelluccio [:marco]]

Attachment: report.html

The minidumps are created in the profile directory, but not in the Crash Reports directory.
If I move them manually to Crash Reports/pending, I can see them in about:crashes and upload them. The attached file is the crash report (using a local report URL).

The product ID is correct.

Comment 6

[Robert Kaiser]

(In reply to Marco Castelluccio [:marco] from comment #5)
> The minidumps are created in the profile directory, but not in the Crash
> Reports directory.

That sounds strange, Benjamin, any idea what's going on there?

(In reply to Benjamin Smedberg  [:bsmedberg] from comment #2)
> Kairo, I'm not sure what you mean by ADI data. Since these are webapprt,
> they aren't going to have separate ADI, just Firefox.

Actually, we do get separate ADI from metrics.

Comment 7

[(not currently active) Ted Mielczarek]

The crash reporter client is responsible for moving crashes from the $profile/minidumps directory to the Crash Reports/pending directory. If the client isn't being executed, that would cause what you're seeing.

Comment 8

[Benjamin Smedberg]

Yeah, are we launching the crash reporter? We calculate the path of the crash reporter app here: http://mxr.mozilla.org/mozilla-central/source/toolkit/crashreporter/nsExceptionHandler.cpp#882

And we do the actual launching here: http://mxr.mozilla.org/mozilla-central/source/toolkit/crashreporter/nsExceptionHandler.cpp#625

Comment 9

[Marco Castelluccio [:marco]]

Looks like the culprit is the DllBlocklist_WriteNotes call, in particular in DllBlockSet::Write the lock isn't acquired.
If I comment out the call, the crash reporter is executed.

Comment 10

[Benjamin Smedberg]

The DllBlocklist issue is bug 958344. The deadlock as a result of that is bug 958348. That was already all diagnosed in bug 958108.

Comment 11

[Marco Castelluccio [:marco]]

So, given that the webapprt is reporting the correct product ID, I guess we can close this bug.