557747 - SkipList of ___TERMINATING_DUE_TO_UNCAUGHT_EXCEPTION___ and _NSRaiseError regressed

Status: VERIFIED FIXED

(Socorro :: General, task)

Description

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

So, suddenly this week I'm seeing the evil, useless ___TERMINATING_DUE_TO_UNCAUGHT_EXCEPTION___ signature again on crash-stats: https://crash-stats.mozilla.com/report/list?version=Camino%3A2.0.2&build_id=&query_search=signature&query_type=exact&query=&date=&range_value=2&range_unit=weeks&process_type=all&plugin_field=&plugin_query_type=&plugin_query=&do_query=&signature=___TERMINATING_DUE_TO_UNCAUGHT_EXCEPTION___&missing_sig=&page=1

This was set to be discarded/skipped entirely in bug 526046; I'm not sure why it's back :/

Can someone please add this back to the irrelevant frames (and double-check the whole list in bug 526046 comment 9 to make sure they are all still configured properly)? Thanks.

Comment 1

[Laura Thomson :laura]

Added to 1.7

Comment 2

[Laura Thomson :laura]

Lars, can you take a look at this?

Comment 3

[K Lars Lohn [:lars] [:klohn]]

"___TERMINATING_DUE_TO_UNCAUGHT_EXCEPTION___" should be in the irreleventSignatures regular expression in the processor configuration.

Comment 4

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

https://crash-stats.mozilla.com/query/query?product=ALL&version=ALL%3AALL&platform=mac&range_value=1&range_unit=weeks&date=05%2F05%2F2010+17%3A53%3A56&query_search=signature&query_type=exact&query=___TERMINATING_DUE_TO_UNCAUGHT_EXCEPTION___&build_id=&process_type=all&do_query=1

shows that there were ~350 crashes in the past week where ___TERMINATING_DUE_TO_UNCAUGHT_EXCEPTION___ was the signature, so *something* is broken somewhere along the way :(

All of the ones I looked at should have been given a signature of
"[@ objc_exception_throw | -[NSException raise] | registerNatives ]" (aka the crash from bug 525566).

(I'm not sure if it's related, but I also noticed recently that __ in signature names were ignored in searches (bug 563451); I have no idea if the same/similar code is doing the regexp parsing as is doing the "contains" search.)

Comment 5

[Laura Thomson :laura]

Aravind, can you please change the processor configuation as in comment 3?

Comment 6

[Aravind Gottipati [:aravind]]

We already have this in the list.. Here is the current config value.

irrelevantSignatureRegEx = cm.Option()
irrelevantSignatureRegEx.doc = 'a regular expression matching frame signatures that should be ignored when generating an overall signature'
irrelevantSignatureRegEx.default = '@0x[0-9a-fA-F]{2,}|@0x[1-9a-fA-F]|RaiseException|_CxxThrowException|Abort|___TERMINATING_DUE_TO_UNCAUGHT_EXCEPTION___|_NSRaiseError|mozilla::ipc::RPCChannel::Call\(IPC::Message\*, IPC::Message\*\)'

Let me know if you need any more information from me.

Comment 7

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

FYI, this is still a problem:

https://crash-stats.mozilla.com/report/list?product=Firefox&platform=mac&query_search=signature&query_type=exact&query=&date=02%2F04%2F2011%2013%3A01%3A08&range_value=1&range_unit=days&hang_type=any&process_type=any&plugin_field=&plugin_query_type=&plugin_query=&do_query=1&admin=1&signature=___TERMINATING_DUE_TO_UNCAUGHT_EXCEPTION___

http://code.google.com/p/socorro/source/browse/trunk/scripts/config/processorconfig.py.dist#100 

irrelevantSignatureRegEx = cm.Option()
irrelevantSignatureRegEx.doc = 'a regular expression matching frame signatures that should be ignored when generating an overall signature'
irrelevantSignatureRegEx.default = '|'.join([
  '@0x[0-9a-fA-F]{2,}',
  '@0x[1-9a-fA-F]',
  'RaiseException',
  '_CxxThrowException',
  'mozilla::ipc::RPCChannel::Call\(IPC::Message\*, IPC::Message\*\)'
  ])

That list looks different from the one Aravind quoted in comment 6; at the very least, the ___TERMINATING_DUE_TO_UNCAUGHT_EXCEPTION___ and _NSRaiseError entries added in bug 526046 are definitely no longer present in SVN (even though they were in the list Aravind quoted when this bug first appeared).

Comment 8

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Here's the corresponding "unskipped _NSRaiseError" reports link:

https://crash-stats.mozilla.com/report/list?product=Firefox&platform=mac&query_search=signature&query_type=exact&query=&date=02%2F04%2F2011%2013%3A01%3A08&range_value=1&range_unit=days&hang_type=any&process_type=any&plugin_field=&plugin_query_type=&plugin_query=&do_query=1&admin=1&signature=_NSRaiseError

Comment 9

[Laura Thomson :laura]

In prod:
irrelevantSignatureRegEx.default = '|'.join([
  '@0x[0-9a-fA-F]{2,}',
  '@0x[1-9a-fA-F]',
  'RaiseException',
  '_CxxThrowException',
  'mozilla::ipc::RPCChannel::Call\(IPC::Message\*, IPC::Message\*\)'
  ])

Comment 10

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Can we get these two frames added back in SVN and then tested and pushed to production at some point, since they're clearly missing now?

Hopefully that will fix it, and if not, at least we'll be no worse off than we were in May of last year/comment 6?

Comment 11

[Robert Helmer [:rhelmer]]

(In reply to comment #10)
> Can we get these two frames added back in SVN and then tested and pushed to
> production at some point, since they're clearly missing now?
> 
> Hopefully that will fix it, and if not, at least we'll be no worse off than we
> were in May of last year/comment 6?

Sorry, my eyes are starting to glaze over from looking at these regexes :) 

Can you confirm the two you need are:

___TERMINATING_DUE_TO_UNCAUGHT_EXCEPTION___
_NSRaiseError

Comment 12

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

(In reply to comment #11)
> Sorry, my eyes are starting to glaze over from looking at these regexes :) 

Mine too ;)

> Can you confirm the two you need are:
> 
> ___TERMINATING_DUE_TO_UNCAUGHT_EXCEPTION___
> _NSRaiseError

Yes, it's those two, added to "irrelevantSignatureRegEx.default"

Thanks again!

Comment 13

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Oops, meant to add/reiterate, these two are not as important as bug 631633, so if this bug doesn't make this weekend's push, that's OK (just as long as it makes it at some point) :)

Comment 14

[Laura Thomson :laura]

Reassigning to lars; he's back from vacation and rhelmer is out.

Comment 15

[K Lars Lohn [:lars] [:klohn]]

IT request created to carry out this change.  Bug 632534

Comment 16

[K Lars Lohn [:lars] [:klohn]]

the changes for Comment #12 were completed on Wednesday, Feb 9 at about 8:06am pst.

I think that wraps it up for this bug.  If you detect any anomalies, please reopen this bug or file a new one.

Comment 17

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

I think this looks good; it appears that those crashes had been appearing fairly regularly at a rate of ~1 every couple of hours, and it doesn't look like there have been any since about 8am PST :-)

I'll keep an eye on it to make sure, but I do think all is finally well here ;)  Thanks!