430042 - crash-stats aborting request when trying to generate week's report for a signature

Status: RESOLVED FIXED

(Socorro :: General, task)

Description

[Mike Shaver (:shaver -- probably not reading bugmail closely)]

I'm trying to see what's been happening with js_GetGCThingTraceKind crashes recently, since they spiked at some point after b5, but this URL just gives me "server reset connection" after a long wait.

http://crash-stats.mozilla.com/report/list?range_unit=weeks&query_search=signature&query_type=contains&signature=js_GetGCThingTraceKind&query=js_GetGCThingTraceKind&range_value=1

It worked before (I took it from a bugzilla comment), though I don't now how recently.

Comment 1

[Aravind Gottipati [:aravind]]

Not sure if why this is happening, but I see this error in the logs.

Report['dump']: <type 'unicode'>
Error - exceptions.AttributeError: 'NoneType' object has no attribute 'strftime'
URL: http://localhost:5000/status
File '/home/reporter/python_extras/lib/Paste-1.4-py2.4.egg/paste/exceptions/errormiddleware.py', line 135 in __call__
  app_iter = self.application(environ, start_response)
File '/home/reporter/python_extras/lib/AuthKit-0.3.0pre5-py2.4.egg/authkit/authenticate/multi.py', line 45 in __call__
  app_iter = app(environ, start_response)
File '/home/reporter/python_extras/lib/AuthKit-0.3.0pre5-py2.4.egg/authkit/authenticate/multi.py', line 29 in app
  return self.default(environ, find)
File '/home/reporter/python_extras/lib/AuthKit-0.3.0pre5-py2.4.egg/authkit/authenticate/__init__.py', line 419 in __call__
  return self.app(environ, start_response)
File '/home/reporter/python_extras/lib/AuthKit-0.3.0pre5-py2.4.egg/authkit/authenticate/multi.py', line 86 in __call__
  return self.app(environ, authkit_start_response)
File '/home/reporter/python_extras/lib/Paste-1.4-py2.4.egg/paste/httpexceptions.py', line 633 in __call__
  self.send_http_response, catch=HTTPException)
File '/home/reporter/python_extras/lib/Paste-1.4-py2.4.egg/paste/wsgilib.py', line 225 in catch_errors_app
  app_iter = application(environ, start_response)
File '/home/reporter/python_extras/lib/PasteDeploy-1.3.1-py2.4.egg/paste/deploy/config.py', line 164 in __call__
  app_iter = self.application(environ, start_response)
File '/home/reporter/python_extras/lib/Pylons-0.9.5-py2.4.egg/pylons/wsgiapp.py', line 261 in __call__
  return self.app(environ, start_response)
File '/home/reporter/python_extras/lib/Beaker-0.7.3-py2.4.egg/beaker/cache.py', line 175 in __call__
  return self.app(environ, start_response)
File '/home/reporter/python_extras/lib/Beaker-0.7.3-py2.4.egg/beaker/session.py', line 388 in __call__
  response = self.wrap_app(environ, session_start_response)
File '/home/reporter/python_extras/lib/Routes-1.7-py2.4.egg/routes/middleware.py', line 104 in __call__
  response = self.app(environ, start_response)
File '/home/reporter/python_extras/lib/Pylons-0.9.5-py2.4.egg/pylons/wsgiapp.py', line 84 in __call__
  response = self.dispatch(controller, environ, start_response)
File '/home/reporter/python_extras/lib/Pylons-0.9.5-py2.4.egg/pylons/wsgiapp.py', line 182 in dispatch
  return controller(environ, start_response)
File '/data/breakpad/pylons/socorro/lib/base.py', line 33 in __call__
  return WSGIController.__call__(self, environ, start_response)
File '/home/reporter/python_extras/lib/Pylons-0.9.5-py2.4.egg/pylons/controllers.py', line 214 in __call__
  response = self._dispatch_call()
File '/home/reporter/python_extras/lib/Pylons-0.9.5-py2.4.egg/pylons/controllers.py', line 161 in _dispatch_call
  response = self._inspect_call(func)
File '/home/reporter/python_extras/lib/Pylons-0.9.5-py2.4.egg/pylons/controllers.py', line 135 in _inspect_call
  result = func(**args)
File '/data/breakpad/pylons/socorro/controllers/status.py', line 27 in index
  c.lastProcessedDate = result[0].strftime('%Y-%m-%d %H:%M:%S')
AttributeError: 'NoneType' object has no attribute 'strftime'

Comment 2

[(not currently active) Ted Mielczarek]

That looks like a bug on the status page, probably unrelated.

Comment 3

[Michael Morgan [:morgamic]]

Try rebuilding it, I think it might be missing range_value in that URL?
http://crash-stats.mozilla.com/report/list?range_unit=weeks&query_search=signature&query_type=contains&product=Firefox&version=Firefox%3A3.0pre&branch=1.9&signature=js_GetGCThingTraceKind&query=js_GetGCThingTraceKind&range_value=1

Comment 4

[Michael Morgan [:morgamic]]

Err nm, I'm stupid.  Anyway, URL seems to work now...

Comment 5

[Michael Morgan [:morgamic]]

One question - does it matter to you if it starts with or contains?  Starts with is faster, and won't kill the db as much.

Comment 6

[Aravind Gottipati [:aravind]]

Moving this to socorro, since fixing this for good will need other app changes, like aggregate tables etc (or so morgamic tells me).

Comment 7

[Michael Morgan [:morgamic]]

Second thought, it is more about smaller partition sizes and better date constraints -- since the modules table can't be put into a summary table due to the nature of the query.  But regardless, this is a socorro bug.

Comment 8

[Mike Shaver (:shaver -- probably not reading bugmail closely)]

(Needed contains instead of starts-with, because there were often random addresses in the top frame; there's a bug to strip those for classification, I think, which would help.)

Comment 9

[(not currently active) Ted Mielczarek]

shaver: you can do "one of the top 10 stack frames" "starts with".

Comment 10

[Michael Morgan [:morgamic]]

But he just said he didn't want that.  I'm so confused.

Comment 11

[(not currently active) Ted Mielczarek]

Given his explanation in comment 8, I don't think he knows what he wants.

Comment 12

[Michael Morgan [:morgamic]]

hah.  fair enough. <3

Comment 13

[Mike Shaver (:shaver -- probably not reading bugmail closely)]

Sorry, I didn't realize that there were two ways to control what was searced for, and "stack signature" looked like what I wanted to search on, so I didn't bother clicking every drop-down on the page to see if there was something more specific.  I guess I'd have expected it to manifest as "contains in the first 10 (faster)" in the contains/starts-with/is-exactly drop-down, since it's a modification of "contains".  (Do people really use "stack signature -> is exactly"?  I wouldn't trust my copy-and-paste into that form to match it character for character, and it's not clear how to specify multiple frames in the field anyway.)

Comment 14

[(not currently active) Ted Mielczarek]

Right, our terminology is not great. "Stack Signature" is just the top frame of the crashing thread right now.

Comment 15

[Mike Shaver (:shaver -- probably not reading bugmail closely)]

Also, I took "stack signature" -> "starts with" to mean "stack signature has this symbol as its first entry", not "has this string as a prefix of its entire value", because when people say "this stack starts with JS_Foo" they don't usually mean "it could be JS_Foolish".  Is that not what it means?

Mad-libs search UI isn't really my thing, I guess.

Comment 16

[Mike Shaver (:shaver -- probably not reading bugmail closely)]

Just rename it to "Crashing function", then?

Comment 17

[timeless]

Stack Signature is a technical term and I'd assume we inheritted it from talkback:

http://talkback-public.mozilla.org/search/start.jsp

I'm not sure how often you use it, but I think most people are familiar w/ at least Stack Signature meaning the top one frame.

Note that talkback uses "Begins with", which I happen to prefer over "starts with"

I'd prefer we not rename Stack Signature

I think that perhaps to avoid your problem we should borrow further from talkback and use radios instead of a <select> with the assumption that the default rendering of <select> will not show the other options and people will therefore neither see nor examine them.

Comment 18

[Benjamin Smedberg]

Eventually, the stack signature will not necessarily be the top frame: if the top frame is very common (e.g. 0x0) but the second frame is unique, we may end up with a stack signature with both frames, or skip the top frame entirely.

The signature is intended to be pretty unique to a crash type, although we're only going for statistical generalizations not perfection.