860514 - [stage][socorro-crashstats-django] 500 Internal Server Error: BadStatusCodeError

Status: VERIFIED DUPLICATE of bug 869033

(Socorro Graveyard :: Middleware, defect)

Description

[Matt Brandt [:mbrandt]]

Steps to reproduce:
1. goto https://crash-stats-django.allizom.org/report/list

Actual:
Returns a 500 error

Traceback (most recent call last):

  File "/data/www/crash-stats-django.allizom.org/app/vendor/lib/python/django/core/handlers/base.py", line 111, in get_response
    response = callback(request, *callback_args, **callback_kwargs)

  File "/data/www/crash-stats-django.allizom.org/app/crashstats/crashstats/views.py", line 154, in inner
    for key, value in _basedata(product, versions).items():

  File "/data/www/crash-stats-django.allizom.org/app/crashstats/crashstats/views.py", line 120, in _basedata
    data['currentversions'] = api.get()

  File "/data/www/crash-stats-django.allizom.org/app/crashstats/crashstats/models.py", line 273, in get
    products = CurrentProducts().get()['hits']

  File "/data/www/crash-stats-django.allizom.org/app/crashstats/crashstats/models.py", line 294, in get
    return self.fetch(url)

  File "/data/www/crash-stats-django.allizom.org/app/crashstats/crashstats/models.py", line 126, in fetch
    raise BadStatusCodeError('%s: on: %s' % (resp.status_code, url))

BadStatusCodeError: 500: on: http://socorro-mware-stage-zlb.webapp.phx1.mozilla.com/bpapi/products/

Comment 1

[Matt Brandt [:mbrandt]]

I've receiving quite a few (100+) tracebacks for BadStatusCodeError that match this url pattern.

- https://crash-stats-django.allizom.org/report/index/fd2266f0-9d0c-4ce1-b5a6-523092130313
- https://crash-stats-django.allizom.org/report/list

Comment 2

[Stephen Donner [:stephend] Not actively reading bugmail]

Just for posterity (since it happens in the same callstack/lines, but with a different error code), this also happens with some 404s returned by the middleware?

[14:03:31.483] GET https://crash-stats-django.allizom.org/query/?product=Firefox&version=ALL%3AALL&range_value=1&range_unit=weeks&date=04%2F23%2F2013+18%3A02%3A26&query_search=signature&query_type=contains&query=&reason=&release_channels=&build_id=http%3A%2F%2Fwww.google.com&process_type=any&hang_type=any [HTTP/1.1 500 INTERNAL SERVER ERROR 1300ms]

BadStatusCodeError: 404: on: http://socorro-mware-stage-zlb.webapp.phx1.mozilla.com/bpapi/search/signatures/build_ids/http%3A%2F%2Fwww.google.com/to/2013-04-23T18%3A02%3A26/plugin_search_mode/contains/result_number/100/plugin_in/filename/products/Firefox/report_type/any/search_mode/contains/report_process/any/from/2013-04-16T18%3A02%3A26/result_offset/0

Stacktrace (most recent call last):

  File "django/core/handlers/base.py", line 111, in get_response
    response = callback(request, *callback_args, **callback_kwargs)
  File "crashstats/crashstats/views.py", line 156, in inner
    return view(request, *args, **kwargs)
  File "crashstats/crashstats/views.py", line 1345, in query
    result_offset=data['results_offset']
  File "crashstats/crashstats/models.py", line 601, in get
    return self.fetch(url)
  File "crashstats/crashstats/models.py", line 126, in fetch
    raise BadStatusCodeError('%s: on: %s' % (resp.status_code, url))

Comment 3

[Peter Bengtsson [:peterbe]]

This feels like a cop-out but this bug is not great to work on. 

The problem :stephend mentioned is #869033 which is that the build_id can be allowed some characters that shouldn't be sent over to the middleware. 

Also, ALL other problems where the middleware is failing (and there's nothing socorro-crashstats can do about it) will result in a BadStatusCodeError. 

For example, the problem with https://crash-stats-django.allizom.org/report/index/fd2266f0-9d0c-4ce1-b5a6-523092130313 is that the dev server uses a postgres database that contains crash IDs that aren't available in the HBase on the dev server. We're working on a solution to that but for now all I have is this: https://bugzilla.mozilla.org/show_bug.cgi?id=871597

Comment 4

[Matt Brandt [:mbrandt]]

Bug 869033 is technically the duplicate, it was filed after this bug.

Marking verified dupe, bug 869033 is more concise in terms of what needs to be remedied.