805323 - [socorro-crashstats] 500 Internal Server Error searching for EXC_BAD_ACCESS / KERN_INVALID_ADDRESS

Status: VERIFIED FIXED

(Socorro :: Webapp, task, P1)

Description

[Stephen Donner [:stephend] Not actively reading bugmail]

STR:

1. Load http://crash-stats-new-dev.allizom.org/query/#advanced
2. Select B2G, All from the <select> options
3. Put "EXC_BAD_ACCESS / KERN_INVALID_ADDRESS" as the Crash Reason
4. Click "Filter All Results" (http://crash-stats-new-dev.allizom.org/query/?product=B2G&version=ALL%3AALL&range_value=1&range_unit=weeks&date=10%2F25%2F2012+03%3A02%3A34&query_search=signature&query_type=contains&query=&reason=EXC_BAD_ACCESS+%2F+KERN_INVALID_ADDRESS&build_id=&process_type=any&hang_type=any#advanced)

Actual:

500 Internal Server Error (not sure, this *might* be a dupe?)

Comment 1

[Stephen Donner [:stephend] Not actively reading bugmail]

BTW, doesn't happen when EXCEPTION_ACCESS_VIOLATION_READ is the Crash Reason:

https://crash-stats.mozilla.com/query/query?product=B2G&version=ALL%3AALL&range_value=1&range_unit=weeks&date=10%2F25%2F2012+03%3A07%3A03&query_search=signature&query_type=contains&query=&reason=EXCEPTION_ACCESS_VIOLATION_READ&build_id=&process_type=any&hang_type=any&do_query=1

Comment 2

[Stephen Donner [:stephend] Not actively reading bugmail]

Er, sorry, link for comment 1 should be: http://crash-stats-new-dev.allizom.org/query/?product=B2G&version=ALL%3AALL&range_value=1&range_unit=weeks&date=10%2F25%2F2012+03%3A10%3A43&query_search=signature&query_type=contains&query=&reason=EXCEPTION_ACCESS_VIOLATION_READ&build_id=&process_type=any&hang_type=any#advanced

Comment 3

[Robert Helmer [:rhelmer]]

Environment:


Request Method: GET
Request URL: http://localhost:8000/query/?product=B2G&version=ALL%3AALL&range_value=1&range_unit=weeks&date=10%2F25%2F2012+03%3A02%3A34&query_search=signature&query_type=contains&query=&reason=EXC_BAD_ACCESS+%2F+KERN_INVALID_ADDRESS&build_id=&process_type=any&hang_type=any

Django Version: 1.4.2
Python Version: 2.6.8
Installed Applications:
['funfactory',
 'compressor',
 'tower',
 'cronjobs',
 'django_browserid',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.staticfiles',
 'commonware.response.cookies',
 'djcelery',
 'django_nose',
 'session_csrf',
 'product_details',
 'crashstats.base',
 'crashstats.crashstats',
 'jingo_offline_compressor',
 'crashstats.auth',
 'django.contrib.messages']
Installed Middleware:
('multidb.middleware.PinningRouterMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'session_csrf.CsrfMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'commonware.middleware.FrameOptionsHeader',
 'mobility.middleware.DetectMobileMiddleware',
 'mobility.middleware.XMobileMiddleware')


Traceback:
File "/home/rhelmer/src/socorro-crashstats/vendor/lib/python/django/core/handlers/base.py" in get_response
  111.                         response = callback(request, *callback_args, **callback_kwargs)
File "/home/rhelmer/src/socorro-crashstats/crashstats/crashstats/views.py" in inner
  162.         return view(request, *args, **kwargs)
File "/home/rhelmer/src/socorro-crashstats/crashstats/crashstats/views.py" in query
  1130.             result_offset=data['results_offset']
File "/home/rhelmer/src/socorro-crashstats/crashstats/crashstats/models.py" in get
  513.         return self.fetch(url)
File "/home/rhelmer/src/socorro-crashstats/crashstats/crashstats/models.py" in fetch
  126.             raise BadStatusCodeError('%s: on: %s' % (resp.status_code, url))

Exception Type: BadStatusCodeError at /query/
Exception Value: 404: on: http://socorro1.dev.dmz.phx1.mozilla.com/bpapi/search/signatures/to/2012-10-25T03%3A02%3A34/reasons/EXC_BAD_ACCESS%20%2F%20KERN_INVALID_ADDRESS/plugin_search_mode/contains/result_number/100/plugin_in/filename/products/B2G/report_type/any/search_mode/contains/report_process/any/from/2012-10-18T03%3A02%3A34/result_offset/0

Comment 4

[[DEACTIVATED] Adrian Gaudebert]

This fails just the same on prod: https://crash-stats.mozilla.com/query?product=B2G&version=ALL%3AALL&range_value=1&range_unit=weeks&date=10%2F25%2F2012+03%3A02%3A34&query_search=signature&query_type=contains&query=&reason=EXC_BAD_ACCESS+%2F+KERN_INVALID_ADDRESS&build_id=&process_type=any&hang_type=any&do_query=1

It is expected, there is a slash in the crash reason, that gets encoded as %2F, and Apache won't accept that. Is that string a valid crash reason: "EXC_BAD_ACCESS / KERN_INVALID_ADDRESS"?

Comment 5

[Stephen Donner [:stephend] Not actively reading bugmail]

(In reply to Adrian Gaudebert [:adrian] from comment #4)
> This fails just the same on prod:
> https://crash-stats.mozilla.com/
> query?product=B2G&version=ALL%3AALL&range_value=1&range_unit=weeks&date=10%2F
> 25%2F2012+03%3A02%3A34&query_search=signature&query_type=contains&query=&reas
> on=EXC_BAD_ACCESS+%2F+KERN_INVALID_ADDRESS&build_id=&process_type=any&hang_ty
> pe=any&do_query=1
> 
> It is expected, there is a slash in the crash reason, that gets encoded as
> %2F, and Apache won't accept that. Is that string a valid crash reason:
> "EXC_BAD_ACCESS / KERN_INVALID_ADDRESS"?

Yep, see https://crash-stats.mozilla.com/report/index/be8a7fa3-c43f-42ce-884a-99a8c2121024:

EXC_BAD_ACCESS / KERN_PROTECTION_FAILURE (should be the same issue)

Comment 6

[Lonnen :lonnen]

I can no longer repro this. Could it be that this was fixed by one of the other fixes for 500 errors?

Comment 7

[Stephen Donner [:stephend] Not actively reading bugmail]

(In reply to Chris Lonnen :lonnen from comment #6)
> I can no longer repro this. Could it be that this was fixed by one of the
> other fixes for 500 errors?

Yeah, taken care of by one of: peterb, rhelmer, or adrian :-)

If one of you knows the fix, could you comment and close it as FIXED?  Thanks.

Comment 8

[[DEACTIVATED] Adrian Gaudebert]

This now shows no error but it still doesn't work. I thus made https://github.com/mozilla/socorro-crashstats/pull/314 and https://github.com/mozilla/socorro/pull/1123 to allow slashes in reasons. 

Steps to QA
-----------

1. Go to advanced search
2. Select product B2G, reason EXC_BAD_ACCESS / KERN_INVALID_ADDRESS, run search
3. Verify that there are results
4. Click a signature and verify that there are results in report/list

Comment 9

[[github robot]]

Commits pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/588ab9951cb0f39508b59e0c68d0dc73fe00bada
Bug 805323 - Double-decode slashes in reasons parameter in search.

https://github.com/mozilla/socorro/commit/99c78e0c7a80bb516d75fa1a8a453b74f76ae38e
Merge pull request #1123 from AdrianGaudebert/805323-allow-slashes-in-reasons

Bug 805323 - Double-decode slashes in reasons parameter in middleware.

Comment 10

[[github robot]]

Commits pushed to master at https://github.com/mozilla/socorro-crashstats

https://github.com/mozilla/socorro-crashstats/commit/0d7f97203c818ce47d2b8cc5d4d2e72f9db6b3f4
Fixes bug 805323 - Double-encode slashes in reasons parameters.

https://github.com/mozilla/socorro-crashstats/commit/ae8e18f6e4d1bae268c5a8e6e6f992769e01adc9
Merge pull request #314 from AdrianGaudebert/805323-allow-slashes-in-reasons

Fixes bug 805323 - Double-encode slashes in reasons parameters.

Comment 11

[[DEACTIVATED] Adrian Gaudebert]

This has a dependency to a change in the middleware, it will not be testable before the middleware our production instance uses is updated to the latest version. Not sure what we use, so I can't give an ETA.

Comment 12

[Stephen Donner [:stephend] Not actively reading bugmail]

(In reply to Adrian Gaudebert [:adrian] from comment #11)
> This has a dependency to a change in the middleware, it will not be testable
> before the middleware our production instance uses is updated to the latest
> version. Not sure what we use, so I can't give an ETA.

Excellent; this is working on dev:

https://crash-stats-dev.allizom.org/query/?product=Firefox&version=ALL%3AALL&range_value=14&range_unit=days&date=07%2F09%2F2013+18%3A00%3A00&query_search=signature&query_type=contains&query=&reason=EXC_BAD_ACCESS+%2F+KERN_INVALID_ADDRESS&release_channels=&build_id=&process_type=any&hang_type=any