630704 - Input staging (release) dashboard serving an ISE

Status: VERIFIED FIXED

(Input :: General, defect)

Description

[Rebecca Billings [:rbillings]]

The staging dashboard is serving an ISE at: http://input.stage.mozilla.com/en-US/release/

It says the error has been logged but I did not receive the TB- hopefully that can be found and added.

Comment 1

[Dave Dash [:davedash, :dd] (assign all bugs to mbrandt)]

Traceback (most recent call last):

 File "/data/www/input.stage.mozilla.com/reporter/vendor/packages/Django/django/core/handlers/base.py", line 100, in get_response
   response = callback(request, *callback_args, **callback_kwargs)

 File "/data/www/input.stage.mozilla.com/reporter/apps/input/decorators.py", line 94, in negotiated
   return release(request, *args, **kwargs)

 File "/data/www/input.stage.mozilla.com/reporter/vendor/packages/Django/django/utils/decorators.py", line 76, in _wrapped_view
   response = view_func(request, *args, **kwargs)

 File "/data/www/input.stage.mozilla.com/reporter/apps/input/decorators.py", line 43, in cached_view
   return f(request, *args, **kwargs)

 File "/data/www/input.stage.mozilla.com/reporter/apps/search/views.py", line 248, in release
   (_, form, product, version, metas) = _get_results(request, metas, client=c)

 File "/data/www/input.stage.mozilla.com/reporter/apps/search/views.py", line 32, in _get_results
   opinions = c.query(query, **search_opts)

 File "/data/www/input.stage.mozilla.com/reporter/apps/search/client.py", line 205, in query
   self.handle_metas(results, kwargs.get('meta', {}), kwargs)

 File "/data/www/input.stage.mozilla.com/reporter/apps/search/client.py", line 311, in handle_metas
   super(RatingsClient, self).handle_metas(results, metas, kwargs)

 File "/data/www/input.stage.mozilla.com/reporter/apps/search/client.py", line 141, in handle_metas
   self.meta['locale'] = self._locale_meta(results, **kwargs)

 File "/data/www/input.stage.mozilla.com/reporter/apps/search/client.py", line 262, in _locale_meta
   for f in result['matches']]

KeyError: 'matches'


<WSGIRequest
GET:<QueryDict: {}>,
POST:<QueryDict: {}>,
COOKIES:{'WT_FPC': 'id=24.4.140.103-674348304.30130776:lv=1296629097406:ss=1296629097406',
 'csrftoken': '0f66856834b6fe6d91bb42ff08aedb1d',
 'intro_seen': 'true',
 'wtspl': '624608'},
META:{'CSRF_COOKIE': '0f66856834b6fe6d91bb42ff08aedb1d',
 'DOCUMENT_ROOT': '/data/www/input.stage.mozilla.com',
 'GATEWAY_INTERFACE': 'CGI/1.1',
 'HTTP_ACCEPT': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
 'HTTP_ACCEPT_CHARSET': 'ISO-8859-1,utf-8;q=0.7,*;q=0.7',
 'HTTP_ACCEPT_ENCODING': 'gzip,deflate',
 'HTTP_ACCEPT_LANGUAGE': 'en-US,en;q=0.9,fr-FR;q=0.7,fr;q=0.6,it-IT;q=0.4,it;q=0.3,en;q=0.1',
 'HTTP_CACHE_CONTROL': 'max-age=0',
 'HTTP_CONNECTION': 'keep-alive',
 'HTTP_COOKIE': 'wtspl=624608; WT_FPC=id=24.4.140.103-674348304.30130776:lv=1296629097406:ss=1296629097406; csrftoken=0f66856834b6fe6d91bb42ff08aedb1d; intro_seen=true',
 'HTTP_HOST': 'input.stage.mozilla.com',
 'HTTP_KEEP_ALIVE': '115',
 'HTTP_USER_AGENT': 'Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13',
 'HTTP_X_CLUSTER_CLIENT_IP': '24.4.140.103',
 'HTTP_X_FORWARDED_FOR': '24.4.140.103',
 'HTTP_X_MOZ_PLATFORM': 'other',
 'PATH_INFO': u'/en-US/release/',
 'PATH_TRANSLATED': '/data/www/input.stage.mozilla.com/reporter/wsgi/reporter.wsgi/en-US/release/',
 'QUERY_STRING': '',
 'REMOTE_ADDR': '10.2.81.103',
 'REMOTE_PORT': '12729',
 'REQUEST_METHOD': 'GET',
 'REQUEST_URI': '/en-US/release/',
 'SCRIPT_FILENAME': '/data/www/input.stage.mozilla.com/reporter/wsgi/reporter.wsgi',
 'SCRIPT_NAME': u'',
 'SERVER_ADDR': '10.2.81.29',
 'SERVER_ADMIN': '[e-mail]',
 'SERVER_NAME': 'input.stage.mozilla.com',
 'SERVER_PORT': '80',
 'SERVER_PROTOCOL': 'HTTP/1.1',
 'SERVER_SIGNATURE': '<address>Apache/2.2.3 (Red Hat) Server at input.stage.mozilla.com Port 80</address>\n',
 'SERVER_SOFTWARE': 'Apache/2.2.3 (Red Hat)',
 'mod_wsgi.application_group': 'input.stage.mozilla.com|',
 'mod_wsgi.callable_object': 'application',
 'mod_wsgi.handler_script': '',
 'mod_wsgi.input_chunked': '0',
 'mod_wsgi.listener_host': '',
 'mod_wsgi.listener_port': '80',
 'mod_wsgi.process_group': '',
 'mod_wsgi.request_handler': 'wsgi-script',
 'mod_wsgi.script_reloading': '1',
 'mod_wsgi.version': (3, 2),
 'wsgi.errors': <mod_wsgi.Log object at 0xa452db8>,
 'wsgi.file_wrapper': <built-in method file_wrapper of mod_wsgi.Adapter object at 0x9a10650>,
 'wsgi.input': <mod_wsgi.Input object at 0xa452b60>,
 'wsgi.loaded': datetime.datetime(2011, 2, 1, 15, 35, 41, 199305),

Comment 2

[Dave Dash [:davedash, :dd] (assign all bugs to mbrandt)]

I can't recreate this locally, so I'll just try to prevent the TB.

http://github.com/fwenzel/reporter/commit/9a26124

Comment 3

[Fred Wenzel [:wenzel]]

Now the same thing occurs with OSes. Looks like this is a sphinx error.

----
Traceback (most recent call last):

 File "/data/www/input.stage.mozilla.com/reporter/vendor/packages/Django/django/core/handlers/base.py", line 100, in get_response
   response = callback(request, *callback_args, **callback_kwargs)

 File "/data/www/input.stage.mozilla.com/reporter/apps/input/decorators.py", line 94, in negotiated
   return release(request, *args, **kwargs)

 File "/data/www/input.stage.mozilla.com/reporter/vendor/packages/Django/django/utils/decorators.py", line 76, in _wrapped_view
   response = view_func(request, *args, **kwargs)

 File "/data/www/input.stage.mozilla.com/reporter/apps/input/decorators.py", line 43, in cached_view
   return f(request, *args, **kwargs)

 File "/data/www/input.stage.mozilla.com/reporter/apps/search/views.py", line 250, in release
   client=c)

 File "/data/www/input.stage.mozilla.com/reporter/apps/search/views.py", line 32, in _get_results
   opinions = c.query(query, **search_opts)

 File "/data/www/input.stage.mozilla.com/reporter/apps/search/client.py", line 205, in query
   self.handle_metas(results, kwargs.get('meta', {}), kwargs)

 File "/data/www/input.stage.mozilla.com/reporter/apps/search/client.py", line 312, in handle_metas
   super(RatingsClient, self).handle_metas(results, metas, kwargs)

 File "/data/www/input.stage.mozilla.com/reporter/apps/search/client.py", line 143, in handle_metas
   self.meta['os'] = self._os_meta(results, **kwargs)

 File "/data/www/input.stage.mozilla.com/reporter/apps/search/client.py", line 245, in _os_meta
   for f in result['matches']]

KeyError: 'matches'

Comment 4

[Dave Dash [:davedash, :dd] (assign all bugs to mbrandt)]

Yeah, that too was fixed last night.  The real error is the blocker.

Rebecca, feel free to adjust the blocker's severity as necessary if this is blocking you (/beta should be fine).

Comment 5

[Fred Wenzel [:wenzel]]

I googled a little and found a hint indicating that Sphinx 0.9.9 is majorly unhappy if it doesn't have enough plain text to chew on, thus considers the index broken and refuses to serve it.

I ended up giving it dummy plain text to knock itself out with:
http://github.com/fwenzel/reporter/commit/45d5fef

This fixes it for me locally, and tests pass, so calling this closed.

QA: If you don't see an ISE anymore going to the release dashboard (might be empty, that's another story) then this worked.

Comment 6

[Matt Brandt [:mbrandt]]

QA verified. Looks great.