677369 - ADU does not understand the new version strings

Status: VERIFIED FIXED

(Socorro :: General, task, P1)

Description

[Brandon Savage [:brandon]]

The ADU reports a key error for the new version strings. The exception is as follows:

127.0.0.1:39250 - - [08/Aug/2011 14:24:17] "HTTP/1.1 GET /201010/topcrash/sig/trend/rank/p/Firefox/v/6.0b3/type/browser/end/2011-08-08T14:00:00T+0000/duration/72/listsize/300" - 500 Internal Server Error
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/web.py-0.33-py2.6.egg/web/application.py", line 242, in process
    return self.handle()
  File "/usr/lib/python2.6/site-packages/web.py-0.33-py2.6.egg/web/application.py", line 233, in handle
    return self._delegate(fn, self.fvars, args)
  File "/usr/lib/python2.6/site-packages/web.py-0.33-py2.6.egg/web/application.py", line 396, in _delegate
    return handle_class(f)
  File "/usr/lib/python2.6/site-packages/web.py-0.33-py2.6.egg/web/application.py", line 387, in handle_class
    return tocall(*args)
  File "/home/bsavage/trunk/socorro/webapi/webapiService.py", line 60, in GET
    raise Exception(stringLogger.getMessages())
Exception: ERROR Caught Error: <type 'exceptions.KeyError'>
ERROR ('Firefox', u'6.0b3')
ERROR trace back follows:
ERROR Traceback (most recent call last):
ERROR File "/home/bsavage/trunk/socorro/webapi/webapiService.py", line 46, in GET
    result = self.get(*args)
ERROR File "/home/bsavage/trunk/socorro/services/aduByDay.py", line 29, in get
    parameters.productdims_idList = [self.context['productVersionCache'].getId(parameters.product, x) for x in parameters.listOfVersions]
ERROR File "/home/bsavage/trunk/socorro/lib/productVersionCache.py", line 47, in getId
    raise KeyError((product, version))
ERROR KeyError: ('Firefox', u'6.0b3')

127.0.0.1:39256 - - [08/Aug/2011 14:24:17] "HTTP/1.1 GET /201005/adu/byday/p/Firefox/v/8.0a1;7.0a2;6.0b3;5.0/rt/any/os/Windows;Mac;Linux/start/2011-08-04/end/2011-08-07" - 500 Internal Server Error

Comment 1

[Brandon Savage [:brandon]]

Taking a second look at this, it's an issue related directly to the work I was working on. Taking it to fix.

Comment 2

[Brandon Savage [:brandon]]

Attachment: Patch for fixing the broken table references

Comment 3

[Laura Thomson :laura]

Comment on attachment 551677 [details] [diff] [review]
Patch for fixing the broken table references

You left the debug logging in, change that for an r+ from me

Comment 4

[Brandon Savage [:brandon]]

Attachment: Removing debug data

This patch is the same but removes the debug data I had included earlier.

Comment 5

[Brandon Savage [:brandon]]

Committed revision 3348.

Comment 6

[Robert Helmer [:rhelmer]]

Attachment: fix broken graphs on stage

The schema change to alter daily_crashes adu_day column to be date instead of timestamp was reverted, since it breaks existing code for graphs.

Here's a quick fix, although really we could do away with a lot of this code by just having daily_crashes populated directly with the ADU counts, since it's a materialized view that's generated from tcbs anyway.

Peter, do you see a more elegant way to patch this for 2.2?

Comment 7

[Peter Bengtsson [:peterbe]]

Comment on attachment 552316 [details] [diff] [review]
fix broken graphs on stage

Review of attachment 552316 [details] [diff] [review]:
-----------------------------------------------------------------

I'm guessing the purpose of this hack is because "adu_day" is a DATE and "adu_date" is a DATETIME (aka. TIMESTAMP). Why not just make the cast inside the SQL. E.g.

    sql = """
      select
          adu_date::DATE as date,
          substring(os_name, 1, 3) as product_os_platform,
          sum(adu_count)::BIGINT
      from

Comment 8

[Robert Helmer [:rhelmer]]

Attachment: cast adu_day as DATE

Ah of course, thanks Peter. Was late and felt like I must be missing something :)

We actually want the cast on the daily_crashes adu_day column, I just tested this against stagedb and it seems to work.

Comment 9

[Robert Helmer [:rhelmer]]

Landed on trunk and 2.2 branch:

Sending        socorro/services/aduByDay.py
Transmitting file data .
Committed revision 3398.
Committed revision 3399.

Comment 10

[Matt Brandt [:mbrandt]]

QA verified. Cycling through the different version strings behaves as expected and does not produce errors.