714338 - Matview changes required for signature summary

Status: VERIFIED FIXED

(Socorro :: Database, task)

Description

[Schalk Neethling [:espressive]]

Please see the following list of bugs, they require matview changes in order to be implemented:

https://bugzilla.mozilla.org/show_bug.cgi?id=711223
https://bugzilla.mozilla.org/show_bug.cgi?id=711224
https://bugzilla.mozilla.org/show_bug.cgi?id=711225

Comment 1

[[:jberkus] Josh Berkus]

I will be testing requirements for implementing the above.  This implementation will also implement the solution for bug 702384 as well.

Comment 2

[[:jberkus] Josh Berkus]

The cpu architecture report will need to be implemented last, and may miss 2.4 due to some plumbing required.  We haven't previously done reports on cpu architecture, so that wasn't data we were rolling up anywhere.  Modifications to reports_clean and possibly other cron jobs will be required.

Comment 3

[[:jberkus] Josh Berkus]

Schalk,

So as a draft I've added a stored procedure to crash-stats-dev which produces all of the various signature summaries.  Here's the API:

signature_summary (
   signature TEXT,
   report_type TEXT,
   days INT,
   product TEXT,
   versions ARRAY OF INT optional
)

signature
  the signature as text
report_type
  one of: products, os, process_type, uptime, flash_version
days
  number of days the report is to cover
product
  the product name
versions
  the product_version_id of the version or versions selected
  an array of INT, this can be passed using the array constructor:
  ARRAY[ #, #, # ]
  optional.  if not supplied, all reports for the product are searched

Examples:


select * from signature_summary('hang | ZwCreateFile','products',7,'Firefox');

select * from signature_summary('hang | ZwCreateFile','process_type',7,'Firefox',ARRAY[ 792 ]);

select * from signature_summary('hang | ZwCreateFile','os',14,'Firefox',ARRAY[ 792, 798, 794, 800 ]);

Obviously, this is a prototype; a final version should really be implemented as a middleware API rather than a stored procedure.  Check the attached code as a model for the python code which would be needed.

Comment 4

[[:jberkus] Josh Berkus]

Attachment: sample stored procedure code for signature summaries

You'll notice a couple of things here.

1) I chose not to create more matviews, and am instead relying on some new indexes on reports_clean.

2) This replaces *all* signature summaries, including the ones currently served by matviews, such as products and oses.

Comment 5

[[:jberkus] Josh Berkus]

This is deployed on crash-stats-dev.

QA: you can test when Brandon's middleware work is done.  In the short term, you can verify that signature summary on crash-stats-dev will be broken.

Comment 6

[Matt Brandt [:mbrandt]]

reopening tracker - a dependent bug 711223 has not landed.

Comment 7

[[:jberkus] Josh Berkus]

reclosing; dependency was false.

Comment 8

[Stephen Donner [:stephend] Not actively reading bugmail]

(In reply to [:jberkus] Josh Berkus from comment #7)
> reclosing; dependency was false.

Indeed, but as we discussed in IRC, we should at least remove it as a dependency, then, so doing so now.

Comment 9

[Matt Brandt [:mbrandt]]

Attachment: qa - verified

Bumping to QA verified. Automation is still passing.

http://qa-selenium.mv.mozilla.com:8080/view/Socorro/job/socorro.stage.saucelabs/217/