938533 - Some signatures passed to signature_history service are truncated

Status: VERIFIED FIXED

(Socorro Graveyard :: Middleware, defect)

Description

[Schalk Neethling [:espressive]]

During my work on https://bugzilla.mozilla.org/show_bug.cgi?id=789526 I asked rhelmer to do some digging as to why some of the graphs returned no data and from he has been able to track down it seems that some Signatures are being truncated by the middleware before being sent to the signature_history service. Or perhaps this is happening in the Django app?

Below follows the details of what rhelmer found. For the entire history see https://bugzilla.mozilla.org/show_bug.cgi?id=789526

---------------------

Hmm OK something is odd here - I think this is the query the mware should be using, I'll have to turn on debug logging for the generated queries to confirm, but this works on dev:

breakpad=> WITH hist AS (
    SELECT
        report_date,
        report_count
    FROM
        tcbs JOIN signatures using (signature_id)
             JOIN product_versions using (product_version_id)
    WHERE
        report_date BETWEEN '2013-11-01' AND '2014-01-01'
        AND product_name = 'Firefox'
        AND version_string = '28.0a1'
        AND signature = 'memmove | LocalBaseRegQueryValue'
    GROUP BY
        report_date, report_count
    ORDER BY 1
),
scaling_window AS (
    SELECT
        hist.*,
        SUM(report_count) over () AS total_crashes
    FROM hist
)
SELECT
    report_date,
    report_count,
    report_count / total_crashes::float * 100 AS percent_of_total
FROM scaling_window
ORDER BY report_date DESC;
 report_date | report_count | percent_of_total 
-------------+--------------+------------------
 2013-11-09  |          195 |       3.80859375
 2013-11-08  |          395 |       7.71484375
 2013-11-07  |         1203 |      23.49609375
 2013-11-06  |         2585 |      50.48828125
 2013-11-05  |          742 |       14.4921875
(5 rows)

EnterBaseline also works (as already expected)

---------------

Ah! I think I see the problem - when a signature like "memmove | LocalBaseRegQueryValue" is passed into the signature_history service, it looks like that gets truncated to "memmove" when I print the params in socorro/external/postgresql/base.py:query

Adrian, could you take a look?

----------------

Comment 1

[Robert Helmer [:rhelmer]]

(In reply to Schalk Neethling [:espressive] from comment #0)
> During my work on https://bugzilla.mozilla.org/show_bug.cgi?id=789526 I
> asked rhelmer to do some digging as to why some of the graphs returned no
> data and from he has been able to track down it seems that some Signatures
> are being truncated by the middleware before being sent to the
> signature_history service. Or perhaps this is happening in the Django app?

Thanks for filing this!

I don't think it could be the Django app since I can reproduce the problem w/ curl like this:

curl 'http://socorro-api-dev-internal/bpapi/crashes/signature_history/product/Firefox/version/28.0a1/signature/memmove | LocalBaseRegQueryValue'

Comment 2

[Schalk Neethling [:espressive]]

Between Adrian and myself this bug will be fixed by https://github.com/mozilla/socorro/pull/1674

Comment 3

[Matt Brandt [:mbrandt]]

Bumping to verified as [qa-] - automation is green http://qa-selenium.mv.mozilla.com:8080/view/Socorro/job/socorro.stage/363/