691842 - Fennec 8.0b1 is showing no reports.

Status: VERIFIED FIXED

(Socorro :: Webapp, task)

Description

[Naoki Hirata :nhirata (please use needinfo instead of cc)]

1. go to https://crash-stats.mozilla.com/topcrasher/byversion/Fennec/8.0b1/7
2. go to https://crash-stats.mozilla.com/daily?p=Fennec&v[]=
3. do an advanced search

Expected: some reports
Actual: no reports

Note:
1. not sure if it's just that there are no reports or something else.
2. not sure what other things I can do to verify where the break is occuring.

Comment 1

[Robert Helmer [:rhelmer]]

I think this is a problem on the DB side, since I see data in the raw data tables (releases_raw, raw_adu).

However there's no entry in daily_crashes (where graphs data comes from) or entry in product_version_builds for Fennec 8.0b1

Looking at http://ftp.mozilla.org/pub/mozilla.org/mobile/candidates/8.0b1-candidates/build1/ and releases_raw I see that we picked up the metadata ok. 

I suspect something is up with the stored procedures that build up the report tables from the raw data, since both DBAs are out I'll take a look.

Comment 2

[Robert Helmer [:rhelmer]]

OK pretty sure this broke in the 2.2.4 release... the underlying problem seems to be that the product_version_builds table is not getting populated, which is the job of the update_product_versions() function.

If I run the "select" part of https://github.com/mozilla/socorro/blob/v2.2.4/scripts/schema/2.2/breakpad_schema.sql#L1743 then it returns the missing entries.

I see in 2.2.4 that this was changed to https://github.com/mozilla/socorro/blob/v2.2.4/scripts/upgrade/2.2.4/update_products.sql#L87 and the select there does not return the missing entries.

In particular, I don't see how this (in the new 2.2.4 version) could be correct:

ON releases_raw.product_name = product_versions.product_name

"mobile" gets mapped to "Fennec" in the join on the products table, which was removed.

Comment 3

[Laura Thomson :laura]

Josh, can you take a look at this, sounds like a regression in the query?

Comment 4

[[:jberkus] Josh Berkus]

Gah, that is a regression.  The product_name is supposed to map to the release_name for precisely that reason.

Should I hot-fix it, or wait for 2.3.1, where we need to backfill all data anyway?

Comment 5

[Laura Thomson :laura]

2.3.1 backfill did *not* fix this; please hot fix it jberkus.

Comment 6

[[:jberkus] Josh Berkus]

When backing out the untested features from 2.3.1 we accidentally backed out the fixes for Fennec as well.  Unfortunately, we need not just to run a backfill, but to fix some matview generators as well, which means staging and QA.   Given the issues we're having with the staging hardware, 2.3.2 deployment on monday looks like the most realistic time for a fix.

My apologies to the Fennec team.

Comment 7

[Matt Brandt [:mbrandt]]

Created Pivotal story to get automation coverage in this area.

https://www.pivotaltracker.com/story/show/20145655

Comment 8

[Laura Thomson :laura]

Josh, can you update this bug please?

Comment 9

[[:jberkus] Josh Berkus]

This should now be fixed on Stage.  Please test, Matt.

Comment 10

[[:jberkus] Josh Berkus]

I'm closing this as fixed in staging.

**IMPORTANT** the fixes for fennec betas are tied to all of the other fixes for 2.3.2.  If we have to roll back DB code, this bug will also get broken.

Comment 11

[Matt Brandt [:mbrandt]]

QA verified on stage. Top crashers for x.xbx are populated (https://crash-stats.allizom.org/topcrasher/byversion/Fennec/8.0b4). The data for x.x.x(beta) stops at 9.27 resulting in an empty top crashers page (https://crash-stats.allizom.org/topcrasher/byversion/Fennec/7.0.1%28beta%29)

Comment 12

[Matt Brandt [:mbrandt]]

Attachment: screenshot of 8.0b4 top crashers page