'Product' for MetroFirefox changed to just 'Firefox' in crash reports 11/30 - product_productid_map update needed

RESOLVED FIXED

Status

Firefox for Metro
General
RESOLVED FIXED
4 years ago
4 years ago

People

(Reporter: jimm, Assigned: selenamarie)

Tracking

Trunk
x86_64
Windows 8.1
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

4 years ago
Something appears to have broken when we switched on profile sharing.

https://crash-stats.mozilla.com/topcrasher/products/MetroFirefox/versions/28.0a1/date_range_type/build/crash_type/browser/os_name/None/result_count/50?days=7

switch the above from 7 days to 3 days, all reports disappear.

Yet we are still seeing bulk number for MetroFirefox product for these days - 

https://crash-stats.mozilla.com/daily?p=MetroFirefox&v=28.0a1

Searching under the produce for recent build ids doesn't turn anything up either.

20131201030203
20131202092621
20131203030202
(Reporter)

Comment 1

4 years ago
Looks like we write out the wrong product name in extra info, and maybe that's what socorro uses to sort reports.

AdapterVendorID=0x8086
StartupTime=1386184254
EMCheckCompatibility=true
ProductName=Firefox
Theme=classic/1.0
Accessibility=Active
Vendor=Mozilla
InstallTime=1386164509
AdapterDeviceID=0x0166
ReleaseChannel=default
Notes=AdapterVendorID: 0x8086, AdapterDeviceID: 0x0166, AdapterSubsysID: 01661414, AdapterDriverVersion: 10.18.10.3262\nD2D? D2D+ DWrite? DWrite+ 
FramePoisonSize=65536
Version=28.0a1
FramePoisonBase=00000000f0de0000
ServerURL=https://crash-reports.mozilla.com/submit?id={99bceaaa-e3c6-48c1-b981-ef9b46b67d60}&version=28.0a1&buildid=20131204073433
useragent_locale=en-US
Add-ons=%7B972ce4c6-7e08-4474-a285-3208198ce6fd%7D:28.0a1
BIOS_Manufacturer=American Megatrends Inc.
ProductID={99bceaaa-e3c6-48c1-b981-ef9b46b67d60}
URL=about:start
BuildID=20131204073433
CrashTime=1386184261
BlockedDllList=
SystemMemoryUsePercentage=37
TotalVirtualMemory=4294836224
AvailableVirtualMemory=3822641152
AvailablePageFile=2557145088
AvailablePhysicalMemory=2604777472

Updated

4 years ago
Blocks: 838081
(Reporter)

Comment 2

4 years ago
Lars, can you help clarify what socorro / socorro web interface uses as an identifier for products? We appear to be sending the data still, but in the socorro interface crashes don't appear to sort right.

some background: the metro browser used to run in its own profile, but last sunday we started sharing a profile with desktop firefox. To do that, we had to change the ProductName in the browser ini file to get the same profile directory. We kept a separate app id and the unique crash submit url. I'm thinking we need to hack that ProductName when we right the extra info out, since it seems this is what socorro uses to filter (vs. using the app id in the submit url).
Flags: needinfo?(lars)
(Reporter)

Comment 3

4 years ago
* when we *write* the extra info out :)
(Reporter)

Comment 4

4 years ago
Laura, can you help direct this to the right person in the socorro team if I've tagged lars incorrectly?
From what I can see looking in Postgres, we're getting crashes from MetroFirefox 28.0a1 (41 since Monday).  Jumping back to the UI, I can use either "Advanced Search" or "Super Search" to find the crashes for MetroFirefox 28.0a1.  It appears to only be aggregate reporting that is missing for this week for that product/version.  

Digging deeper, I can see that there is data in the Top Crash By Signature materialized view. Not sure why it isn't displaying in the UI.   

I'm turning this over to selenamarie, this is in her realm.
Flags: needinfo?(sdeckelmann)
I've looked and I see data in TCBS for 28.0a1, grouped by version string and report date: 

select version_string, report_date, sum(report_count) from tcbs JOIN product_versions USING (product_version_id) where product_version_id = 1919 and report_date > '2013-11-25' group by version_string, report_date order by report_date;

 version_string | report_date | sum 
----------------+-------------+-----
 28.0a1         | 2013-11-26  |  28
 28.0a1         | 2013-11-27  |  32
 28.0a1         | 2013-11-28  |  18
 28.0a1         | 2013-11-29  |  20
 28.0a1         | 2013-11-30  |  19
 28.0a1         | 2013-12-01  |  20
 28.0a1         | 2013-12-02  |  15
 28.0a1         | 2013-12-03  |  16
 28.0a1         | 2013-12-04  |   5
 28.0a1         | 2013-12-05  |   5
(10 rows)

This does look like a UI or maybe a middleware bug. Checking with :adrian!
Flags: needinfo?(sdeckelmann) → needinfo?(adrian)
The link in comment #1 concerns TCBS by build date. The table that service uses is `tcbs_build` and not `tcbs`. Selena, could you please check data in that table? 

See that Top crashers by crash date contains data for MetroFirefox: 

https://crash-stats.mozilla.com/topcrasher/products/MetroFirefox/versions/28.0a1/date_range_type/report/crash_type/all/os_name/None/result_count/50?days=7
Flags: needinfo?(adrian)
breakpad=# select version_string, report_date, sum(report_count) from tcbs_build JOIN product_versions USING (product_version_id) where product_version_id = 1919 and report_date > '2013-11-25' group by version_string, report_date order by report_date;
 version_string | report_date | sum 
----------------+-------------+-----
 28.0a1         | 2013-11-26  |  22
 28.0a1         | 2013-11-27  |  25
 28.0a1         | 2013-11-28  |  15
 28.0a1         | 2013-11-29  |  16
 28.0a1         | 2013-11-30  |  12
 28.0a1         | 2013-12-01  |  14
 28.0a1         | 2013-12-02  |   3
 28.0a1         | 2013-12-03  |   2
 28.0a1         | 2013-12-04  |   1
(9 rows)

Yup, data is there.
Flags: needinfo?(adrian)
(Reporter)

Updated

4 years ago
Whiteboard: [triage]
(Reporter)

Comment 9

4 years ago
Requesting tracking since we don't want to release without top crash reports.
tracking-firefox28: --- → ?
This is a database problem: 

breakpad=> select version_string, t.build_date, sum(report_count) from tcbs_build t JOIN product_versions USING (product_version_id) where product_version_id = 1919 and report_date > '2013-11-25' group by version_string, t.build_date order by t.build_date;
 version_string | build_date | sum 
----------------+------------+-----
 28.0a1         | 2013-11-21 |   2
 28.0a1         | 2013-11-22 |   4
 28.0a1         | 2013-11-23 |   6
 28.0a1         | 2013-11-24 |   8
 28.0a1         | 2013-11-25 |  31
 28.0a1         | 2013-11-26 |  23
 28.0a1         | 2013-11-27 |  19
 28.0a1         | 2013-11-28 |  17
(8 rows)

The build dates stopped after Nov 28, even though the report dates kept going. I have no idea where that could come from though. Can I bounce this back to you Selena?
Flags: needinfo?(sdeckelmann)
Flags: needinfo?(lars)
Flags: needinfo?(adrian)
(In reply to Jim Mathies [:jimm] from comment #1)
> Looks like we write out the wrong product name in extra info, and maybe
> that's what socorro uses to sort reports.

> ProductName=Firefox

I completely misunderstood this comment earlier.

So, is there a way to determine that this crash was a Metro crash? The ProductID or BuildID maybe?
Flags: needinfo?(sdeckelmann)

Comment 12

4 years ago
We definitely key the crash reports off the ProductName.

We can do rewrites in processing though to fix this if we have some other field to key on in addition, just like we do for FennecAndroid, see http://www.twobraids.com/2012/02/socorro-rule-system.html

Comment 13

4 years ago
If the ProductId (i.e. the ID used for add-ons to specify which product they apply to) is different between standard Firefox and Metro, then that would be easy to use as we already have this.
Metro app id:
ID={99bceaaa-e3c6-48c1-b981-ef9b46b67d60}
(Which is different from the desktop one)
And, turns out there is a feature that :rhelmer pointed me to and here was the fix: 

breakpad=# select * from product_productid_map ; 
    product_name     |               productid                | rewrite | version_began | version_ended 
---------------------+----------------------------------------+---------+---------------+---------------
 WebAppRuntimeMobile | {webapprtmobile@mozilla.com}           | f       | 16.0          | 
 B2G                 | {3c2e2abc-06d4-11e1-ac3b-374f68613e61} | f       | 17.0          | 
 MetroFirefox        | {99bceaaa-e3c6-48c1-b981-ef9b46b67d60} | f       | 16.0          | 
 WebappRuntime       | {webapprt@mozilla.org}                 | t       | 0.0           | 
 Fennec              | {a23983c0-fd0e-11dc-95ff-0800200c9a66} | f       | 0.1           | 
 FennecAndroid       | {aa3c5121-dab2-40e2-81ca-7ea25febc110} | t       | 0.1           | 
 Firefox             | {ec8030f7-c20a-464f-9b0e-13a3a9e97384} | f       | 0.7           | 
 Thunderbird         | {3550f703-e582-4d05-9a08-453d09bdfdc6} | f       | 0.3           | 
 SeaMonkey           | {92650c4d-4b8e-4d2a-b7eb-24ecf4f6b63a} | f       | 1.0a          | 
 Camino              | {camino@caminobrowser.org}             | f       | 0.0           | 
(10 rows)

breakpad=# begin; 
BEGIN
breakpad=# update product_productid_map set rewrite = 't' where product_name = 'MetroFirefox'; 
UPDATE 1
breakpad=# select * from product_productid_map; 
    product_name     |               productid                | rewrite | version_began | version_ended 
---------------------+----------------------------------------+---------+---------------+---------------
 WebAppRuntimeMobile | {webapprtmobile@mozilla.com}           | f       | 16.0          | 
 MetroFirefox        | {99bceaaa-e3c6-48c1-b981-ef9b46b67d60} | t       | 16.0          | 
 B2G                 | {3c2e2abc-06d4-11e1-ac3b-374f68613e61} | f       | 17.0          | 
 WebappRuntime       | {webapprt@mozilla.org}                 | t       | 0.0           | 
 Fennec              | {a23983c0-fd0e-11dc-95ff-0800200c9a66} | f       | 0.1           | 
 FennecAndroid       | {aa3c5121-dab2-40e2-81ca-7ea25febc110} | t       | 0.1           | 
 Firefox             | {ec8030f7-c20a-464f-9b0e-13a3a9e97384} | f       | 0.7           | 
 Thunderbird         | {3550f703-e582-4d05-9a08-453d09bdfdc6} | f       | 0.3           | 
 SeaMonkey           | {92650c4d-4b8e-4d2a-b7eb-24ecf4f6b63a} | f       | 1.0a          | 
 Camino              | {camino@caminobrowser.org}             | f       | 0.0           | 
(10 rows)

breakpad=# commit; 
COMMIT


I'll re-process the Metro Crashes and re-run TCBS and TCBS Build reports.
Summary: Crash reports stopped showing up for metrofx on 11/30 → 'Product' for MetroFirefox changed to just 'Firefox' in crash reports 11/30 - product_productid_map update needed
I've reprocessed 645 crashes and rewrite rules are operating as expected. For example: 

2013-12-10 11:42:27,827 DEBUG - Thread-27 - product name changed from Firefox to MetroFirefox based on productID {99bceaaa-e3c6-48c1-b981-ef9b46b67d60}

Now once these are done, I'll re-run TCBS reports for the last few days to pick them up.
Assignee: nobody → sdeckelmann
Started the backfilling job for the last 10 days. 

Should be done in 9 hrs or so.
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
(Reporter)

Comment 19

4 years ago
Thanks!
For the future - Selena, please let the DB team know when you're doing a large backfill like this, so we can be informed should a problem arise, as it did in bug 948772. Thanx!
Blocks: 948772

Updated

4 years ago
No longer blocks: 838081
tracking-firefox28: ? → ---
You need to log in before you can comment on or make changes to this bug.