Questions regarding Mac 67 Nightly Content Data
Categories
(Cloud Services :: Mission Control, enhancement)
Tracking
(Not tracked)
People
(Reporter: marcia, Assigned: wlach)
Details
During the Channel meeting today we discussed the following items:
(1) The fact that we cannot see any Mac content information in https://missioncontrol.telemetry.mozilla.org/#/nightly/mac. Is it 0 because there isn't enough data to report on the platform?
(2) For 66 Nightly it is reporting 8.10 which sounds a bit high
(3) https://missioncontrol.telemetry.mozilla.org/#/?channel=nightly, Mac shows .18 which seems quite low, even compared to Linux.
|
Assignee | |
Comment 1•5 years ago
|
||
Yeah, there's definitely a bug here, I think it's related to the low volume of crashes we're seeing on Mac nightly. Still trying to figure out the root cause.
Will ping back when I know more / have a fix.
|
|
Assignee | |
Comment 2•5 years ago
|
||
(In reply to Marcia Knous [:marcia - needinfo? me] from comment #0)
During the Channel meeting today we discussed the following items:
(1) The fact that we cannot see any Mac content information in https://missioncontrol.telemetry.mozilla.org/#/nightly/mac. Is it 0 because there isn't enough data to report on the platform?
Ok, yeah, the problem is this innocuous-looking piece of code:
We take the 99th percentile of 5-minute counts of errors when calculating the rate. This normally works fine and serves to automatically reject cases where one or two pings have a ridiculous error count (thus skewing the aggregate count). However, on Mac, less than 1% of the 5 minute error aggregations had any content crashes at all, thus we'd calculate a rate of 0.
I don't really want to spend a huge amount of time on this -- I think we already know at this point that the error aggregates table is dumb and we should replace it with something better (i.e. a dynamic aggregation of individual pings which can automatically reject outliers). So I'm going to do the minimal thing and bump the threshold to the 99.9th percentile. That makes the Mac results make sense again and shouldn't effect other rate calculations too much.
(2) For 66 Nightly it is reporting 8.10 which sounds a bit high
This is the adjusted / all issue again. The default (adjusted) compares release-to-release only for the duration that the most recent one has been around. If you click on "all" on the dashboard it should give you a number more in line with what you'd expect.
I'm going to make "all" the default since it seems to be the one that people expect to see.
(3) https://missioncontrol.telemetry.mozilla.org/#/?channel=nightly, Mac shows .18 which seems quite low, even compared to Linux.
This is related to (1).
|
|
Assignee | |
Comment 3•5 years ago
|
||
Changed the rate caculation in mc:
https://github.com/mozilla/missioncontrol/pull/353
You can see the new results here:
https://data-missioncontrol.dev.mozaws.net/#/?channel=nightly
As you can see, this revises the numbers upwards on channels with a lower amount of data (primarily mac and linux on beta/nightly), but the main takeaway is that the number of content crashes is now closer to what we'd expect.
Assuming there are no major objections, I'll get whd to promote this to the main missioncontrol site (https://missioncontrol.telemetry.mozilla.org) tomorrow or so.
|
|
Assignee | |
Comment 4•5 years ago
|
||
Fix has been promoted, going to call this done. Should probably blog about this at some point.
Description
•