Closed Bug 1118310 Opened 9 years ago Closed 7 years ago

Dashboard: users affected by Flash hangs

Categories

(Data Platform and Tools :: General, defect)

Product:
Data Platform and Tools
Component:
General
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED WONTFIX

People

(Reporter: benjamin, Assigned: hulmer)

References

Details

User Story

For each week, take all users active during the week:

* Group them by the number of plugin hangs that occurred at any point during the week.
* Calculate a MTBF by dividing the number of plugin hangs by the total session length

Plotted week over week (for the past 180 days):

* plot the % of active users with no plugin hangs at all
* plot the MTBF

It should be possible to filter/group by the following items:
* release channel
* operating system
* OS version (Windows only)

In order to produce this for the release channel, it will be necessary to extract data from FHR. It may be possible to do the same for the prerelease channels, but it may be faster to aggregate the prerelease channels separately using telemetry data.

The scripts used to collect data and product the dashboard should land into version control somewhere. The dashboard should be set up on a cron job to update automatically (weekly).
A Q1 goal for our team is to reduce the number of users affected by Flash hangs. In order to measure and track this properly, I need a regularly-updating dashboard as specified in the "user story" field.
Hardware: x86_64 → All
(does this bug need mentoring, and is it intended for community or in house?)
I don't think that a non-employee could currently create this dashboard: we don't currently have contributor access to either FHR or telemetry data in a way that would make this easily possible.
(I am happy to own / maintain an FHR script here if needful, and if metrics doesn't want in.)
Gregg, that would make me happy! Please go for it. Do you already have access to the peach cluster?

We can probably do this against the weekly 5% sample. If you can publish results to some public JSON spot, there are a number of people (employees or volunteers) who can help with the HTML/JS to produce the graphs.
Assignee: nobody → hulmer
Gregg + :bsmedberg - 

I don't have the bandwidth in the next week or so to crunch the FHR data, but metrics would be more than happy to provide the dashboard, since we already have all that machinery in place to do precisely what is stated above.
The data crunching is the important part of this bug. The data extraction which pulls the rest of the data for your dashboard does not appear to be in github, so I can't write patches to pull this data. Joy, is that something you can help with (either adding the extraction yourself, or putting up the extraction scripts so I can write something)?
Flags: needinfo?(sguha)
I wont have the time this week to get that data. I can in a week or so.
Flags: needinfo?(sguha)
(In reply to Benjamin Smedberg  [:bsmedberg] from comment #6)
> The data crunching is the important part of this bug. The data extraction
> which pulls the rest of the data for your dashboard does not appear to be in
> github, so I can't write patches to pull this data. Joy, is that something
> you can help with (either adding the extraction yourself, or putting up the
> extraction scripts so I can write something)?

Hi - 

I made a private github repo here for the Fennec data. It uses mrjob, and is hopefully not COMPLETELY unreadable. This is not for Firefox, unfortunately, which is the target you want at any rate. If you're not part of the metrics gorup I can add you - just shoot me an email.

https://github.com/mozilla/fennec-rollup-script

HTH
:bsmedberg I have the bandwidth to work on this now if no one has claimed it yet.

Believe I just need to look at plugin-hang instances within sessions, outlined here:

https://hg.mozilla.org/mozilla-central/file/tip/services/healthreport/docs/dataformat.rst#l1137
Yes.
Great. I'll be working on this here: https://github.com/mozilla/plugin-crash-dashboard

I take your first comment to mean that we are assuming all plugin hangs are Flash? I'm behind on this conversation so sorry if I'm asking something already brought up.
It's a safe-enough assumption to go with for now.
Sounds good. I'll have something this week by thursday afternoon - testing the script against sample data now, and a dashboard will be quick to put together once I've verified that I've caught all potential malformed things.
https://metrics.mozilla.com/protected/dashboards/plugin-hang/

Sorry for the delay in getting this together. Still experiencing some problems of subsetting on os version, given that you're looking at the windows OS.
Blocks: 1210637
With Flash going click-to-play and this not having activity in 2 years, I'm gonna go ahead and WONTFIX this.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → WONTFIX
Product: Webtools → Data Platform and Tools
Component: Telemetry Dashboards (TMO) → General
You need to log in before you can comment on or make changes to this bug.