Closed Bug 736484 Opened 8 years ago Closed 5 years ago

[Telemetry] Improve UX of Telemetry dashboard

Categories

(Mozilla Metrics :: Frontend Reports, enhancement)

enhancement
Not set

Tracking

(Not tracked)

RESOLVED WONTFIX
Moved to JIRA

People

(Reporter: lmandel, Unassigned)

References

(Depends on 1 open bug)

Details

(Whiteboard: [JIRA BIV-6] [Telemetry:P1])

Attachments

(1 file)

816.05 KB, application/octet-stream
Details
As the number of Telemetry probes has grown, Telemetry is being used by more people including devs, managers, release drivers, etc. Many people new to Telemetry need to be shown around before being able to make use of the Telemetry and Telemetry Evolution dashboards. I think we can improve the UX to make the dashboard more useable.

Some suggestions:
- descriptive text for Telemetry and Evolution - in what cases should I use each?
- descriptive text for the histograms
- provide a method to find a probe that is of interest. this may be adding a description or keywords for each probe, a search mechanism, or something else
- make it clear how to apply filters
Whiteboard: [Telemetry] → [Telemetry:P1]
I think the thing to do here is to put an actual designer on it.
@Lawrence: 

I understand your indications and absolutely agree that if we have a wide range of potential users at this stage, we should focus on decreasing the learning curve of the dashboard. Your indications are very constructive and really helpful. I see you understand that any Design process and specially a Dashboard Design is driven by the quality of the input provided.
We are doing a huge effort trying to fit all the requirements from your team and will continue doing so. 

First thing Monday morning I will personally see that an updated version of the dashboard ( including your input) gets into your hands so that we can discuss on it and do a serious effort to see this dashboard stepping up to your expectations.

Thanks,
Nuno
There at two different methods I would like us to consider.

1. Cleanup and refactor -- This would be an approach if you feel that most of the layout and functionality is suitable for the various different classes of users, and you just want to get design polish and fix rough spots.

2. Design overhaul -- This would be a process where we take everything we have learned so far from using Telemetry and spend some time formally designing one or more dashboards to meet the needs of each distinct class of user.  We would start with looking at how the needs differ between users, then look at which visualizations or metrics work well for each need.  When the needs at significantly different, we should separate out the dashboards to reduce confusion.  After we have a good understanding of the needs and metrics, the designers can mock up a few different approaches for review and it would be good to run those designs by the different types of users to see if they look promising rather than guessing and deciding for them.
Taras - Nuno and Boriss are both UX.

Daniel, I would like to get UX input on your questions. It's not my intention to dictate what should be done with the Telemetry dashboards. The dashboards continue to provide more information to the Mozilla dev community and continue to be used by more people. With the increased attention to Telemetry I want to ensure that the dashboard is functioning well from a UX perspective as well as a data perspective.
Absolutely understood. 
First. I tried to get something for us to discuss today but I will have to postpone to tomorrow ( it's father's day and I'm already slightly late for my kid's party at school).

2nd. After reading Daniel's reply and understanding that we might be facing more than simple info inserts on the the dashboards, I suggest we all find some time to meet via skype or vydeo and talk through your concerns and identify all the areas we will need intervention on. I would suggest that we start by dividing the dashboard into sections and identifying them ( .1; .1.1; .2  etc) - describe the objectives and additional requirements you might have on each of the sections. This way we would be building a solid document that would give us ( me and Boriss) a stable document to work on.  
What do you think ?

Thx,
Nuno
Attached file Item list PDF
Hello all,

In an effort to coordinate information flow, I have identified each item of the current dashboards.  If you agree, we should open a wiki to combine comments on the identified items and add others that are not present on the dashboards.
The plain text list is below. 


Review Items

TH Telemetry Histogram

TH0. Filters
	TH0.1 Selectors  
	TH0.2 Range

TH1. Header
	TH1.1 Dashboard navigation 
	TH1.2 Date Range Selector
	TH1.3 Chart Visualization

TH2. Histogram
	TH2.1 Measure Selector 
	TH2.2 Comparison system
	TH2.3 Chart Visualization
	TH2.3 Info bar

TH3. Submissions By Date
	TH3.1 Chart Visualization

TH4. Submissions By Platfom ID
	TH4.1 Chart Mode Selector
	TH4.2 Chart Visualization

--------------------------------------------------------------------------------

TE Telemetry Evolution

TE0. Filters
	TE0.1 Selectors  
	TE0.2 Info note

TE1. Header
	TE1.1 Dashboard Navigation 

TE2. Histogram
	TE2.1 Measure Selector 
	TE2.2 Chart Visualization

TE3. Latest Changes Table

Thanks, 
Nuno
Nuno, thanks for creating the list and for the visually informative PDF. I'm happy to meet to discuss the Telemetry dash UX. Feel free to use any tools (you mentioned wiki) that you need.

Some questions that I've fielded directly from Telemetry users:
* Do you have a metric for X? (Can't determine what probe to select.)
* How do I filter on a release?
* What am I looking at? Or, what do the numbers represent?
* What does the "Sampled to %" line mean?
* How many daily submissions does Telemetry get?

Note that in all cases it is not a question of if the data is in Telemetry. With some explanation everyone that I've helped has found what they need. The issue is that these people need hand holding to find the data. Can we improve the UX so that more people can find what they need on their own?
Migrated to be tracked in JIRA with the rest of metrics work. Internal folks see METRICS-774 for replacement.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → INCOMPLETE
Work now being tracked as part of: https://metrics.mozilla.com/projects/browse/METRICS-774
updating whiteboard
Whiteboard: [Telemetry:P1] → Telemetry P1 migrated to JIRA
updating status and assignee
Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---
Specific JIRA: https://metrics.mozilla.com/projects/browse/METRICS-834
Status: REOPENED → ASSIGNED
New JIRA migration identifier - 55 BZ issues being considered in current planning for Q3
Target Milestone: Unreviewed → Targeted - JIRA
We really need to do something about this. Telemetry data is fantastic but right now it's basically restricted to people who can and are willing to jump through a bunch of hoops. We want as many eyes as possible looking for good bits of data. Its neglected UI is also a pretty big distrust indicator for anyone who does look at it. It looks like a forgotten or neglected project, not the central component of Mozilla's data-driven development efforts that it is.

1. It's not even clear to me what the official URL is. I have to look it up painfully every time. Let's just do the obvious thing - "telemetry.mozilla.com".

2. The main page just looks broken, like the content didn't load. Can we at least write some text telling you to log in in order to see something?

3. Please describe histogram vs. evolution. This should be a really simple thing to fix, and it'll help immensely.
Depends on: 776712, 776714, 776716
Whiteboard: Telemetry P1 migrated to JIRA → [JIRA BIV-6] Telemetry P1
Whiteboard: [JIRA BIV-6] Telemetry P1 → [JIRA BIV-6] [Telemetry:P1]
There are new Telemetry metrics in the pipeline, any requests should go through that.
Status: ASSIGNED → RESOLVED
Closed: 7 years ago5 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.