534697 - UI New Trend Report - ADU

Status: RESOLVED FIXED

(Socorro :: General, task)

Description

[Austin King [:ozten]]

This is the UI half of Bug#526673

chowse is designing an integrated report UI, but for starters we can create a new Trend Report that graphs ADU against Total Number of Crashes per Prod/Ver/Platform

Comment 1

[Samuel Sidler (old account; do not CC)]

Isn't this exactly the same as bug 470620?

Comment 2

[Austin King [:ozten]]

My notes from last weeks meetings were to file a UI bug to track the frontend work. That one is assigned to lars.

Comment 3

[Samuel Sidler (old account; do not CC)]

Ah, ok.

Comment 4

[Austin King [:ozten]]

Sharing is caring.

Comment 5

[Chris Howse [:chowse]]

New ADU report mockups here:

   http://people.mozilla.org/~chowse/drop/crash_trends/adu/adu-report.png

Will have a more thoroughly documented version posted later today.

Targeting 3 major tasks with this mock:

  * What: Identify short-term crash trends.
    How: Plot crashes/user for a single version over the past 7-90 days.

  * What: Compare crash trends across recent versions.
    How: Plot crashes/user for multiple recent versions.

  * What: Identify OS versions with higher crash incidence.
    How: Plot crashes/user for each OS for a single version.

Comment 6

[Samuel Sidler (:ss)]

A couple of comments...

Overall, I like this design.

However, it's not clear to me how version numbers get chosen for the "Multiple Versions" section. What if I want 3.5.1 with 3.0.11 or something strange like that? Will that box list all known Firefox versions somehow?

It also wasn't clear how I align them, but then I noticed the checkbox for "Align start dates". That should be fine. :)

Comment 7

[chris hofmann]

yeah, offering/picking "interesting" releases gets complicated to do programatically.

its a combination of major releases with lots of users, or betas that we expect to reach lots of users.

ken also had to do a bunch of charting on this in the last few weeks so he might have some input and/or examples of the charts that he produced that people starting getting used to and understanding.

Comment 8

[Chris Howse [:chowse]]

(In reply to comment #7)
> yeah, offering/picking "interesting" releases gets complicated to do
> programatically.

If this is impractical, I can change the design to support arbitrary versions (using <select>s, auto-completes, or something similar). I'd still recommend having a few smart defaults, even if they're static. Could this be a piece of metadata that's associated with a version, like the version's release date?

And more examples would be welcome. A lot of this design is inspired by sample charts from 470620. :)

Comment 9

[chris hofmann]

I guess the list of candidates for wanting to graphs of comes from something like

https://crash-stats.mozilla.com/admin/branch_data_sources  

mostly "major" or "milestone" by Product (e.g. Firefox) are the ones of interest but maybe sometimes we might want to graph "development"

we could pick from a (long) list of those, or maybe even just type them in like I do when I want to see traffic graphs at alexa with their entry fields and [compare to]  button.  

http://www.alexa.com/siteinfo/mozilla.com?p=tgraph&r=home_home

google trends has just one entry field with comma separated values.

http://www.google.com/trends?q=firefox%2Ccrash&ctab=0&geo=all&date=all&sort=0

Comment 10

[Chris Howse [:chowse]]

(In reply to comment #9)
> I guess the list of candidates for wanting to graphs of comes from something
> like
> 
> https://crash-stats.mozilla.com/admin/branch_data_sources  

Doesn't seem to care for my LDAP password. :| But if it's like similar lists I've seen, it would be quite exhaustive to browse through to find the version you wanted.

> we could pick from a (long) list of those, or maybe even just type them in like
> I do when I want to see traffic graphs at alexa with their entry fields and
> [compare to]  button.  
> google trends has just one entry field with comma separated values.
> http://www.google.com/trends?q=firefox%2Ccrash&ctab=0&geo=all&date=all&sort=0

This would be my preference. When paired with auto-complete, this is by far the easiest way of inputting versions. Its also fairly easy to implement with existing libraries (http://docs.jquery.com/Plugins/Autocomplete), even with Google-style CSVs. The only downside is that it doesn't give you a complete listing of options available, but I'd venture that most users of this report could manage without, or would know where to find such a list elsewhere. Given the time available, I'd consider it a suitable compromise.

Comment 11

[Austin King [:ozten]]

Attachment: Screenshot of admin branches page

Comment 12

[Austin King [:ozten]]

(In reply to comment #10)
See screenshot since you don't have admin access.

Comment 13

[Ryan Snyder [:ryansnyder] [:rsnyder] [:rysny]]

Autocomplete is certainly a viable option.

Another option would be to only show checkbox inputs for all of the major and milestone versions of a product that have a release end date that is in the future.  This list could be expanded to show all versions for the selected product upon request.

Comment 14

[Ryan Snyder [:ryansnyder] [:rsnyder] [:rysny]]

Updated ADU API specs at http://code.google.com/p/socorro/wiki/AduAPI based on the most recent mockup at http://people.mozilla.org/~chowse/drop/crash_trends/adu/adu-report.png

Added:
* Ability to include multiple versions in the URI, to reduce the number of calls we will need to make from the front-end.
* Ability to query for a specific operating system or multiple operating systems.

Updated:
* Returned data to include multiple versions in the response.

Comment 15

[Chris Howse [:chowse]]

Updated wireframes:

   http://people.mozilla.org/~chowse/drop/crash_trends/adu/adu-report-wireframe.pdf

Mostly interface refinements and documentation, so no updates to AduAPI necessary. :)

There may still need to be a few changes to the chart (e.g. what ranges/scales to use, whether to group by week/month, etc.), but I'm refraining until we have some real data to test with. If I can get my hands on some CSVs beforehand, I'll experiment with it in Excel and update the wireframes with more detailed specs.

Comment 16

[Ryan Snyder [:ryansnyder] [:rsnyder] [:rysny]]

I'm working on implementing the Select a Report form for ADU, and I'm not sure that we need to have 3 separate forms for this page.  I think we can combine the 3 report options into 1 single form.

This Select a Report form would contain:
a. Versions - 4 version input fields
b. Operating Systems - 3/4 operating system checkboxes
c. Duration - The sliding scale selector ranging from 7 to 90 days
d. Generate - The Generate submit button

I definitely like some of the changes that you've made Chris, including the sliding scale selector for the Duration and the using input fields for the version selection.

Also, will this report be used for other products besides Firefox?  Do we need to be able to select a product other than Firefox in the Select a Report form?

Comment 17

[Chris Howse [:chowse]]

(In reply to comment #16)
> This Select a Report form would contain:
> a. Versions - 4 version input fields
> b. Operating Systems - 3/4 operating system checkboxes
> c. Duration - The sliding scale selector ranging from 7 to 90 days
> d. Generate - The Generate submit button

I like the idea, but the way (b) and (c) work for different reports causes some complications. (b) is used for filtering for Single/Multi-Version reports, but for comparison in Multi-OS reports. (c) also changes completely for the Multi-Version report.

I previously tried to create a unified UI that could handle all these variations, but it came across too convoluted and didn't serve the common use cases (from comment #5) as well. Having multiple forms introduces some redundancy, but if kept consistent, I believe it to be more usable.

For simplicity's sake, each form could be submitted to the same URL, using the same field names for common elements (e.g. the duration slider) and hidden elements to identify which form was used. Assuming you're not using AJAX, the common values could be distributed to each form on page refresh (e.g. if the user selected a 60 day duration, the duration slider for every form would be set to 60 days on page update).

> Also, will this report be used for other products besides Firefox?  Do we need
> to be able to select a product other than Firefox in the Select a Report form?

The same question applies to versions: the trend report menu in the top-right only shows reports for the latest version, and I can't find any UI for selecting a different version. Have people simply been changing it in the URL and bookmarking the trend reports they need? Is there any reason to keep this limited (e.g. trend reports are only available for a select few products/version)?

Assuming this is just an oversight, I could add some widgets above the Report forms so that you could select a different product/version (even something as simply as 2 <select>s).

Comment 18

[Ryan Snyder [:ryansnyder] [:rsnyder] [:rysny]]

For product and version selection, I definitely think this should be something that can be explicitly changed from the form.  Otherwise, there is no way to change this other than by changing the URL or exiting the page and re-entering from the page via a link on another page, both of which are tedious.

For the duration (c), I don't see a need to handle date selection separately between the single version and multi version forms. I understand that each version will have different start dates, but the user can manually change the date with the slider, so I think they'll be able to alter the form as needed to get the start dates for the query aligned as necessary.

For the single-version and multi-version forms, I personally think the 2 can be combined into 1.  The data returned will be the same, if I understand correctly, except that more data will be shown in the table and in the graph if more than 1 version is selected.  I've done a mockup of what it would look like to combine those forms (but still need to add the slider in for the Duration field):
http://bit.ly/55P5Sn

The multi-OS report is the one that I'm having trouble wrapping my head around.  The data / graph output for this particular selection seems to be different than the single-version and multi-version output.  How do you envision the data being represented for this query?

Comment 19

[Chris Howse [:chowse]]

(In reply to comment #18)
> For the duration (c), I don't see a need to handle date selection separately
> between the single version and multi version forms. I understand that each
> version will have different start dates, but the user can manually change the
> date with the slider, so I think they'll be able to alter the form as needed to
> get the start dates for the query aligned as necessary.

I think we're interpreting 'aligned' in different ways. When I talk about 'aligning' start dates, I don't mean just finding the earliest date. I mean actually moving all the lines to the left until they begin at the same x-coordinate. If the user is comparing 3.6 to 3.5, and they were released on 11/01 and 06/01 respectively, then 3.6's data would be moved back 5 months so it begins on the same day as 3.5. You can then compare crash trends from the same points in their lifecycles.

As for adjusting the time frame slider: if this trend report's performance is similar to existing ones, generating it is slow and resource-intensive, making it unagreeable with trial and error. If we know a version's release date (which it appears we do from the Admin panel), we should provide an automated way of using that as a time period (i.e. 'Since release date' should be an option alongside 'Last 30 days').

> For the single-version and multi-version forms, I personally think the 2 can be
> combined into 1.  The data returned will be the same, if I understand
> correctly, except that more data will be shown in the table and in the graph if
> more than 1 version is selected.  I've done a mockup of what it would look like
> to combine those forms (but still need to add the slider in for the Duration
> field):
> http://bit.ly/55P5Sn

You are correct, the data presented for each is very similar. I'd even go far to say that 'Single Version' is just a specialization of 'Multiple Version'. The differences lie in the time periods that are being explored and how they are specified. 'Single Version' is about looking at recent trends (e.g. has the incidence of crashes improved in the last month?). 'Multiple Version' is about looking at historic trends (e.g. has the incidence of crashes in 3.6 improved at the same rate that it did for 3.5?).

> The multi-OS report is the one that I'm having trouble wrapping my head around.
>  The data / graph output for this particular selection seems to be different
> than the single-version and multi-version output.  How do you envision the data
> being represented for this query?

Each report has a different set of data series (i.e. lines in the chart, columns in the table) and supports different kinds of comparisons. The Multi-OS report has a data series for each checked OS. For comparison, here are some sample data series for each report:

   SINGLE VERSION     MULTIPLE VERSION    MULTIPLE OS
   ==============     ================    ===========
   3.5.6 (Every OS)   3.5.6 (Every OS)    3.5.6 (Every OS)
                      3.5.5 (Every OS)    3.5.6 (Windows)
                      3.0.13 (Every OS)   3.5.6 (Mac)
                      3.0.11 (Every OS)   3.5.6 (Linux)                      

Each report supports a different kinds of comparisons and discoveries. Multi-OS lets you see if one particular OS has a higher incidence of crashes. Multi-version lets you see if crashes are trending differently across versions. Single-version doesn't really afford comparisons, it's just for identifying recent trends. Maybe I should have named based on the kinds of comparisons they were intended to support (No Comparsion, Compare by Version, Compare by OS).

I hope this clarifies my design choices a bit. For comparison, I'll include 2 of my earlier designs which used a single form. I'll throw in some of the product selection UI while I'm at it. My preference is still for multiple forms, but until we can get feedback from Release/Metrics, I don't want to let this one detail slow progress.

Comment 20

[Chris Howse [:chowse]]

Attachment: Alternate UI for Specifying Report Options

From previous iterations. Uses a single form for all options.

Comment 21

[Ryan Snyder [:ryansnyder] [:rsnyder] [:rysny]]

Thanks Chris - these explanations were very helpful.  My main concern was that developers wouldn't understand the difference between the expected results provided by each of the forms, but I can totally work with what you've provided.  I'll dig in tomorrow and get rolling again on implementing your design.  Thanks!

Comment 22

[Ryan Snyder [:ryansnyder] [:rsnyder] [:rysny]]

Attachment: Patch 1 for 534697

Implementing ADU report by version and report by operating system.  Working version available at http://rsnyder.khan.mozilla.org/reporter/daily

Comment 23

[Austin King [:ozten]]

Comment on attachment 419988 [details] [diff] [review]
Patch 1 for 534697

Code Review:
Great work. This is an awesome report.

General notes:
Be sure to include Upgrades notes on daily.php-dist

Many files such as the controller and model had tabs vs spaces formatting issues

r- for two issues, the rest is just nits or feedback for discussion
1) remove inline JS
2) remove eval in daily.js

Detailed review:

= controllers/daily.php =

line: 47 - the function 'csv' could reuse $this->renderCSV($title) if the controller was more RESTFul and the $form_selection was part of the url instead of a query string parameter.
Example: http://aking.khan.mozilla.org/reporter/daily/daily_csv_by_version? ... etc

nits: $this->get is a confusing object property and isn't used outside of the 'index' function. Remove it and make it a function scoped variable named params, data, or something along those lines.

= models/daily.php =
line: 203 Consider escaping data into temp variables with short names so you can have a 1 line url like
$resp = $service->get("${host}/200911/topcrash/sig/trend/history/p/${p}/v/${v}/sig/${rsig}/end/${end_date}/duration/180/steps/60",
which is easier to read than a series of string concatenations.

line: 204 When no data is available or a service error occurs, the UI doesn't show any error message, instead it logs an error and displays an empty UI.
We should try to inform the user what happened and not display the graph, etc.
Maybe remove 'get' function and put this code into the controller?

nits:
Daily_Model subclasses Model which has a bunch of DB code baked in... I don't like it, but it's probably not worth the effort to refactor this under a new Webservice base class, etc. In top crashers, I put this logic into the controller, since there is very little 'model' type behavior going on.


Views - Looking good, nice and clean
* remove inline JavaScript [please fix 1]
* include JS and CSS from the view, not in the controller
* daily_index.php - sub views are setup in the controller, versus the overall view. Is there a compelling reason for this? Otherwise for consistency, only setup view data in controller and manage view inclusion from the views.
** typically the "master" view should be the same name as the controller action and the "sub-views" will be under views/common/
* daily_csv_by_version.php Why do we html encode values which are being output as CSV?
* daily_csv_by_version.php Lots of logic instead of being "view data". Check out common/csv.php. Basically the controller prepares a nested array and uses csv.php to output the data.

js/socorro/daily.js
Why are we using eval here? [please fix 2]

Comment 24

[Ryan Snyder [:ryansnyder] [:rsnyder] [:rysny]]

Attachment: Patch 2 for 534697

Thanks Austin!  

This follow-up patch includes the following:

* Removed eval() from js/socorro/daily.js.
* Removed inline jquery
* Refactored Daily_Model::formatURL()
* Removed Model extension from Daily_Model
* Refactored $this->get in Daily_Controller()
* Moved javascript and css statements from controller to view
* Escaping removed from csv views. Oops.
* Displaying "no results" message when no results are available
* Moved views/daily_index.php to views/index.php and included View instantiation within that view

Notes:
* Notes on daily.php-dist had already been added to http://code.google.com/p/socorro/wiki/SocorroUpgrade#Socorro_1.3
* The implemented ReST in Crash Stats differs from my understanding of ReST.  Would be a good future discussion.
* I agree that the CSV could be refactored. I can't get that in by code freeze.

Comment 25

[Austin King [:ozten]]

Comment on attachment 420242 [details] [diff] [review]
Patch 2 for 534697

Great work.

Comment 26

[Ryan Snyder [:ryansnyder] [:rsnyder] [:rysny]]

Thanks Austin!  Committing code...

==

Adding         webapp-php/application/config/daily.php-dist
Adding         webapp-php/application/controllers/daily.php
Sending        webapp-php/application/libraries/MY_Controller.php
Sending        webapp-php/application/libraries/timeutil.php
Sending        webapp-php/application/models/branch.php
Adding         webapp-php/application/models/daily.php
Adding         webapp-php/application/views/daily
Adding         webapp-php/application/views/daily/daily_crash_data_by_os.php
Adding        
webapp-php/application/views/daily/daily_crash_data_by_version.php
Adding         webapp-php/application/views/daily/daily_csv_by_os.php
Adding         webapp-php/application/views/daily/daily_csv_by_version.php
Adding         webapp-php/application/views/daily/daily_search.php
Adding         webapp-php/application/views/daily/index.php
Sending        webapp-php/application/views/layout.php
Adding         webapp-php/css/daily.css
Adding         webapp-php/js/socorro/daily.js
Transmitting file data ...............
Committed revision r1676.