640241 - Get chofmann's plugin crash with Flash version reports into Socorro

Status: RESOLVED WONTFIX

(Socorro :: General, task)

Description

[Robert Kaiser]

We should work on integrating a number of current "third-party" reports into Socorro proper, with the benefit of better optimized data gathering (using our databases) and nicer UI.

One of those is the plugin crash with Flash version reports chofmann has in http://people.mozilla.org/~chofmann/crash-stats/ (top-plugin-crashes-*.html in the dated subdirs, if I'm seeing things correctly).

chofmann will fill in further details on that.

Comment 1

[Laura Thomson :laura]

Wait for 1.9 to see if we can do this with ES.

Comment 2

[Brandon Savage [:brandon]]

For the sake of time, moving off to 2.1

Comment 3

[Schalk Neethling [:espressive]]

Attachment: Initial state with first table hidden

Comment 4

[Schalk Neethling [:espressive]]

Attachment: Expanded state, showing first table

Comment 5

[Schalk Neethling [:espressive]]

First, I would love some feedback on the two screenshots added.
Second, is there anything that should be highlighted as in the other report. Something that should be visible right off the bat.

Comment 6

[Schalk Neethling [:espressive]]

I guess I need to provide some context for the screen shots ;) They are based on https://crash-analysis.mozilla.com/chofmann/20111104/top-plugin-crashes-10.0a1.html

Comment 7

[chris hofmann]

the crash per user part of the my report just uses data from https://crash-stats.mozilla.com/daily?p=Firefox&v[]=

we could suck in data for the specific release you are looking at.  that makes it handy to put the top crash list in perspective of where the data is trending.

the other parts of the report that use either "in firefox releases" or "in flash releases"  look fine.   In some ways it might be useful for users to configure or toggle that column selection depending on if they are trying to associate the crashes with given releases of firefox or flash.   it might also be interesting to put both of those columns next to each other on a report to help spot interesting combonations where specific releases of both might be behind compatibility crash problems.

Comment 8

[Schalk Neethling [:espressive]]

Thanks for the feedback Chris

Comment 9

[Schalk Neethling [:espressive]]

"that makes it handy to put the top crash list in perspective of where the data is trending." - So you feel this should not be hidden by default but, give the user the option to hide them if so desired? (refering to the first table here)

Comment 10

[chris hofmann]

you might want to get some feedback from others on that.

Comment 11

[Schalk Neethling [:espressive]]

Chris so, can those two tables be mixed into one? 

Reason I ask is, the first has the Firefox crashes and he second the Flash versions so, if we want those two side be side, there will need to be a mix of the two, unless of course, I am missing something. Which is completely possible ;)

Comment 12

[Robert Kaiser]

I don't think we need a ADU data table in there, we have that accessible easily in Socorro elsewhere. Chris, why is that report more special than others in possibly wanting the perspective on one view?

Comment 13

[chris hofmann]

I agree the data is elsewhere, but here is the rational for putting on the page.

If I'm trying to figure out how crashy beta is,  I look at my beta page and all the data is there.  ADU and crashes per 100 users first tells me how things are going with uptake and submitted crashes.  If we have lots of users and not many crashes I might not look deeper into the top crash list, and move on to analyzing another channel.

If there are problems with those numbers I'd move deeper into looking at the top crash list for that channel, and drilling down on particular crashes.

Comment 14

[Schalk Neethling [:espressive]]

Chris, I get what you are saying. I am going to work on a revision and will add the revised prototype a little later today.

Comment 15

[Robert Kaiser]

(In reply to chris hofmann from comment #13)
> I agree the data is elsewhere, but here is the rational for putting on the
> page.

Well, when the report is directly in Socorro, I already see the overall crashiness of that version in its overview page, before I get to this report at all, i.e. I'm not opening this report at all if I don't want to dig deeper.

Comment 16

[Schalk Neethling [:espressive]]

Attachment: Initial state rev1

So this is now with the ADU table visible and the detailed report hidden in the initial state.

Comment 17

[Schalk Neethling [:espressive]]

Attachment: Expanded state rev1

This is then the user driven expanded state with the details visible.

Comment 18

[Schalk Neethling [:espressive]]

Attachment: Toggle columns

You can also toggle the Firefox crashes and In Flash Releases columns on and off.

Comment 19

[Robert Kaiser]

(In reply to Schalk Neethling from comment #16)
> Created attachment 572950 [details]
> Initial state rev1
> 
> So this is now with the ADU table visible and the detailed report hidden in
> the initial state.

Umm, if anything, it should be the other way round and the actual report shown but the ADU table hidden. And I still argue that we don't even want the ADU display on this page.

(In reply to Schalk Neethling from comment #17)
> Created attachment 572953 [details]
> Expanded state rev1
> 
> This is then the user driven expanded state with the details visible.

The "Firefox Crashes" column should just be "Crashes", the report is for a specific product and version from the beginning, no need to name columns after that as well. And what's the difference between "Count" and "Firefox Crashes"?
The "In Flash Releases" column is hard to read, can those items be at least separated by commas just like the bug IDs?
The Bug IDs should be a separate link for each, with the info about the bug in a tooltip and the ID striked through when it's fixed, just like we do it in the top crashers report.
Why do we need to "Toggle" buttons? We don't have something like that in other reports right now.

Comment 20

[Schalk Neethling [:espressive]]

Thanks for the feedback KaiRo.

So, the reason I swapped the visibility around to the way I had it initially was based on the comment (13) from Chris:

"ADU and crashes per 100 users first tells me how things are going with uptake and submitted crashes.  If we have lots of users and not many crashes I might not look deeper into the top crash list, and move on to analyzing another channel."

Second, will change the column heading name.

Next, "difference between "Count" and "Firefox Crashes" - Guess Chris needs to answer this one.

Will definitely seperate those with a comma to make it more readable.

I saw that the current links points to a search for the bug id's so, this is why all of them points to one link. We can definitely change it around to behave the way you suggests.

Then lastly, the toggle behaviour was added based on the following comment(7)

"In some ways it might be useful for users to configure or toggle that column selection depending on if they are trying to associate the crashes with given releases of firefox or flash."

Comment 21

[Robert Kaiser]

(In reply to Schalk Neethling from comment #20)
> So, the reason I swapped the visibility around to the way I had it initially
> was based on the comment (13) from Chris

As I stated in comment #15, when this is directly in Socorro, we get that info before we even hit this report, so I think it's not useful there.

> Next, "difference between "Count" and "Firefox Crashes" - Guess Chris needs
> to answer this one.

OK, waiting for that, it's confusing to me at least so we'll need to explain it in some way anyhow.

> Then lastly, the toggle behaviour was added based on the following comment(7)
> 
> "In some ways it might be useful for users to configure or toggle that
> column selection depending on if they are trying to associate the crashes
> with given releases of firefox or flash."

Well, if I'm looking at a report of "plugin crashes with Flash version" I pretty much always expect the Flash versions to be visible...

Comment 22

[Schalk Neethling [:espressive]]

Any updates on this one?

Comment 23

[chris hofmann]

Yes "Count" is the number of "Firefox Crashes"

Comment 24

[Robert Kaiser]

(In reply to chris hofmann from comment #23)
> Yes "Count" is the number of "Firefox Crashes"

Why are they two different columns then?

Comment 25

[Schalk Neethling [:espressive]]

I reckon I should simply drop the count column

Comment 26

[Robert Kaiser]

If anything, drop the "Firefox Crashes" column, as it's somewhat unsure what it really is, while the table seems to be sorted by the "Count" column, so that seems to be the amount of crashes of the current version (which I'd expect). Also, having "Firefox" in a column title doesn't work with support of other products.

Comment 27

[Schalk Neethling [:espressive]]

Yes, I remember you mentioning about not having a product name in these titles, will remove that one.

Comment 28

[Schalk Neethling [:espressive]]

Attachment: Latest revision based on feedback from KaiRo and choffman

Comment 29

[Schalk Neethling [:espressive]]

What will this report be called once in Socorro?
Some ideas:

Top Plugin Crashes (think this already exists actually) 
Top Crashes with Flash
Top Flash Crashes
Plugin With Flash Crashes

Comment 30

[Robert Kaiser]

After some discussion on IRC, let's call it "Top Flash Crashes" in the UI and unlike chofmann's only have it list only Flash ones, the backend should be able to do that by filtering on the plugin name.

Comment 31

[Robert Kaiser]

(In reply to Schalk Neethling from comment #28)
> Created attachment 576127 [details]
> Latest revision based on feedback from KaiRo and choffman

This one looks good, though the Flash version column is still slightly buggy - if there is only [blank] that should be "none", it's surely not "one or more" as "1+" would suggest. Also, there's a leading comma after the colon that should not be there. And then, I'm not even sure if we need the leading counter of different Flash versions at all.

Comment 32

[Schalk Neethling [:espressive]]

Thanks for the feedback. The leading counter is not always available for each version group for example here:

11+:, 11.0.1.152, none, 10.3.181.14, 10.0.32.18, 10.3.183.5, 10.3.181.34, 10.3.181.26, 10.3.181.22, 10.1.53.64, 10.1.102.64

It has a version counter for 11 but not 10, unless what was [blank] should have been the leading version counter. I am also unsure whether these are really version counters as I am sure no one is using Flash version 1 anymore ;)

For example: 1+:, 10.3.183.10

I am sure the 1+: should be 10+:

Maybe just drop these altogether?

Comment 33

[Schalk Neethling [:espressive]]

Attachment: Changes [blank] to none, removed leading counter and related comma

Comment 34

[Robert Kaiser]

I like that better, yes. Now let's see if Chris agrees. :)

Comment 35

[chris hofmann]

the report looks fine, but I'm not sure if "none" is correct. to me "none" says that flash might not be involved in some set of the crashes.

for some reports we don't get accurate data about the flash version due to memory corruption or a variety of other reasons.  [blank] is more reflective of the situation of us just not knowing if flash was involved or not.

Comment 36

[Schalk Neethling [:espressive]]

(In reply to chris hofmann from comment #35)
> the report looks fine, but I'm not sure if "none" is correct. to me "none"
> says that flash might not be involved in some set of the crashes.
> 
> for some reports we don't get accurate data about the flash version due to
> memory corruption or a variety of other reasons.  [blank] is more reflective
> of the situation of us just not knowing if flash was involved or not.

Maybe [unknown] or [unknown version]?

Comment 37

[Robert Kaiser]

(In reply to Schalk Neethling from comment #36)
> Maybe [unknown] or [unknown version]?

Hah, had almost the same thought, but I don't see any reason for using square brackets around it. Also "version" would be redundant with the column header, we already know it's versions there. I'd therefore go with plain "unknown".

Comment 38

[chris hofmann]

to me "unknown" implies flash doesn't know what version it is, or the crash reporting system can't figure it out.  It's really more of a case that the data is "missing".

How about we go with "missing"

Comment 39

[Robert Kaiser]

(In reply to chris hofmann from comment #38)
> to me "unknown" implies flash doesn't know what version it is, or the crash
> reporting system can't figure it out.  It's really more of a case that the
> data is "missing".

Actually, that (particularly the latter) is even the case in a number of reports comping in, esp. on Mac or Linux/Android.

> How about we go with "missing"

Works for me as well.

Comment 40

[Schalk Neethling [:espressive]]

[:bsavage] The skeleton and all UI code has been added. The branch is at:
https://github.com/ossreleasefeed/socorro/tree/top_flash_crashes

Demo page: https://espressive-dev.allizom.org/topflashcrashes/index

Comment 41

[[:jberkus] Josh Berkus]

Espressive,

Presumably you could use a matview for this?

Comment 42

[[:jberkus] Josh Berkus]

Not sure why this was assigned to me.  Reassigning to Schalk.

Comment 43

[Laura Thomson :laura]

Is this still wanted?

Comment 44

[Robert Kaiser]

We probably will want something like this, but I think it may not be in this exact form, and we'll mostly be able to achieve it through searches.

Comment 45

[Lonnen :lonnen]

we are not investing in new, specific reports