448894 - Attempt to remedy user confusion with useless "details" button in crash reporter

Status: NEW

(Toolkit :: Crash Reporting, defect)

Description

[Dave Garrett]

Problem: Firefox crashes and the Mozilla crash reporter comes up. The user decides to file a support request or bug report, so they click the "details" button and copy the text. They restart and paste it into a report, however this information isn't really that useful. To do much of anything, we need the crash report ID from about:crashes.

This is confusing to first-time users of this system. I suggest the "details" button be removed from the initial dialog and moved to a new "thank you" dialog to popup after the report has been successfully submitted. Thus, we'll have the crash ID at this point and can include it in the details.

Comment 1

[(not currently active) Ted Mielczarek]

I don't think we want to do this. We tried to optimize for the more common case (we think) where the user isn't going to file a bug and doesn't care about the report. I think we could make the details text not selectable to avoid some of this problem. 

Comment 2

[Jesse Ruderman]

We only need to modify the experience for users who click Details; we don't need to change the common case at all.  The solution could be as simple as adding a note when you click Details: "After submitting this raw crash report, load about:crashes in Firefox to see the processed report".

Making the Details text non-selectable isn't much of a solution.  The same users who copy and paste it today will take screenshots instead.

Comment 3

[Dave Garrett]

I'm not sure how the ID is generated, but is it possible to just generate the potential ID beforehand and stick it in the details box as-is?

Comment 4

[Samuel Sidler (old account; do not CC)]

The ID is generated server-side.

I'm opposed to doing this (as filed), but think that adding some explanatory text about about:crashes to the Details window would be fine (comment 2).

Comment 5

[Dave Garrett]

Comment #2 seems like a good simple compromise to deal with the problem.

Comment 6

[Dave Garrett]

An extreme example of the aforementioned user confusion:  bug 453007  ;)

Comment 7

[Dave Garrett]

This still happens every so often, more than before. It's fairly reasonable that people expect to get important details about the report when they click the button labeled "details", so the simplest thing to do here is to just label the thing better. I suggest we change its name to "Report metadata" or similar, as that's what this really is. Along with adding a simple blurb about checking "about:crashes" (comment 2) this should be more than enough to alleviate the common confusion here.

Comment 8

[Dave Garrett]

The frequency of users getting confused with this is too high. It adds an unnecessary extra hurdle to getting a user to successfully file a useful crash bug or support request.

This is an example of what's in this dialog:
----------------------------------------------------------------------
Add-ons: {972ce4c6-7e08-4474-a285-3208198ce6fd}:3.5.2
BuildID: 20090729211829
CrashTime: 1252102938
InstallTime: 1249345008
ProductName: Firefox
SecondsSinceLastCrash: 1676735
StartupTime: 1252102897
Theme: classic/1.0
Throttleable: 1
Vendor: Mozilla
Version: 3.5.2

This report also contains technical information about the state of the application when it crashed.
----------------------------------------------------------------------

There is essentially no information in this that is useful to the user. It has a non-human-readable add-on dump, 3 non-human-readable times, basic stuff the user already knew like the application name/vendor/version/build, the theme name, throttle status, and the time since last crash. This output is not productive. If the user wants to investigate here they're going to want to use the full crash report; if they don't they're not going to care about any of this.

My opinion has changed from "lets please fix this" to "lets please kill this". Can we please just get rid of this button and dialog entirely? The dialog is labeled "Report Contents" and it's not, but users think it is. All it does is confuse users and it has no real redeeming value.

If we really want to keep the crash time, app name, version, build, etc. info, then I suggest we add a blurb of non-selectable text stating it simply to the top of the main dialog. Frankly, though, I don't think it's worth keeping. I think we should just go with Jesse's idea in comment 3 and stick "After submitting this raw crash report,
load about:crashes in Firefox to see the processed report" in the bottom of the main dialog and scrap the details button/dialog completely.

Comment 9

[Dave Garrett]

To clarify by what I mean by "too high" frequency: On Bugzilla for this past week alone, there were 7 different cases of users posting the "details" thinking it was the actual crash report. (8 if you count the one bug quoting the report comments with the "details" pasted into it for some reason)

Comment 10

[(not currently active) Ted Mielczarek]

I agree that this sucks, but I don't agree that it's useless UI. The whole point is to show the user what we're sending so they can trust that we're not submitting lots of details about their system. Letting them see it after the fact is not a solution. We can certainly make it non-selectable, and I'd take a patch for that. If we can come up with better text, that's fine too. I don't think I want to just get rid of this without a replacement.

Comment 11

[Jesse Ruderman]

Showing these details as if they represent the extent of the privacy risk is dishonest.  The URL is not shown.  The list of extensions isn't human-readable.  There isn't even a hint that the report includes a module list, which reveals what malware has infected your system and some of the games you play.  In some cases, four sequential byes from a random part of memory will be submitted as part of the stack trace.

What protects users' privacy is what we don't do.  We don't store the IP address or collect email addresses. We don't make the URL public.  We don't use the module list and time-since-last-crash to fingerprint users, even though we could.  We don't correlate crash data with much of anything.

Comment 12

[Dave Garrett]

(In reply to comment #10)
> I agree that this sucks, but I don't agree that it's useless UI. The whole
> point is to show the user what we're sending so they can trust that we're not
> submitting lots of details about their system.

I completely agree with your intention, but the current implementation has nothing to do with it. It doesn't show what's going to be submitted but rather a tiny bit of not that useful info about it. The vast majority of the submission isn't even possible to be human-readable, seeing as the debug symbols aren't on the client side.

I'm not entirely against redoing this thing in a way that might theoretically be useful. It'd have to be human-readable, list all the possibly readable info to be submitted, labeled more accurately, and not imply that it's in any way the "report content" as it does now. However, I just don't think it's worth the time and effort to spiff up this thing. Even if this was all fixed so that it wasn't easily misunderstood there wouldn't be much use for it.

Fundamentally, it's not possible to do what we'd really want here which is to allow for full informed consent. If that's the route we want we could try to add an explanation of what's actually in the report, but the dump itself ain't going to be read by the submitter. This is currently a partial attempt at showing something rather than nothing, but it's not really better than nothing as the info is quite useless and frequently confusing to users.

In its current form this thing is useless at best and a liability at worst. This past week gave us over one confused bug reporter per day (probably more on SUMO), so this is a measurable and worsening problem. Getting rid of it doesn't really take anything important away and would be a quick and simple solution.

Comment 13

[Samuel Sidler (old account; do not CC)]

(In reply to comment #11)
> Showing these details as if they represent the extent of the privacy risk is
> dishonest.  The URL is not shown.  The list of extensions isn't human-readable.
>  There isn't even a hint that the report includes a module list, which reveals
> what malware has infected your system and some of the games you play.  In some
> cases, four sequential byes from a random part of memory will be submitted as
> part of the stack trace.

The URL *is* shown if the box is checked. The module list is included in the "This report also contains technical information about the state of the application when it crashed." line, which could be improved, certainly.

This isn't useless UI, it's UI that needs to be improved.

Comment 14

[(not currently active) Ted Mielczarek]

We could, in fact, link the Breakpad processor library with the crash reporter client, and have it load the minidump and display some information from it, such as the OS/CPU info and the module list (and even a stack trace without symbols). I'm not sure it would mean any more to people, but it would be a more honest display of what we're sending, for sure. I'm not sure if that's worth the effort for what it gives, though.

I've already said I'd take a band-aid patch, though. Making the text non-selectable and/or changing the strings to mention crash-stats would both seem to be improvements here. Let's look for a solution instead of wholesale removal.

Comment 15

[Jesse Ruderman]

I don't think making the text non-selectable would help much.  See comment 2.

Comment 16

[Jesse Ruderman]

Sam, you misunderstand what the useless-UI keyword means.

Comment 17

[Jan]

I was pointed to this Bug after complaining about my disappointment with the Crash-Reporter.

The "Details"-Button is very well meant, but misleading, because the user basically sees only the portion of the report which could be be considered PII (Personally identifiable information). IMO the button should be renamed to better reflect this.

Additionally some text like "For the entire report use about:crashes after Firefox restarts" could be added to "This report also contains technical information about the state of the application when it crashed".
Especially the current German translation is IMO massively misleading, 'cause it misses the equivalent of "also" and leaves the user with the impression that the few lines he sees are the entire report. (I've IT-background and I was wondering a loong time how smbdy could debug a crash with this little bit of information. After seeing about:config for the first time my reaction was something like "Gosh, why do they hide this and give me a false impression with this 'Details'-sh*?!") Generally I very much second the Teds thought (Comment 14) to show a minidump to the user because it would be more ... uhm ... honest towards the user.