907998 - Firefox OS crash reports should allow for sending email addresses

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: Gaia::System, defect)

Description

[Benjamin Smedberg]

The Firefox OS crash report UI should allow for comments and email addresses. Although I'm also thinking about a more comprehensive "report a problem" app/subapp. I'll take this for now.

Comment 1

[Robert Kaiser]

One problem with that is that we are sending crash reports automatically as per decision from UX (lco at that point) that prompting on every crash is probably not a good idea.

Comment 2

[Robert Kaiser]

We now have bug 1013979 landed on Socorro so people can see their crash reports listed when they submit their email address. This means that if we find some way to submit an email address with FxOS crashes, we can query them on crash-stats and don't need to go to adb tricks to see them listed.

How could we get there?

Comment 3

[Robert Kaiser]

I'd like to focus this on email for now, comments are much harder to solve.

Could we have a "send email with crash reports" pref in settings? Maybe just send the FxA login email there?

Fabrice, who could decide on that or who do we need input from?

Comment 4

[[:fabrice] Fabrice Desré]

(In reply to Robert Kaiser (:[e-mail]) from comment #3)
> I'd like to focus this on email for now, comments are much harder to solve.
> 
> Could we have a "send email with crash reports" pref in settings? Maybe just
> send the FxA login email there?
> 
> Fabrice, who could decide on that or who do we need input from?

Would that be good enough to hook that up to the "I want to improve firefox OS" setting? You want the privacy team to agree but we should try to not have too many checkboxes.

Comment 5

[Robert Kaiser]

(In reply to Fabrice Desré [:fabrice] from comment #4)
> Would that be good enough to hook that up to the "I want to improve firefox
> OS" setting? You want the privacy team to agree but we should try to not
> have too many checkboxes.

Whatever we can do that is still available in settings after FTE (so people can decide to turn it off/on or change the email later) is fine with me.
I just like to get this moving at least for 2.1 as together with the feature of crash-stats to show all your crash reports based on the email, it would make QA flows *much* easier.

Who needs to make the decisions here? Who do we need from the privacy team to agree?

Comment 6

[[:fabrice] Fabrice Desré]

Jishnu, we need your input there!

Comment 7

[Robert Kaiser]

FWIW, I apparently also filed bug 948601 on the email piece itself. That might end up being a dupe.

Comment 8

[Mika]

Hi Kairo, Jishnu asked me to respond here since I've been working on improving our user facing data screens.  I have a bunch of comments and questions below.  Please schedule a call if that's easier to go through this list.  My calendar is updated.

Thanks,
Mika

Background:
- do you have metrics for FxOS on how many people elected to send crash reports automatically always, never, or ask each time?
- do you have metrics for Fx on how many people provide their email at the time of sending the crash report? Or choose not to send a report, even if they have selected "always send" in settings?
- Why is the FxoS UX different from Fx? My understanding is that on Fx, no mater what the user puts in Tools > Options > Data Choices, the user is always prompted with this screen to (1) confirm if he/she wants to send the report and (2) provide the email address: http://screencast.com/t/ktF7lktgBr  
- Do you currently get a device identifier or IP in connection with the crash reports sent from FxOS?
- Do crash reports ever occur in connection with the services crashing (sync, location, marketplace), or just a client side Fx or FxOS issue?

Emails:
- Getting a Firefox account or signing up for an email is optional, so not 100% of FxOS users will send their email address to us.  So you still need another option to give people the opportunity to provide an email.  
- Our position is that if a person provides their email to us for an account or email, we want to be transparent about everything we do with that email.  To date, we only use the email to provide services in connection with the account, or to send newsletters.  Your suggestion is to also use the email in connection with sending Mozilla crash reports in connection with the account holder's device.  Before we figure out if that's okay, can you give me some more info:
(1) why is the email address needed?
(2) where would the email address go [i.e. what server? what country? who has access?]
(3) how long would the email address be retained? 
(4) would the email in connection with FxOS be treated any differently than it is currently treated with Fx?
(5) If we did this, would we also extend this account email detection to the browser?

Comment 9

[Robert Kaiser]

(In reply to Mika from comment #8)
> - do you have metrics for FxOS on how many people elected to send crash
> reports automatically always, never, or ask each time?

Not that I know of, AFAIK Firefox OS does not have any metrics at all due to partner concerns from what I understand.

> - do you have metrics for Fx on how many people provide their email at the
> time of sending the crash report? Or choose not to send a report, even if
> they have selected "always send" in settings?

We don't have any metrics on how many people provide an email as we do not have any measure for "people" there. I could probably query our crash DB for what portion of crash reports do have email addresses, if we really want it.
For the second part, there is no "always send" in settings. We are only getting to the point where FHR will record the actual number of crashes and the amount of reports sent, and only once we have that in place, we'll be able to even find out the amount of people sending a crash report.

> - Why is the FxoS UX different from Fx? My understanding is that on Fx, no
> mater what the user puts in Tools > Options > Data Choices, the user is
> always prompted with this screen to (1) confirm if he/she wants to send the
> report and (2) provide the email address: http://screencast.com/t/ktF7lktgBr

Yes, you are right on the Firefox UX flow (though the prompt actually has sending the report as a default-on checkbox and the buttons are Close or Restart, so there's a bias to sending here as well). When we implemented crash reporting on FxOS, the UX team wanted something that does not modally block the UI on every OS or app crash, so they came up with the different flow that we have right now. We prompt after the first crash (in case it's a gecko/OS crash, after a restart of the UI) and have an "always send" option marked by default there, so that reporting is silent from there on. People can also opt to never send or get prompted every time if they really want to. As I said, that's what the Firefox OS UX team came up with and got singed off by everyone back then.
The bias to sending by default is intentional in both Firefox and OS cases, just that OS also goes to non-prompted by default after the first prompt.

> - Do you currently get a device identifier or IP in connection with the
> crash reports sent from FxOS?

No. Both in Firefox and OS we do not get any user-unique ID or such for privacy reasons.

> - Do crash reports ever occur in connection with the services crashing
> (sync, location, marketplace), or just a client side Fx or FxOS issue?

It always has to be a binary code issue, web code cannot crash by itself in a way that is sent as a report - but web code can of course trigger a crashing issue in binary code.

> - Getting a Firefox account or signing up for an email is optional, so not
> 100% of FxOS users will send their email address to us.  So you still need
> another option to give people the opportunity to provide an email. 

Yes, that's true. We do not want to force people to send an email. It should be optional in some form.
The most important target here is people helping to test FxOS (including our own and partner QA) and people wanting to help to track down their issues - we want to enable them to send an email address with the crash, mostly so they can find their processed reports by logging into the crash-stats server, possibly also so we can email them to get more info on how to reproduce crash issues we are seeing in the stats (this is something we do on Firefox as well).
We could potentially add another checkbox in some place so people can opt out (or in?) to sending the FxA address with crash reports - but then, we are also trying to not add too many options that people might not understand. I guess that call needs to be made between privacy and UX concerns and teams.

> - Our position is that if a person provides their email to us for an account
> or email, we want to be transparent about everything we do with that email. 
> To date, we only use the email to provide services in connection with the
> account, or to send newsletters.  Your suggestion is to also use the email
> in connection with sending Mozilla crash reports in connection with the
> account holder's device.  Before we figure out if that's okay, can you give
> me some more info:
> (1) why is the email address needed?

So the user can find their own reports when logging into the crash-stats server, and for us contacting them for help with reproducing crashes.
(Actually, the latter is the primary reason why we added optional emails to crash reports in the beginning - the former is the only way for themselves to find their own report without resorting to adb, though, as the about:crashes list that we have in Firefox does not work on OS.)

> (2) where would the email address go [i.e. what server? what country? who
> has access?]

It would go to the crash-stats server/cluster, same as with emails specified on crash reports on Firefox desktop/Android. The servers run on Mozilla hardware in the US, IIRC in the PHX datacenter at the moment. Only a small, specially vetted group of people has access to those emails as well as the URLs sent with the crash reports - even employees need vetting from the crash-stats team and have their managers agree that they need access.

> (3) how long would the email address be retained? 

According to the Socorro Data Expiration Policy on mana, max 12 months (if it's on a release channel), otherwise 6 months (we keep all raw crash data for that time).

> (4) would the email in connection with FxOS be treated any differently than
> it is currently treated with Fx?

No.

> (5) If we did this, would we also extend this account email detection to the
> browser?

We have not thought about that yet, but we could offer it to ease the workflow and not need to re-enter the same address. But this isn't in the plans yet.

Comment 10

[Robert Kaiser]

I'm rephrasing the summary to be about email only and putting a ni? on Mika.

Comment 13

[Mika]

Hi - from a legal/privacy pov, yes, it is fine to provide an option in settings for users to optionally send their email with crash reports.  The email collection has to be default off.  This means the email will be collected only if the user specifically adds an email address (or selects an option to automatically send the FxA email). 

Let me know if this makes sense for the request, or if you want to talk further.
Mika

Comment 14

[Robert Kaiser]

Fabrice, is the above enough to make something happen in code there? I would love to see something in 2.2, but at least 3.0 should have it.

Comment 15

[[:fabrice] Fabrice Desré]

(In reply to Robert Kaiser (:[e-mail]) from comment #14)
> Fabrice, is the above enough to make something happen in code there? I would
> love to see something in 2.2, but at least 3.0 should have it.

Looks like it's fine, yes. Now the hardest part, finding someone that can write the patch ;)

Comment 16

[BMO Automation]

Firefox OS is not being worked on