852308 - FHR error reporting policy for non-pre-release channels

Status: RESOLVED FIXED

(Firefox Health Report Graveyard :: Client: Desktop, defect, P1)

Description

[Gregory Szorc [:gps]]

+++ This bug was initially created as a clone of Bug #848861 +++

In bug 845431 and bug 848861 we added error reporting to FHR's payload. It allows us to see what errors clients are experiencing and is shedding valuable light on real bugs and helping us make FHR as robust as possible.

There have been questions in these bugs on whether this is acceptable to roll out to the beta and release channels. My developer perspective is that I'd like error reporting everywhere. I would especially like error reporting on the release channel because that is where most of our users are and is where some of the more esoteric issues occur.

Anyway, this bug is mainly to get Privacy and Legal sign-off on errors. The choices are essentially:

a) Errors everywhere
b) Errors on certain channels
c) Error nowhere

Here are some example errors:

2013-03-18		{(Provider error: org.mozilla.appInfo: Failed to initialize: Error(s) encountered during statement execution.)}
2013-03-18		{(Provider error: org.mozilla.appInfo: Failed to initialize: Error(s) encountered during statement execution.)}
2013-03-18		{(Provider error: org.mozilla.appInfo: Failed to initialize: Error(s) encountered during statement execution.,Error registering pull-only provider: Error(s) encountered during statement execution.)}
2013-03-18		{(Provider error: org.mozilla.appInfo: Failed to initialize: Error(s) encountered during statement execution.,Error registering pull-only provider: Error(s) encountered during statement execution.)}
2013-03-18		{(Provider error: org.mozilla.appInfo: Failed to initialize: Error(s) encountered during statement execution.,Error registering pull-only provider: Error(s) encountered during statement execution.)}
2013-03-18		{(Provider error: org.mozilla.appInfo: Failed to initialize: Error(s) encountered during statement execution.,Error registering pull-only provider: Error(s) encountered during statement execution.,Provider error: org.mozilla.addons: Failed to collect: Error(s) encountered during statement execution.,Provider error: org.mozilla.appSessions: Failed to collect: Error(s) encountered during statement execution.)}
2013-03-18		{(Provider error: org.mozilla.appInfo: Failed to initialize: Error(s) encountered during statement execution.,Provider error: org.mozilla.appInfo: Failed to initialize: Error(s) encountered during statement execution.)}
2013-03-19	20130310030906	{(Provider error: org.mozilla.addons: Failed to collect: Component returned failure code: 0x80520012 (NS_ERROR_FILE_NOT_FOUND) [nsIFile.lastModifiedTime],Provider error: org.mozilla.addons: Failed to collect: Component returned failure code: 0x80520012 (NS_ERROR_FILE_NOT_FOUND) [nsIFile.lastModifiedTime])}
2013-03-19	20130311030946	{(Provider error: org.mozilla.addons: Failed to collect: Error(s) encountered during statement execution.,Provider error: org.mozilla.appInfo: Failed to collect: Error(s) encountered during statement execution.,Provider error: org.mozilla.crashes: Failed to collect: Error(s) encountered during statement execution.,Provider error: org.mozilla.profile: Failed to collect: Error(s) encountered during statement execution.,Provider error: org.mozilla.appSessions: Failed to collect: Error(s) encountered during statement execution.,Provider error: org.mozilla.places: Failed to collect: Error(s) encountered during statement execution.,Provider error: org.mozilla.appInfo: Failed to collect: Error(s) encountered during statement execution.,Provider error: org.mozilla.crashes: Failed to collect: Error(s) encountered during statement execution.,Provider error: org.mozilla.appSessions: Failed to collect: Error(s) encountered during statement execution.)}
2013-03-19	20130314030914	{(Provider error: org.mozilla.places: Failed to collect: this.bookmarks is undefined)}

Note that the error format will change once bug 848861 is deployed in the wild.

Comment 1

[Gregory Szorc [:gps]]

To capture it explicitly in this bug, while the contents of some error messages are dynamically generated and we may not know what the content is until we see them, we hypothesize (and have verified from the data we've seen so far) that there is low risk for PII to be included in the contents of the error. The reasoning is that any data likely to be in the error content is data that is already within FHR's sphere of influence. And, all data in FHR's sphere of influence is likely subject to upload in the payload. So, all the data is stuff we would be collecting anyway.

The lone exception to this that we've identified is file paths. We could potentially leak full system paths containing things like the username of the current user. We've taken steps to scrub these paths from error content.

We are committed to reacting quickly if and when PII leaks into the error messages.

To the best of my knowledge, no PII is currently leaking in the errors currently being reported by this feature. I could send someone a full output of the errors for verification if you so desire.

Comment 2

[Gregory Szorc [:gps]]

This was addressed in bug 855710.

Comment 3

[u279076]

Is there a way to simulate or force errors so I can verify this is fixed?

Comment 4

[Gregory Szorc [:gps]]

(In reply to Anthony Hughes, Mozilla QA (:ashughes) from comment #3)
> Is there a way to simulate or force errors so I can verify this is fixed?

Not without monkeypatch or corrupting your .sqlite file.

IMO this bug doesn't need to be explicitly verified. We have other bugs tracking implementation of error reporting. And, we know from payloads that clients are sending errors.

Comment 5

[u279076]

(In reply to Gregory Szorc [:gps] from comment #4)
> (In reply to Anthony Hughes, Mozilla QA (:ashughes) from comment #3)
> > Is there a way to simulate or force errors so I can verify this is fixed?
> 
> Not without monkeypatch or corrupting your .sqlite file.
> 
> IMO this bug doesn't need to be explicitly verified. We have other bugs
> tracking implementation of error reporting. And, we know from payloads that
> clients are sending errors.

Thanks Greg, that's good enough for me.