Measure usage of Sync in health report

VERIFIED FIXED in Firefox 29

Status

Firefox Health Report
Client: Desktop
P1
normal
VERIFIED FIXED
4 years ago
4 years ago

People

(Reporter: Benjamin Smedberg, Assigned: gps)

Tracking

unspecified
Firefox 30
Points:
---

Firefox Tracking Flags

(firefox29+ fixed, firefox30 fixed)

Details

MozReview Requests

Submitter Diff Changes Open Issues Last Updated
Loading...
Error loading review requests:

Attachments

(1 attachment)

(Reporter)

Description

4 years ago
Because Firefox Account Sync is a central product strategy in 2014, we should actually measure whether users are finding it useful and whether it's helping promote Firefox-Android adoption and desktop retention.

The sync service itself will have pretty good numbers about how many accounts there are, but FHR should itself measure some things:

* P1: the basic sync-enabled status. This will let us determine whether users with lots of bookmarks or history are more likely to use sync, etc. We can also ofter FHR tips to user with lots of bookmarks to promote sync as backup and multi-device service
* P1: The number of devices attached to a sync account. I'd also like to know what the type of the other connected accounts are (other desktops/android/b2g), but I'm not sure we have that information in the payload (NI?rnewman). This data will correlate against the event tracking data, and we can also provide a view to the user of when the last time each device synced, if we have that data on the client.
* P2: Basic event tracking: failure to sync, wipe events. Frequent failure to sync or frequent wiping should show tips linking the user to SUMO at least.
* P3: Performance data? I don't have a good idea if there's anything in particular we want to measure here.

We should do something very similar or identical for Android. gps can you file that bug if it should be separate?
Flags: needinfo?(rnewman)
(Reporter)

Updated

4 years ago
Priority: -- → P1
Worth asking: are those items all "council-approved" to be included in payloads, or is this your personal wishlist?


Each device that's successfully syncing can know how many devices are connected to a Sync account.

We know if they're "mobile" or not, where "mobile" means "is not running the desktop XUL UI", pretty much (Metro was "mobile" for a while because of its genes.)

There is no single bug on file to record more accurate device types (FxOS versus desktop, say), but see Bug 745412, Bug 742478 Comment 9, and Bug 695134.

We know the name of the device -- "Richard's Nightly on HTC One". Broadly speaking this is an opaque string. We shouldn't submit this via FHR.

Soon we'll know which Firefox version you're running (e.g., "29.0a1") -- see Bug 956936 and friends. This is less identifying than the device name, but it's still fingerprintable.


We can't feasibly de-dupe across FHR accounts, so all eight of your devices will report that you have eight devices syncing.


Sketched format, daily:

sync.enabled: true
sync.hosted: true       // Self-hosting.
sync.devices: {
  mobile: 2,
  other: 1
}
sync.events: {
  succ: 5,
  fail: 1,
  toolong: 1
}

How does that sound?

N.B., dropping the "FxA" part from the bug subject, because there's nothing about the data you've suggested that's at all related to FxA. If there is FxA data that you'd like to record, please expand on that part!
Flags: needinfo?(rnewman)
Summary: Measure usage of FxA-sync in health report → Measure usage of Sync in health report
(Reporter)

Comment 2

4 years ago
(In reply to Richard Newman [:rnewman] from comment #1)
> Worth asking: are those items all "council-approved" to be included in
> payloads, or is this your personal wishlist?

I'm driving this, so let's figure out what we should measure and then worry about approval.

> We know the name of the device -- "Richard's Nightly on HTC One". Broadly
> speaking this is an opaque string. We shouldn't submit this via FHR.

Right, definitely shouldn't do that.

> How does that sound?

Fine.

> N.B., dropping the "FxA" part from the bug subject, because there's nothing
> about the data you've suggested that's at all related to FxA. If there is
> FxA data that you'd like to record, please expand on that part!

As I understand the current proposal, we are forking users from old-sync to FxA-sync in Firefox 29. We really want to measure the uptake of new-sync while still tracking old-sync. So I think we should at least include information about which sync "version" we're using.
(In reply to Benjamin Smedberg  [:bsmedberg] from comment #2)
 
> I'm driving this, so let's figure out what we should measure and then worry
> about approval.

Sounds good.

> So I think we should at least include
> information about which sync "version" we're using.

  sync.protocol: "1.5"

probably does the trick. 1.1 is current Sync, 1.5 is Sync with FxA auth, and the protocol version will change for any future reworking of Sync itself.
(Reporter)

Updated

4 years ago
Priority: P1 → P2
Existing Android-side bug for this: Bug 891600.
See Also: → bug 891600
I was bogged down trying to get bug 875562 into 29. This bug really needs to land in 29 if not 28 so we have useful Sync data to coincide with the Sync relaunch. Setting release tracking flag.
tracking-firefox29: --- → ?
tracking-fennec: --- → ?
tracking-fennec: ? → ---
Created attachment 8370978 [details]
Add Sync to FHR

rnewman for code. bsmedberg for FHR probe sign-off.
Attachment #8370978 - Flags: review?(rnewman)
Attachment #8370978 - Flags: feedback?(benjamin)
Comment on attachment 8370978 [details]
Add Sync to FHR

Some questions left on the RB request.
Attachment #8370978 - Flags: review?(rnewman) → feedback+
(Reporter)

Updated

4 years ago
Attachment #8370978 - Flags: feedback?(benjamin)
tracking-firefox29: ? → +
We will go ahead and track this since we understand the importance of tracking the landing of this feature.
Comment on attachment 8370978 [details]
Add Sync to FHR

See new patch on RB.
Attachment #8370978 - Flags: review?(rnewman)
I've seen chatter about possibly new requests for FHR's Sync probes. Should we ship what we have or refine the requirements?
tracking-firefox29: + → ?
Flags: needinfo?(benjamin)
Context: I imagine pretty soon we'll have FxA users who don't use Sync, Mozilla FxA users who use self-hosted Sync, and potentially self-hosted FxA users who self-host Sync.
(Reporter)

Comment 12

4 years ago
FxA is separate, let's not confuse things. This is specifically about sync. Right now I don't think we care about the edge cases of self-hosted FxA/sync.
Flags: needinfo?(benjamin)
Comment on attachment 8370978 [details]
Add Sync to FHR

I believe mhammond ensured that the FxA enablement pref is updated correctly. lgtm.
Attachment #8370978 - Flags: review?(rnewman)
Attachment #8370978 - Flags: review+
Attachment #8370978 - Flags: feedback+
https://hg.mozilla.org/integration/fx-team/rev/dc26e8d484f8
Status: NEW → ASSIGNED
Target Milestone: --- → Firefox 30
https://hg.mozilla.org/mozilla-central/rev/dc26e8d484f8
Status: ASSIGNED → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
tracking-firefox29: ? → +

Updated

4 years ago
Status: RESOLVED → VERIFIED
Richard, As said by Gregory in comment 5, it would be nice to have this in 29. Could you request an uplift?

Thanks
Flags: needinfo?(rnewman)
Comment on attachment 8370978 [details]
Add Sync to FHR

[Approval Request Comment]
Bug caused by (feature/regressing bug #): FxAccounts Sync measuring
User impact if declined: We won't know how FxAccounts/Sync is working for our users
Testing completed (on m-c, etc.): It's been on Nightly for a little while with no apparent issues.
Risk to taking this patch (and alternatives if risky): New FHR probes are low risk. Highest risk is bad data is going to FHR. But this is FHR's problem, not a release drivers problem.
String or IDL/UUID changes made by this patch: None
Attachment #8370978 - Flags: approval-mozilla-aurora?
Flags: needinfo?(rnewman)
Attachment #8370978 - Flags: approval-mozilla-aurora? → approval-mozilla-aurora+
https://hg.mozilla.org/releases/mozilla-aurora/rev/7ae9d8bc8ba6

Contained a very minor change to 2 Assert.ok() lines to work around a missing patch from Assert.jsm that was causing test_healthreport.js to fail.
status-firefox29: --- → fixed
status-firefox30: --- → fixed
Target Milestone: Firefox 30 → ---
(Reporter)

Updated

4 years ago
status-firefox29: fixed → ---
tracking-firefox29: + → ---
Priority: P2 → P1
Target Milestone: --- → Firefox 30
(Reporter)

Comment 19

4 years ago
wtf flags disappeared
status-firefox29: --- → fixed
tracking-firefox29: --- → +
You need to log in before you can comment on or make changes to this bug.