1508743 - Shield Study to Increase FxA/Sync Discoverability on Desktop

Status: RESOLVED FIXED

(Data Science :: General, task)

Description

[Alex Davis [:adavis]]

Brief description of the request:
I've been asked to prepare a SHIELD study for the aDAU tiger team that consists in making FxA/Sync more discoverable in the browser.

I've started my PHD here:
https://docs.google.com/document/d/129HzVoufID4S2HRKMivXNW6BvlR6lYMa7Nyg-D-ZenA/edit


Is there a specific data scientist you would like or someone who has helped to triage this request:
I'd ideally like to work on this with Leif.

Comment 1

[Alex Davis [:adavis]]

Making moco confidential because this was not just done for growth. It is also addressing a private security bug. When we release it and inevitably some users will be unhappy, I don't want it to be said we're just doing it for growth when in fact, it's not the whole story.

Comment 2

[Alex Davis [:adavis]]

Per my last comment, the security bug is Bug 1300086

Comment 3

[[e-mail]]

I'd like to have a discussion about this. It seems less favorable to users to keep bugs like this moco-confidential for ambiguous security reasons, as these are ultimately experiments that are affecting user experiences. This does not follow the principles of openness and transparency.

Comment 4

[Alex Davis [:adavis]]

You’re absolutely right Rebecca. I think I took the lazy approach. Perhaps we can share details in this bug about what we are also trying validate with this study in regards to improving user security.

Ultimately this bug was opened for one reason but in the end we decided to work on it for multiple reasons. As many of those should be described here without hopefully revealing too much about the security issue that was left as confidential.

Comment 5

[Leif Oines [:loines]]

I should have added a few more details on this experiment so anyone curious would have a better understanding of all the reasons we are doing this study and related ones, namely Bug 1524390 and and Bug 1528321 .

First, one goal is to increase sync and Firefox account usage. We believe sync is an under-utilized feature of Firefox browsers (both desktop and mobile) and we have evidence that there is a significant population of multi-device Firefox users that nevertheless don't use sync. We believe this may be partly due to the fact that sync functionality is not visible to users in the top level of the Firefox interface. We also have survey data showing that awareness, knowledge and understanding of sync is low.

Second, we believe that making sync more visible will improve the experience of existing sync users. Because the top-level interface does not change dependent on whether a user is connected to sync, its likely that many users falsely believe that they are signed into sync when they actually aren't, and vice-versa.

Comment 6

[Leif Oines [:loines]]

Putting a summary of the results of this experiment here so we can close it out:

• We did see an overall increase in FxA authentications for the treatment branch.
• We did not see that the treatment branch reliably logged out more often, though the numbers here were very low for both branches.
• Users did not browse or search less in the treatment group
• Based on the post-experiment survey, users overall were happy with the new feature,