[Search experiment] Notify user about the end of the experiment

VERIFIED FIXED

Status

()

Firefox
Search
VERIFIED FIXED
4 years ago
a year ago

People

(Reporter: Felipe, Assigned: Felipe)

Tracking

Trunk
Points:
3
Dependency tree / graph
Bug Flags:
firefox-backlog +

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Assignee)

Description

4 years ago
From bug 1029275, we should notify the user about the end of the experiment.

Q: Is that only for the users from cells 3 and 5 who were notified at the beginning of the experiment? Or does that apply to cells 2 and 4 too?


On a technical side, we will have to tell the experiment manifest to run the experiment a bit longer than what's desired, and let the add-on code decide by itself when to finish it, display the notification (and decide the best moment to do so), and uninstall itself. Also avoid doing that in case of a uninstall/reinstall due to an upgrade. It's a bit more complex for these reasons, but should be doable.

Updated

4 years ago
Flags: firefox-backlog+
Do we need a UX design for this end-of-experiment user notification? Or do we already have it?
(Assignee)

Updated

4 years ago
Flags: needinfo?(philipp)
There's a mockup in bug 1029189 about this (https://bugzilla.mozilla.org/attachment.cgi?id=8446642).
This is really just the same thing as at the beginning of the experiment but with a Google logo. I'm very open for other suggestions here, but that should be a pretty safe route to go.
Flags: needinfo?(philipp)
(Assignee)

Comment 3

4 years ago
Same text, even? Just displaying the then-restored Google provider?
(Assignee)

Comment 4

4 years ago
For the exact behavior, here's what I propose:

3 days before the end date of the experiment, the experiment:
 - restores the default search provider
 - resets the counter of number of times the notice has been displayed, allowing it to be shown 5 more times

This means that there will be a 3-day window buffer where users have a chance to be notified about the reset. If a user doesn't open Firefox during that window, the add-on will be uninstalled and the provider reset, but it won't have a chance to notify about the change.

I need to double check this, it might still be possible to do something on uninstallation. But if it isn't, does this sound reasonable?
Sounds good to me!

I went with the same text because it is my understanding that we don't want to tell people specifically that they've been part of an experiment. I might be wrong about that though. Chad?
Flags: needinfo?(cweiner)
Having just read about the recent Facebook experiment at http://www.theatlantic.com/technology/archive/2014/06/everything-we-know-about-facebooks-secret-mood-manipulation-experiment/373648/, it seems like disclosing to people that they've been part of an experiment might be the ethical thing to do, particularly if we don't notify them at the start.  (Notifying the control group is probably more optional. ;)
(Assignee)

Updated

4 years ago
Assignee: nobody → felipc
Status: NEW → ASSIGNED
Added to Iteration 33.2
Felipe, can you provide a point value and if the bug should be marked as [qa+] or [qa-] for QA verification.
Iteration: --- → 33.2
QA Whiteboard: [qa?]
QA Whiteboard: [qa?] → [qa+]
Created attachment 8448035 [details]
Mockup for switching back to the old provider

Here's a mockup with some updated copy that tells people about the experiment. After seeing it, I absolutely agree that telling people about the study is the right thing to do.

Note that Google should only be restored if the user hasn't made any changes to the default search provider during the experiment.
(Assignee)

Comment 10

4 years ago
Work done at https://hg.mozilla.org/users/felipc_gmail.com/search-experiment and to be reviewed through
bug 1029818
Status: ASSIGNED → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
Hi Felipe, can you provide a point value.
Flags: needinfo?(felipc)
Another question Felipe, should this bug be marked as [qa-] if the testing will be conducted through bug 1029818?
(Assignee)

Comment 13

4 years ago
Setting p=3. Most of the testing will be done in bug 1029818, but since this is a tricky case, I think it's worth for us to leave [qa+] in this bug to verify the behavior described in comment 4.
Points: --- → 3
Flags: needinfo?(felipc)
(Assignee)

Comment 14

4 years ago
The QA for everything related to the search experiment was done in bug 1029818 so I'll change what I said in comment 13 and mark this one as qa-
QA Whiteboard: [qa+] → [qa-]
Marking this as verified as it was already tested in:
* bug #1029818 comment #34
* bug #1029818 comment #35

Used the following build:
- http://ftp.mozilla.org/pub/mozilla.org/firefox/releases/31.0b6/win32/en-US/ [Build ID: 20140630185627]
Status: RESOLVED → VERIFIED
Dropping NI from long-closed bug.
Flags: needinfo?(cweiner)
You need to log in before you can comment on or make changes to this bug.