Open Bug 1258586 Opened 4 years ago Updated 4 years ago
Switchboard experiment: display tab queue notification sooner
imo, tab queues is a killer feature but most users probably never get to see it if they drop Firefox after only a few days of use. Quick switchboard prototype: should we make the contextual hint appear sooner for new users than it does now? Barbara, per Margaret's suggestion, can you help design this experiment to ensure we can reach conclusions with statistical significance?
4 years ago
4 years ago
Awesome - Thanks for filing this Mike! Coincidentally, this is something I was talking to Barbara about the other day as a part of some onboarding (different from First Run) experiments. We should use this bug to track that work :) Some variations we briefly talked about and could start with (based on our other onboarding work): Control - Show helper UI after the user opens 3 links in Firefox A) Show helper UI after the user opens 1 link in Firefox B) Show helper UI after the user opens Firefox the next time C) Notification 1 days after the user has installed -> Show helper UI (the radical idea, isn't as contextual)
Didn't we already see a very high adoption rate? I think at some point we even feared that we trap users and they might not know how to get out anymore? When developing variants think of the Android 6+ permission "issue" and that we are still using a sub-optimal prompt there: bug 1240709.
(In reply to Sebastian Kaspari (:sebastian) from comment #2) > Didn't we already see a very high adoption rate? I think at some point we > even feared that we trap users and they might not know how to get out > anymore? > When developing variants think of the Android 6+ permission "issue" and that > we are still using a sub-optimal prompt there: bug 1240709. I don't think the metric should be how many users have this feature enabled, but overall retention. Hypothesis: Tab queues are a valuable differentiator, and if people know about it, they will continue using Firefox. I'd argue we should have more extreme experiment groups. We should consider *never* showing the helper UI as the real control group. Do we retain those users just as much as users who learn about tab queues? If so, our hypothesis is definitely disproved.
For perhaps after our onboarding experiment: To address Sebastian's concern in comment 2, we could add a helper UI to get the user out of tab queue (e.g. they use it a few times, we display, "Do you like it? Want to keep it?"). Also, we could make the tab queue helper UI more descriptive to give users a better sense of what they're getting themselves into (e.g. include a video or pictures).
I will mark this as "yes" but not now (sorry), they are other experiments that need our attention, but we will get back to this once Mark has finalized his predictive analysis data. That will help us figuring out how and when to show which contextual hints sooner (or later). Maybe TQ could be shown in the second run. As most of you have said, TQ is a already a big retention piece so we know it's working well, but def. something to play around with. This work should fall under the contextual hints revamp.
You need to log in before you can comment on or make changes to this bug.