1253588 - Enable bookmark history menu experiment by default

Status: RESOLVED FIXED

(Firefox for Android Graveyard :: General, defect)

Description

[:Margaret Leibovic]

[Tracking Requested - why for this release]: Feature intended to release in 46.

Right now this experiment is limited to nightly, but we should enable it by default to let it ride to release.

This experiment is actually in 45, so we could enable it for release users if we want, but I think we should start by enabling it for beta users.

Comment 1

[Barbara Bermes [:barbara] - NI please!]

Can we enable it for both, beta and release? What's your rationale to start with beta first?

Comment 2

[:Margaret Leibovic]

(In reply to Barbara Bermes [:barbara] from comment #1)
> Can we enable it for both, beta and release? What's your rationale to start
> with beta first?

Just being cautious. So far only Nightly users have seen this change, so it would be good to at least spend a few days with it on beta before rolling out to release.

Comment 3

[Barbara Bermes [:barbara] - NI please!]

I agree that we should use switchboard to do staged roll-out and can also do channel roll-outs, but do you really think we need to be cautious for this specific addition? 

I could see being cautious for other changes could be more valuable.

So my question, why not just pushing it to all -- it's a great addition.

The only thing I would support is to actually to staged roll-out on a defined channel, and monitor cohorts usage of history and bookmark panel.

Comment 4

[:Margaret Leibovic]

(In reply to Barbara Bermes [:barbara] from comment #3)
> I agree that we should use switchboard to do staged roll-out and can also do
> channel roll-outs, but do you really think we need to be cautious for this
> specific addition? 
> 
> I could see being cautious for other changes could be more valuable.
> 
> So my question, why not just pushing it to all -- it's a great addition.
> 
> The only thing I would support is to actually to staged roll-out on a
> defined channel, and monitor cohorts usage of history and bookmark panel.

Yes, this is what I'm proposing. Let's roll it out to beta users next week, and then if people are using the menu item (I suppose this is where we need some threshold to determine what we think is good), then we can roll it out to release users as well.

I also view this as a good opportunity to go through the motions of a staged roll-out, even if we know we want to release to all users. Let's test this out with a low-controversy feature like menu items, before we try to do more impactful things like notifications.

Comment 5

[Barbara Bermes [:barbara] - NI please!]

ok, so this will be 46 Beta next week and after x days that can be turned on for release 45 as well?

Comment 6

[Liz Henry (:lizzard) (relman/hg->git project)]

Staged rollout sounds good to me, no need to rush it to release. Tracking for 46 since this is a new feature.
Sylvestre, adding tracking? for 45 in case this ends up moving to release.

Comment 7

[:Margaret Leibovic]

Attachment: PR to enable bookmarks/history menu items on aurora/beta

This is something new: I'm asking for approval to merge this Switchboard config change to production. Just hitting the merge button on this PR would automatically turn this feature on for all Aurora/Beta users.

Approval Request Comment
[Feature/regressing bug #]: bug 1231792

[User impact if declined]: Users won't see bookmark/history menu items.

[Describe test coverage new/current, TreeHerder]: No automated tests, baked on Nightly for a few weeks. We should have UI telemetry data to tell us whether people are using these menu items, but I have not seen updated dashboards with that data.

[Risks and why]: Low-risk, enables two menu items that are currently disabled on aurora/beta. The main risk comes from using Switchboard, but it's much better to be taking that risk now with this small feature than later with a bigger one.

[String/UUID change made/needed]: None.

Comment 8

[Liz Henry (:lizzard) (relman/hg->git project)]

Comment on attachment 8727425 [details] [review]
PR to enable bookmarks/history menu items on aurora/beta

Discussed this with Margaret, and it's OK by me. 
Let's test this process out. If something goes wrong, we can fix it with Switchboard without having to wait for a new beta build.

Comment 9

[:Margaret Leibovic]

I just merged this config change, the menu items should be live on aurora and beta.

Comment 10

[u421692]

Verified as fixed on Firefox 46 Beta 1. Tested on several devices

Comment 11

[Sylvestre Ledru [:Sylvestre]]

Ok, tracking in case we want to enable that in 45.

Comment 12

[:Margaret Leibovic]

Using release version tracking flags is confusing here, because the config change here only enabled this for aurora/beta, not release.

I will create a PR to enable this on release, but shipping this feature can happen independently of shipping Firefox 46.

Comment 13

[:Margaret Leibovic]

Attachment: PR to enable bookmark-history experiment on all channels

I'm requesting approval for this to make this process more transparent. For the future, I think we should create match cases with version numbers, instead of just app IDs, so that we can have things be automatically enabled when a new version moves to release.

We'll want to merge this tomorrow with the release of 46 if we want it to have the bookmark/history menu item. I don't think the precise timing really matters, so I don't want us to stress out about this too much.

Approval Request Comment
[Feature/regressing bug #]: bug 1231792 
[User impact if declined]: no bookmark/history menu item
[Describe test coverage new/current, TreeHerder]: baked on beta
[Risks and why]: low-risk, can disable feature again with another PR
[String/UUID change made/needed]: none

Comment 14

[Sebastian Kaspari (:sebastian; :pocmo)]

Comment on attachment 8745122 [details] [review]
PR to enable bookmark-history experiment on all channels

(Comments/Questions on GitHub)

Comment 15

[Teodora Vermesan (:TeoVermesan)]

Bookmarks and history are displayed in Menu. Verified as fixed on Firefox 47 Beta 1. Tested on One A2001 (Android 5.1.1), Nexus 9 (Android 6.0), ZTE Grand X (Android 4.0.4)

Comment 16

[Liz Henry (:lizzard) (relman/hg->git project)]

Comment on attachment 8745122 [details] [review]
PR to enable bookmark-history experiment on all channels

OK for 46 release.

Comment 17

[:Margaret Leibovic]

https://github.com/mozilla-services/switchboard-experiments/commit/8c920ddb10af3e3d907226720e2547182a641f86