Closed Bug 757184 Opened 8 years ago Closed 5 years ago

MU Channel Switch from release channel to beta channel

Categories

(Firefox :: General, defect, P5)

defect

Tracking

()

RESOLVED INVALID

People

(Reporter: akeybl, Assigned: lsblakk)

References

(Blocks 1 open bug)

Details

Attachments

(1 file)

Type of update: Billboard MU
From target: 14.0 release channel users
To target: 15.0beta
Target window: 8/6-8/10 (first 2 weeks that the beta is available, 3 weeks prior to release of FF15 for flexibility)

Other:
* We do want to have the "No thanks" button available on the MU
* There will be no overlap between 15.0 on the wire for release channel users and 15.0 on the beta channel 

To be verified:
* We need to make sure that 14.0 users who say 'No Thanks' to 15.0beta still get 15.0 when it's released
* The channel that is reported in about:support and the about dialog once updated to the beta channel
* That 15.0beta users will get 16.0beta properly
* Metrics data and blocklist pings are going to the right channel

TBD:
* What locales/platforms we want to target
* The billboard URL
Nick, I know this was still blocked on some signed channel MAR stuff - would you mind blocking on that?

Also,

(In reply to Alex Keybl [:akeybl] from comment #0)
> Target window: 8/6-8/10 (first 2 weeks that the beta is available, 3 weeks
> prior to release of FF15 for flexibility)

should have read:

Target window: 8/6-8/10 (third week into updating our FF13 users to FF14, 3 weeks prior to release of FF15 for flexibility)
Assignee: nobody → nrthomas
One other thing of note - we should target the initial testing (under to be verified) by mid-June so that we know whether to move forward with having the billboard created.
From email conversation on this UX:

bbondy: "So if you want to go from Release -> Beta then the MAR that does that switch needs to be a MAR marked as a release MAR.
If it is, then I don't see a problem with it, but please test it before widely deploying."

How soon could we get a test update snippet to beta, made with MAR_CHANNEL_ID=firefox-mozilla-release, and signed?
Not to scope creep too much and please feel free to bash on this idea but...more Telemetry data on Beta could be useful in measuring quality attributes on this channel. As long we we're advertising to these users to help test Firefox releases, can we advertise to these users to also submit feedback via Telemetry data?
(In reply to Lawrence Mandel [:lmandel] from comment #4)
> Not to scope creep too much and please feel free to bash on this idea
> but...more Telemetry data on Beta could be useful in measuring quality
> attributes on this channel. As long we we're advertising to these users to
> help test Firefox releases, can we advertise to these users to also submit
> feedback via Telemetry data?

Are you suggesting that we keep that as part of the billboard? Or just have a Telemetry advertising effort to beta users separately? I'm not sure what options we have (outside of an add-on hotfix) for presenting Telemetry to users.
per channel mtg, there are two reasons for doing this:

1) increase size of beta population
2) increase the % of beta users that are non-en-US. The exact set of *which* locales are of interest is still TBD.
(In reply to Lawrence Mandel [:lmandel] from comment #4)
You could include messaging about Telemetry in the whatsnew page after the update, but that's about all we can do in this bug.
(In reply to Alex Keybl [:akeybl] from comment #5)
> Are you suggesting that we keep that as part of the billboard? Or just have
> a Telemetry advertising effort to beta users separately? I'm not sure what
> options we have (outside of an add-on hotfix) for presenting Telemetry to
> users.

I was suggesting that we include Telemetry in the message that we send out about moving to beta.

(In reply to Nick Thomas [:nthomas] from comment #7)
> (In reply to Lawrence Mandel [:lmandel] from comment #4)
> You could include messaging about Telemetry in the whatsnew page after the
> update, but that's about all we can do in this bug.

I don't know all the details of the billboard but thought that this would be an in product prompt. If so, can we prompt to enable Telemetry before updating to a newer beta build?

To be clear, I do not want to show a what's new page solely to advertise Telemetry opt-in.
AIUI the billboard is treated as regular webpage, and so has no chrome privileges to affect preferences. As ever, you can ask rstrong for his thoughts as the domain expert.

If it is to be done in the billboard then could we please move that discussion to the bug that creates the billboard, which is normally separate from the RelEng one (we just want to know the url to use). Could we file that, or if already filed set it as a dependency here ?
Component: Release Engineering → Release Engineering: Releases
Priority: -- → P2
QA Contact: release → bhearsum
Almost have a test ready. Going for 13.0 -> 13.0b7, so that we can then do the update to 14.0b6; using win32 en-US. The repacking of the 13.0b7 complete mar is working, downloading and applying that is successful, and updating onwards to 14.0b6 also works. 

I just need to tweak the update.xml so that it prompts with the billboard and waits, rather than downloading like an automatic/background update. Should have the test up first thing next week.
have we thought much about how users will react to this and if there will be users that annoyed by the offer, or get confused and get on a channel that doesn't fit with their expectations?

is there a way to get UX testing to make sure these things don't happen or we minimize these kinds of negatives?

I'm all for increasing the beta pool, but not at the cost of loosing users that really should be on the release channel and not be bothered by beta.
Billboard issue is now resolved (showPrompt should be true rather than True, and watch out for caching if using http:// for xml). Attached are my notes for generating the mar file and xml.
Steps to try it out
1, Use Firefox 13.0 win32 en-US (https://ftp.mozilla.org/pub/mozilla.org/firefox/releases/13.0/win32/en-US/Firefox%20Setup%2013.0.exe)
2, Load about:config
3, Right click and create a new string preference called app.update.url.override, set the value to 
 http://people.mozilla.com/~nthomas/bug757184/update.xml

4, Help > About Mozilla Firefox, click on Apply Update 

A dialog will open and load the billboard will be shown (I've just used an old billboard we had already). You'll be updated to 13.0b7 (check about:buildconfig and follow the hg.m.o link to see the tag). If you then reset app.update.url.override then you'll get updated to 14.0b6 (or the latest beta when you're testing).

Note that most users would go straight to the billboard because the update check would be done in the background. To simulate this modify these prefs too:
   app.update.interval  --> 60
   app.update.idletime  --> 10
and restart Firefox. You should get the billboard after a minute or two.
Confirmed...

1. Set app.update.url.override
> Firefox 13.0 updates to 13.0b7 with billboard
2. Restart and reset app.update.url.override
> Firefox 13.0b7 updates to 14.0b6
Thanks for confirming this is a feasible option guys! That's great news.

Now for the bad news - dslater called a meeting today based upon concerns with Grace's initial proposal, and has asked that we push off the target window till at least the following cycle.

We'll keep you all updated, but we're in a holding pattern for now.
Assignee: nrthomas → nobody
Priority: P2 → P5
Component: Release Engineering: Releases → General
Product: mozilla.org → Firefox
QA Contact: bhearsum → general
Version: other → unspecified
This has been moved out and will not be looking to happen on the next release date, a month from now. I'm untracking it but assigning it to myself so it doesn't have to go through any Firefox triage (hopefully).
Assignee: nobody → lsblakk
Summary: MU Channel Switch from 14.0 to 15.0beta → MU Channel Switch from release channel to beta channel
no one has replied to my questions in comment 11.  were these also discussed in the meeting talked about in comment 15.
This bug is stale and doesn't reflect current practices or desires so I'm closing it and we'll file new bugs when we have a strategy in place for growing/altering our beta population.
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → INVALID
You need to log in before you can comment on or make changes to this bug.