1295873 - Funnelcakes 87/88 49.0 + auto migration code for Onboarding Add-on v1.2 Experiment (Sept 28th launch)

Status: RESOLVED FIXED

(Release Engineering :: Release Requests, defect)

Description

[Chris More [:cmore]]

We are doing a minor incremental improvement to the v1.0 onboarding add-on funnelcake that was previously done in bug 1272776. The funnelcake configs should be pretty much identical to funnelcake 85/86 with just the funnelcake ID changed.

v1.2 funnelcake:

FC 87: vanilla Firefox Control
FC 88: onboarding funnelcake w/all-aboard XPI add-on (v1.2 XPI is still in progress)

Firefox version: 49.0
Lang: en-US
OS: Windows

We are also going to need this build to use the auto-import code created in bug 1271775.

Comment 1

[Chris More [:cmore]]

Three questions:

1) :Gijs: Can we do another auto migration build (like we did with the Larch brand in the previous funnelcake) with Firefox 49.0 release channel?

2) :nthomas: Any immediate blockers you can think of here?

3) :catlee: Anything we should do different this time around? 

Development of the v1.2 add-on started last week and we are trying to keep as much as possible the same with this next improvement. We need to validate some hypotheses that weren't extremely successful with the v1.0. Like any v1.0, we can do better, which is the next iteration.

Comment 2

[:Gijs (away/busy until Mar 10)]

(In reply to Chris More [:cmore] from comment #1)
> Three questions:
> 
> 1) :Gijs: Can we do another auto migration build (like we did with the Larch
> brand in the previous funnelcake) with Firefox 49.0 release channel?

Dolske is probably a better person to answer this question.

Comment 3

[Chris AtLee [:catlee]]

(In reply to Chris More [:cmore] from comment #1)
> Three questions:
> 
> 1) :Gijs: Can we do another auto migration build (like we did with the Larch
> brand in the previous funnelcake) with Firefox 49.0 release channel?
> 
> 2) :nthomas: Any immediate blockers you can think of here?
> 
> 3) :catlee: Anything we should do different this time around? 
> 
> Development of the v1.2 add-on started last week and we are trying to keep
> as much as possible the same with this next improvement. We need to validate
> some hypotheses that weren't extremely successful with the v1.0. Like any
> v1.0, we can do better, which is the next iteration.

Let's make sure we keep release-drivers in the loop this time (cc'ed Sylvestre).

Can we try and get all the binaries ready for the 21st so we don't have to rush to get updates, testing, etc. done place for the 28th?

Comment 4

[Chris More [:cmore]]

(In reply to Chris AtLee [:catlee] from comment #3)
> (In reply to Chris More [:cmore] from comment #1)
> > Three questions:
> > 
> > 1) :Gijs: Can we do another auto migration build (like we did with the Larch
> > brand in the previous funnelcake) with Firefox 49.0 release channel?
> > 
> > 2) :nthomas: Any immediate blockers you can think of here?
> > 
> > 3) :catlee: Anything we should do different this time around? 
> > 
> > Development of the v1.2 add-on started last week and we are trying to keep
> > as much as possible the same with this next improvement. We need to validate
> > some hypotheses that weren't extremely successful with the v1.0. Like any
> > v1.0, we can do better, which is the next iteration.
> 
> Let's make sure we keep release-drivers in the loop this time (cc'ed
> Sylvestre).
> 
> Can we try and get all the binaries ready for the 21st so we don't have to
> rush to get updates, testing, etc. done place for the 28th?

+1 Funny you mentioned the 21st because that's the milestone date we are targeting. Completely agree.

Comment 5

[Nick Thomas [:nthomas] (UTC+12)]

Can we get all the changes in the mainline 49.0 and avoid a special compile ? Or do we still hit the 'distribution prefs are loaded too late' problem, and need a build to enable the importer in firefox.js ? 

Going by the list of changes in 
 http://hg.mozilla.org/releases/mozilla-release/graph/0cc1138c8b72f90f53da504fe3bd4b654be455f1
back to the branch point at 7f5abf95991b we had
* in v50: bug 1283565, bug 1283542, bug 1283267 (+ rebase fix 7c68bc7de10f), bug 1199678, bug 1281602, bug 1271799, bug 1271775
* in v48: bug 1275114, bug 1260483
plus the automigration toggled on with http://hg.mozilla.org/releases/mozilla-release/rev/7e9d07aa2361

Comment 6

[Justin Dolske [:Dolske]]

Sorry for not updating this earlier, we were discussing this in email.

Gijs is going to be working this week on porting & uplifting the automigration work to beta-49 -- disabled behind a pref -- so that Funnelcakes wanting to use this will just be normal pref-flip Funnelcakes and not require a custom binary build. We wanted to do this anyway so we can do some automigration trials on Beta.

Comment 7

[:Gijs (away/busy until Mar 10)]

(In reply to Justin Dolske [:Dolske] from comment #6)
> Sorry for not updating this earlier, we were discussing this in email.
> 
> Gijs is going to be working this week on porting & uplifting the
> automigration work to beta-49 -- disabled behind a pref -- so that
> Funnelcakes wanting to use this will just be normal pref-flip Funnelcakes
> and not require a custom binary build. We wanted to do this anyway so we can
> do some automigration trials on Beta.

I will definitely work on getting all this uplifted, but:

(In reply to Nick Thomas [:nthomas] from comment #5)
> Or do we still hit the 'distribution prefs are loaded too late' problem,
> and need a build to enable the importer in firefox.js ?

This might still be true? I haven't tested this (and am not sure how to do so).

Comment 8

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to :Gijs Kruitbosch from comment #7)
> This might still be true? I haven't tested this (and am not sure how to do
> so).

Roughly - install a build/run make package. Drop a distribution/distribution.ini file into it, eg https://github.com/mozilla-partners/funnelcake/blob/master/desktop/funnelcake85/distribution/distribution.ini, plus the migrator pref. Move existing profiles away before running.

Comment 9

[Justin Dolske [:Dolske]]

(In reply to :Gijs Kruitbosch (away 26-29 incl.) from comment #7)

> (In reply to Nick Thomas [:nthomas] from comment #5)
> > Or do we still hit the 'distribution prefs are loaded too late' problem,
> > and need a build to enable the importer in firefox.js ?
> 
> This might still be true? I haven't tested this (and am not sure how to do
> so).

Hmm, god catch, this would definitely be good to test so there are no last-minute surprises.

There's already a distribution.ini setting to disable migration entirely (EnableProfileMigrator), so if needed I'd expect it should be fairly simple to add a similar setting to enable automigration.

Comment 10

[Mike Kaply [:mkaply]]

> There's already a distribution.ini setting to disable migration entirely (EnableProfileMigrator), so if needed I'd expect it should be fairly simple to add a similar setting to enable automigration.

FYI, that works by adding an extra read of the distribution.ini similar to where override.ini is read:

https://dxr.mozilla.org/mozilla-central/source/toolkit/xre/nsAppRunner.cpp#4083

Comment 11

[:Gijs (away/busy until Mar 10)]

(In reply to Nick Thomas [:nthomas] from comment #8)
> (In reply to :Gijs Kruitbosch from comment #7)
> > This might still be true? I haven't tested this (and am not sure how to do
> > so).
> 
> Roughly - install a build/run make package. Drop a
> distribution/distribution.ini file into it, eg
> https://github.com/mozilla-partners/funnelcake/blob/master/desktop/
> funnelcake85/distribution/distribution.ini, plus the migrator pref. Move
> existing profiles away before running.

Checked, it doesn't work. It does show the modified pref in about:config, but at the time where the migration code runs the distribution.ini stuff hasn't kicked in yet. I'll file a bug so that there's a distribution.ini method of enabling/disabling it so we don't need a separate build.

Comment 12

[Nick Thomas [:nthomas] (UTC+12)]

Landed some initial configs based on 85 & 86:
https://github.com/mozilla-partners/funnelcake/commit/8585e706ceaa9fb508fbd05e5d7872489b02fe68
https://github.com/mozilla-partners/funnelcake/commit/e00304c7a36ae72ed993032cd0d91b05ae71cc75

88 is still using the v1.0 XPI but setting the browser.migrate.automigrate.enabled pref to true.

Test builds:
https://archive.mozilla.org/pub/firefox/candidates/49.0-candidates/build2/funnelcake87/win32/en-US/Firefox%20Setup%2049.0.exe
(SHA1: e51c72f074276fc135329cdc22e46b8853899ca9)

https://archive.mozilla.org/pub/firefox/candidates/49.0-candidates/build2/funnelcake88-testing/v1/win32/en-US/Firefox%20Setup%2049.0.exe 
(SHA1: 94d04acafaab4e757e1d340641d9d3d3d48ac456)

[Generated with stock partner script, manual upload of 88 because upload_to_candidates=true commented out]

FYI, in 88 there's a notification bar shown with 'We automatically imported your data from Google Chrome. Would you like to keep it?'. Might have to make sure that fits with the rest of the onboarding experience.

Comment 13

[:Gijs (away/busy until Mar 10)]

(In reply to Nick Thomas [:nthomas] from comment #12)
> FYI, in 88 there's a notification bar shown with 'We automatically imported
> your data from Google Chrome. Would you like to keep it?'. Might have to
> make sure that fits with the rest of the onboarding experience.

This can be turned off by flipping the pref browser.migrate.automigrate.ui.enabled to false.

Comment 14

[Chris More [:cmore]]

Attachment: v1.2 onboarding add-on (latest version)

Here's the latest version of the XPI. We have a few more minor adjustments that we will have wrapped up by Friday, Sept 16th.

Comment 15

[Chris More [:cmore]]

:nthomas: given that we had issues before with these funnelcake builds, I want to start as early as possible creating funnelcakes to work out any issues now. 

In comment 14 is the latest version of the XPI.

Can you create the first version of funnelcake 88 with:

* XPI + 
* Firefox build with auto-migration enabled (browser.migrate.automigrate.ui.enabled = true) + 
* disable signing for these first few testing versions (xpinstall.signatures.required=false) 

?

We may need :Gijs here to help figure out exactly what build we should be using that has auto-migration in it.

We need to do some functional testing on the onboarding team to ensure everything is working as planned before kicking it over to QA for their testing.

Comment 16

[Nick Thomas [:nthomas] (UTC+12)]

I'll generate the builds. Based on testing the builds comment #12 (which are based on 49.0 RC2), the auto migrator can be enabled in distribution.ini and seems to work fine.

Comment 17

[Nick Thomas [:nthomas] (UTC+12)]

Test builds (unsigned installer and XPI):
http://archive.mozilla.org/pub/firefox/candidates/49.0-candidates/build3/funnelcake87/win32/en-US/Firefox%20Setup%2049.0.exe
(SHA1: 552c0d3a0336c84af478cdbc83c634d49393943a)

http://archive.mozilla.org/pub/firefox/candidates/49.0-candidates/build3/funnelcake88-testing/v2/win32/en-US/Firefox%20Setup%2049.0.exe
(SHA1: 552c0d3a0336c84af478cdbc83c634d49393943a)

Based on https://github.com/mozilla-partners/funnelcake/commit/87a4908ceb653376be0103b84dea8c6d625b3d0f, and 49.0 rc3 (mozilla-release b22f7ff530bb).

In funnelcake88, I've left browser.migrate.automigrate.ui.enabled alone, since it defaults to true. I think you meant browser.migrate.automigrate anyway, and that is set in the config.

Comment 18

[:Gijs (away/busy until Mar 10)]

Just so we're all on the same page here:

- all the code is on 49, so any builds off recent beta (and soon, maybe already? release) should do. 
- I wrote code that explicitly reads the pref from distribution.ini on startup in bug 1299445 (which is now also already on 49), so although setting just the pref in the distribution config didn't work when I wrote comment #11, it works now.
- There are two prefs at play:

browser.migrate.automigrate.enabled -- defaults to false, if set to true, enables the automigration functionality. The funnelcake will need this set to true.

browser.migrate.automigrate.ui.enabled -- defaults to true, if set to false, disables the notification bar that is mentioned in the tail end of comment #12. We will prompt the user from about:home when the automatic migration is complete, telling them (a) that we migrated their data and (b) offering them the chance to undo that (or keep it, or do nothing). We'll keep prompting until the user makes an explicit choice, or until the prompt has been shown on 3 different days (but it could be shown more than once on the same day if the user repeatedly visited about:home / opened Firefox).

I don't really have a strong opinion on whether the default UI stays on for this funnelcake - it's up to Chris + team. The builds from comment #17 should let you test the UI and decide whether you want it or not.

Comment 19

[Chris More [:cmore]]

(In reply to :Gijs Kruitbosch from comment #18)
> Just so we're all on the same page here:
> 
> - all the code is on 49, so any builds off recent beta (and soon, maybe
> already? release) should do. 
> - I wrote code that explicitly reads the pref from distribution.ini on
> startup in bug 1299445 (which is now also already on 49), so although
> setting just the pref in the distribution config didn't work when I wrote
> comment #11, it works now.
> - There are two prefs at play:
> 
> browser.migrate.automigrate.enabled -- defaults to false, if set to true,
> enables the automigration functionality. The funnelcake will need this set
> to true.
> 
> browser.migrate.automigrate.ui.enabled -- defaults to true, if set to false,
> disables the notification bar that is mentioned in the tail end of comment
> #12. We will prompt the user from about:home when the automatic migration is
> complete, telling them (a) that we migrated their data and (b) offering them
> the chance to undo that (or keep it, or do nothing). We'll keep prompting
> until the user makes an explicit choice, or until the prompt has been shown
> on 3 different days (but it could be shown more than once on the same day if
> the user repeatedly visited about:home / opened Firefox).
> 
> I don't really have a strong opinion on whether the default UI stays on for
> this funnelcake - it's up to Chris + team. The builds from comment #17
> should let you test the UI and decide whether you want it or not.

We'll test now and see what it looks like with both UI's and if it is too much. Thanks so much for all of the information! Stay tuned

Comment 20

[Chris More [:cmore]]

(In reply to Nick Thomas [:nthomas] from comment #17)
> Test builds (unsigned installer and XPI):
> http://archive.mozilla.org/pub/firefox/candidates/49.0-candidates/build3/
> funnelcake87/win32/en-US/Firefox%20Setup%2049.0.exe
> (SHA1: 552c0d3a0336c84af478cdbc83c634d49393943a)
> 
> http://archive.mozilla.org/pub/firefox/candidates/49.0-candidates/build3/
> funnelcake88-testing/v2/win32/en-US/Firefox%20Setup%2049.0.exe
> (SHA1: 552c0d3a0336c84af478cdbc83c634d49393943a)
> 
> Based on
> https://github.com/mozilla-partners/funnelcake/commit/
> 87a4908ceb653376be0103b84dea8c6d625b3d0f, and 49.0 rc3 (mozilla-release
> b22f7ff530bb).
> 
> In funnelcake88, I've left browser.migrate.automigrate.ui.enabled alone,
> since it defaults to true. I think you meant browser.migrate.automigrate
> anyway, and that is set in the config.

:nthomas: Just want to confirm which funnelcake is which. Is 87 the onboarding cohort and 88 the control?

If so, that's fine, but my original requirements above were reversed. Just want to make sure we are all aligned and I can update my documentation.

Comment 21

[Chris More [:cmore]]

:nthomas: it looks like the pref is not enabled to allow unsigned add-ons in the funnelcake. We installed funnelcake 87, but the add-on was automatically disabled. Can you double check your configs and confirm 87 vs 88 builds mentioned above?

Thanks!

Comment 22

[Chris More [:cmore]]

:Gijs: can you point me to a screenshot of what the auto-migration decision UI (delete or keep) prompt looks like?

Comment 23

[Chris More [:cmore]]

Attachment: v1.2 onboarding add-on (latest version)

Latest XPI is here (not signed yet)

Comment 24

[:Gijs (away/busy until Mar 10)]

(In reply to Chris More [:cmore] from comment #21)
> :nthomas: it looks like the pref is not enabled to allow unsigned add-ons in
> the funnelcake. We installed funnelcake 87, but the add-on was automatically
> disabled. Can you double check your configs and confirm 87 vs 88 builds
> mentioned above?
> 
> Thanks!

I think as of 48 on release builds that pref is not honoured anymore. So it'll be set, but not be making a difference. You could check in about:config, of course...

(In reply to Chris More [:cmore] from comment #22)
> :Gijs: can you point me to a screenshot of what the auto-migration decision
> UI (delete or keep) prompt looks like?

http://imgur.com/a/eWw7a

Comment 25

[Chris More [:cmore]]

:nthomas: another thing. As Gijs mentioned in comment 18, the auto-import UI is able to be turned on/off. For these funnelcakes, we want them turned off (browser.migrate.automigrate.ui.enabled=false) to eliminate a variable that may or may not conflict with onboarding. This isn't something we would do normally, do for this experiment, we don't want to add something to the mix that makes it hard to compare to v1.0.

Comment 26

[Nick Thomas [:nthomas] (UTC+12)]

cmore, I've checked the configuration, and the builds I pointed to in comment #17. 87 is the control and 88 has the xpi, as requested. Sounds like it's the deprecation of the xpinstall pref that's causing confusion there.

I'll prep a config change to add browser.migrate.automigrate.ui.enabled, but hold off building anything until we have a signed xpi, or some other way to use an unsigned XPI.

Comment 27

[Chris More [:cmore]]

Attachment: v1.2 onboarding add-on (latest signed version)

:nthomas: here is the signed XPI. Can you make the funnelcake with this XPI so that QA can start testing?

Comment 28

[Chris More [:cmore]]

Attachment: v1.2 onboarding add-on (latest signed version)

Just obsoleted the previous uploads.

Comment 29

[Chris More [:cmore]]

:Gijs:

I just want to confirm that we can still check the success of the auto migration (and undo promise) with these new funnelcakes and the branch :nthomas is working from with logic like:

This was our try/catch for the July funnekcake experiment:

try {
   Cu.import('resource:///modules/AutoMigrate.jsm');
   var canUndoPromise = AutoMigrate.canUndo();
}
catch (e) {
  console.error('Unable to access automigrate.jsm' + e);
}

Comment 30

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

Hi Ryan, similar to the sign offs from the last Funnelcake experiment in July, will someone from your team help with QA sign-offs? Based on Chris's latest email, they are hoping to go-live with this experiment on Sept 28th. Here's the link to the test plan: https://docs.google.com/document/d/1ImvdNZUE6ontKPhrUZkkmq4gjqf6GfUQethEeyCFY_w/edit#

Hi Chris, could you please update the test plan with the new scenarios that need testing since v1.0 experiment? Thanks!

Comment 31

[Michelle Funches - QA]

QA Update: Yes Vegas location is familiar and we can assist in testing.

Comment 32

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

Hi Chris, I hope the FC build will be based off on m-r and include the latest commits like the cert pinning one here https://hg.mozilla.org/releases/mozilla-release/rev/416dc3163a1f27b8783ed14660d1b373e830df2f

Comment 33

[Chris More [:cmore]]

(In reply to Chris More [:cmore] from comment #27)
> Created attachment 8792568 [details]
> v1.2 onboarding add-on (latest signed version)
> 
> :nthomas: here is the signed XPI. Can you make the funnelcake with this XPI
> so that QA can start testing?

:rail: is this something you can help if :nthomas will be out? We need to do some internal functional testing to ensure everything is working correctly and for QA to do their tests.

Comment 34

[Chris More [:cmore]]

(In reply to Ritu Kothari (:ritu) from comment #32)
> Hi Chris, I hope the FC build will be based off on m-r and include the
> latest commits like the cert pinning one here
> https://hg.mozilla.org/releases/mozilla-release/rev/
> 416dc3163a1f27b8783ed14660d1b373e830df2f

yup! I will create a new doc and start to outline it all now.

Comment 35

[Rail Aliiev [:rail]]

Let me take a look at this.

Comment 36

[:Gijs (away/busy until Mar 10)]

(In reply to Chris More [:cmore] from comment #29)
> :Gijs:
> 
> I just want to confirm that we can still check the success of the auto
> migration (and undo promise) with these new funnelcakes and the branch
> :nthomas is working from with logic like:
> 
> This was our try/catch for the July funnekcake experiment:
> 
> try {
>    Cu.import('resource:///modules/AutoMigrate.jsm');
>    var canUndoPromise = AutoMigrate.canUndo();
> }
> catch (e) {
>   console.error('Unable to access automigrate.jsm' + e);
> }

Ugh, I should have brought this up sooner. No, I changed the canUndo contract, and it no longer returns a promise but returns whether undo is possible synchronously. So:

try {
  Cu.import("resource:///modules/AutoMigrate.jsm");
  var canUndo = AutoMigrate.canUndo();
}
catch (ex) {
  console.error("Unable to access Automigrate.jsm" + ex);
}

if (canUndo) {
  // Undo is possible.
}


whereas before, you'd have had to use canUndoPromise.then(function(canUndo) { if (canUndo) { ... } }), etc.

Comment 37

[Rail Aliiev [:rail]]

Shall I wait for 1.3? Otherwise I'll merge https://github.com/mozilla-partners/funnelcake/pull/33 and generate an installer.

Comment 38

[Chris More [:cmore]]

(In reply to :Gijs Kruitbosch from comment #36)
> (In reply to Chris More [:cmore] from comment #29)
> > :Gijs:
> > 
> > I just want to confirm that we can still check the success of the auto
> > migration (and undo promise) with these new funnelcakes and the branch
> > :nthomas is working from with logic like:
> > 
> > This was our try/catch for the July funnekcake experiment:
> > 
> > try {
> >    Cu.import('resource:///modules/AutoMigrate.jsm');
> >    var canUndoPromise = AutoMigrate.canUndo();
> > }
> > catch (e) {
> >   console.error('Unable to access automigrate.jsm' + e);
> > }
> 
> Ugh, I should have brought this up sooner. No, I changed the canUndo
> contract, and it no longer returns a promise but returns whether undo is
> possible synchronously. So:
> 
> try {
>   Cu.import("resource:///modules/AutoMigrate.jsm");
>   var canUndo = AutoMigrate.canUndo();
> }
> catch (ex) {
>   console.error("Unable to access Automigrate.jsm" + ex);
> }
> 
> if (canUndo) {
>   // Undo is possible.
> }
> 
> 
> whereas before, you'd have had to use canUndoPromise.then(function(canUndo)
> { if (canUndo) { ... } }), etc.

ok, I will file a github issue. It should be an easy change. Thanks

Comment 39

[Chris More [:cmore]]

I filed a github issue for the undo migration condition here https://github.com/mozilla/all-aboard/issues/370

Comment 40

[Chris More [:cmore]]

(In reply to Rail Aliiev [:rail] from comment #37)
> Shall I wait for 1.3? Otherwise I'll merge
> https://github.com/mozilla-partners/funnelcake/pull/33 and generate an
> installer.

I'd build the funnelcake with the XPI in comment 28 as we need to get on to functional testing, because we have seen before that nothing is real until we are testing inside a real build as JPM doesn't always simulate everything.

Comment 41

[Chris More [:cmore]]

Also, :rail, the July experiment had 10 revisions to the funnelcake until we got one that had all of the bug resolved and configs figured out. I expect to have some revisions too throughout the QA process.

Comment 42

[Rail Aliiev [:rail]]

hehe, OK, I'll build one then.

Comment 43

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Notes on generating builds

Rail's PR to update the xpi - https://github.com/mozilla-partners/funnelcake/commit/47dd4a01be2e2dcf8c942c8c7111683682b71faa
My PR to tweak prefs and xpi location - https://github.com/mozilla-partners/funnelcake/commit/e92b9a9545392971705509b6a6c4aed61d557722

Based on 49.0 build4 (rev: 416dc3163a1f, buildID: 20160916101415)

fc87:
https://archive.mozilla.org/pub/firefox/candidates/49.0-candidates/build4/funnelcake87/win32/en-US/Firefox%20Setup%2049.0.exe (SHA1: 684072968c5437172e6546597a52d21f0741d4c2)

fc88:
https://archive.mozilla.org/pub/firefox/candidates/49.0-candidates/build4/funnelcake88-testing/v3/win32/en-US/Firefox%20Setup%2049.0.exe (SHA1: 4bcac296db76af9ce5019d75211201d7c017f1eb)

Passes a smoketest for importing bookmarks from Chrome, and modifying the first run page.

Comment 44

[Rail Aliiev [:rail]]

Nick's back! :)

Comment 45

[Chris More [:cmore]]

(In reply to Rail Aliiev [:rail] from comment #44)
> Nick's back! :)

woo hoo. Thanks :rail and :nthomas. We're doing some testing while I am building the QA test plan.

Comment 46

[Nick Thomas [:nthomas] (UTC+12)]

Bouncer setup is done on the assumption that we'll extrapolate from bug 1272776, and use:
 https://download.mozilla.org/?product=firefox-stub-f87&os=win&lang=en-US 
 https://download.mozilla.org/?product=firefox-stub-f88&os=win&lang=en-US 

The matching locations are:
/firefox/releases/49.0-funnelcake87/win32/en-US/Firefox Setup 49.0.exe
/firefox/releases/49.0-funnelcake88/win32/en-US/Firefox Setup 49.0.exe

[Also trimmed the list of products with the Check Now flag set from 278 to 71]

Comment 47

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Push commands

My availability may be variable in the next day or two, so this comment is directed at Release Engineers who need to pick this work up:

Process from here:
* regenerate builds at cmore's request (eg updated XPI or additional pref changes) - we're expecting one more for https://github.com/mozilla/all-aboard/issues/370. See comment #43 for notes, and the sort of information I've been providing. Please note that I've been publishing 88 to funnelcake88-testing/v{N}/ to avoid CDN caching issues, so increment N each time. Funnelcake87 is the control and static, so I haven't versioned that.

* when cmore and release-drivers are ready to ship we need to copy files around. This could be done on the partner repack machine with the commands attached here. Then verify the urls in comment #46 work after about 5 minutes (SHA1 checks are good to avoid mixing files up).

In the last round of these funnelcakes (bug 1272776) there was a need to update the xpi after shipping. See the last few comments of that bug for details of how the paths were changed (avoiding caching again), with matching updates to bouncer.

Comment 48

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to Nick Thomas [:nthomas] from comment #47)
> See the last few comments of that bug for details of how the paths were changed 
> (avoiding caching again), with matching updates to bouncer.

Make that bug 1272776 comment #122 to 125.

Comment 49

[Chris More [:cmore]]

Quick update:

Here's the QA test plan we are working on:

https://docs.google.com/document/d/1T-lY0r44aAWW9nHsg6Mr5s3aWq2LdMl5KC48KY7EuCw/edit#

We discovered a bug in testing the first version of the add-on. We believe it is now resolved and I am signing the add-on now and updating the add-on in the funnelcake to see if it is resolved. If it is, We'll have :nthomas spin up a new funnelcake and proceed with testing.

Will update this bug when we have a new XPI.

Comment 50

[Chris More [:cmore]]

Another update.

We discovered a few bugs with the XPI that came up during our own internal testing. There's no need to have QA review the build until everyworks works as expect, because they'll get blocked quickly.

We also found it is easier to just locally sign the XPI and re-install it in the funnelcake 88 that :nthomas created above thus saving the back and forth between teams to spin another funnelcake. Once we get the bugs resolved, we'll do a final one for Softvision QA and others to review/approve.

More soon!

Comment 51

[Chris More [:cmore]]

Attachment: v1.2 onboarding add-on (latest signed version)

Woo hoo! We got all the main bugs resolved. We are ready for a new funnelcake 88 :nthomas and then Michelle and team can get rolling with their testing.

Comment 52

[Chris More [:cmore]]

Until :nthomas creates the new funnelcake, here's the instructions on how to get testing the onboarding experience.

1) Install the old funnelcake 88 from comment 43 on a Windows machine.

2) Type firefox.exe -p and kill any local profile

3) Open Firefox

4) Go to about:addons

5) Go to the wrench icon and install from file the v1.2 latest version XPI from the attachments above.

6) Go back to first run and refresh the page: https://www.mozilla.org/en-US/firefox/49.0/firstrun/?f=88

7) Answer the questions and follow this doc on expected behaviors:

https://docs.google.com/document/d/1T-lY0r44aAWW9nHsg6Mr5s3aWq2LdMl5KC48KY7EuCw/edit#

When you want to re-start the onboarding experience, just go back to step 2 above and kill the profile and do the steps over again.

Comment 53

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Notes on generating builds, with push commands

PI updated at https://github.com/mozilla-partners/funnelcake/commit/7423588b5853eceeb601cd59bffba0533b66a8b4

Based on 49.0.1 build3 (rev: 2d931a5eaf8a, buildID: 20160922113459)

fc87:
https://archive.mozilla.org/pub/firefox/candidates/49.0.1-candidates/build3/funnelcake87/win32/en-US/Firefox%20Setup%2049.0.1.exe (SHA1: 1fa678a20f7d2d8c3a9c1a9d5cd2c78182179102)
 
fc88:
https://archive.mozilla.org/pub/firefox/candidates/49.0.1-candidates/build3/funnelcake88-testing/v4/win32/en-US/Firefox%20Setup%2049.0.1.exe (SHA1: a0ea298f90fcd9a4d914cac98500bc9743f3d39b)

These are fully signed and ready for release, pending QA results. Bouncer has been updated for 49.0.1 locations.

Comment 54

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Notes on generating builds, with push commands

Minor fixes to push commands.

Comment 55

[Chris More [:cmore]]

Attachment: v1.2 onboarding add-on (latest signed version)

:nthomas: here's the latest XPI to create v5 of fc88. There will be one more after this (famous last words).

Comment 56

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Notes on generating builds, with push commands, v5

PI updated at https://github.com/mozilla-partners/funnelcake/commit/c3adf7ebd3dbb06e8690620e4f88f85d1f49599c

Based on 49.0.1 build3 (rev: 2d931a5eaf8a, buildID: 20160922113459)

fc87: see comment #53

fc88:
https://archive.mozilla.org/pub/firefox/candidates/49.0.1-candidates/build3/funnelcake88-testing/v5/win32/en-US/Firefox%20Setup%2049.0.1.exe (SHA1: 17eac15965e1b8d81e7baec186cba17d042d7c29)

Comment 57

[Chris More [:cmore]]

Attachment: v1.2 onboarding add-on (latest signed version)

Here's RC5 of v1.2 add-on. Looking good!

Comment 58

[Michelle Funches - QA]

SV-Vegas has completed Funnelcake 88 testing - All is well.
Focus was placed on Windows 7, 8.1 and 10 OS for :
   Funnelcake overall functionality
   Web Compatibility
Full test results can be seen in the Funnelcake workbook tab by accessing the link below
   https://docs.google.com/spreadsheets/d/1x5_ZvApST5cFfOqg2JePJ6B9GoZPDIPgN3n0OQs-Hd4/edit#gid=68420664
Smoke testing was also complete for 1.2.9-RC5 - no issues to report.

Comment 59

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Notes on generating builds, with push commands, v6

PI updated at https://github.com/mozilla-partners/funnelcake/commit/a84aceb1927319cd564719905b01f680e077ee93

Based on 49.0.1 build3 (rev: 2d931a5eaf8a, buildID: 20160922113459), with RC5 of the XPI.

fc87: See comment #53

fc88:
https://archive.mozilla.org/pub/firefox/candidates/49.0.1-candidates/build3/funnelcake88-testing/v6/win32/en-US/Firefox%20Setup%2049.0.1.exe (SHA1: 0e27d5edb12683fbf19fde28622c447dc638656d)

Comment 60

[Chris More [:cmore]]

Attachment: v1.2 onboarding add-on (latest AMO signed version)

We just had the XPI listed and approved on AMO, thus AMO signed the XPI and that's the version we want to bundled in the final funnelcake. 7 times is a charm!

Comment 61

[Chris More [:cmore]]

Add-on is reviewed, listed and approved by the AMO team! https://addons.mozilla.org/en-US/firefox/addon/firefox-all-aboard-1-2/

Comment 62

[Chris More [:cmore]]

:nthomas: after you the v7 of funnelcake 88 build with the AMO signed add-on in comment 60, we should be good. Just need to get the build up on the CDN and the bouncer links should then work. The final test is to just that this URL downloads the correct build on a win32 machine:

https://www.mozilla.org/en-US/firefox/new/?f=87 (control)

https://www.mozilla.org/en-US/firefox/new/?f=88 (auto migration build + onboarding xpi)

Comment 63

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Notes on generating builds, with push commands, v7

PI updated at https://github.com/mozilla-partners/funnelcake/commit/452d86b4ca2d4a720e6b582a4e1ecaa13d031c5b

Based on 49.0.1 build3 (rev: 2d931a5eaf8a, buildID: 20160922113459)

fc87: See comment #53

fc88: https://archive.mozilla.org/pub/firefox/candidates/49.0.1-candidates/build3/funnelcake88-testing/v7/win32/en-US/Firefox%20Setup%2049.0.1.exe (SHA1: 16b86f4a4277468f98b2d9ef70f05fb28c1b7dd8)

cmore, could you please spot-check that before I push to the CDN.

Comment 64

[Chris More [:cmore]]

(In reply to Nick Thomas [:nthomas] from comment #63)
> Created attachment 8795563 [details]
> Notes on generating builds, with push commands, v7
> 
> XPI updated at
> https://github.com/mozilla-partners/funnelcake/commit/
> 452d86b4ca2d4a720e6b582a4e1ecaa13d031c5b
> 
> Based on 49.0.1 build3 (rev: 2d931a5eaf8a, buildID: 20160922113459)
> 
> fc87: See comment #53
> 
> fc88:
> https://archive.mozilla.org/pub/firefox/candidates/49.0.1-candidates/build3/
> funnelcake88-testing/v7/win32/en-US/Firefox%20Setup%2049.0.1.exe (SHA1:
> 16b86f4a4277468f98b2d9ef70f05fb28c1b7dd8)
> 
> cmore, could you please spot-check that before I push to the CDN.

Doing so now.

Comment 65

[Chris More [:cmore]]

(In reply to Chris More [:cmore] from comment #64)
> (In reply to Nick Thomas [:nthomas] from comment #63)
> > Created attachment 8795563 [details]
> > Notes on generating builds, with push commands, v7
> > 
> > XPI updated at
> > https://github.com/mozilla-partners/funnelcake/commit/
> > 452d86b4ca2d4a720e6b582a4e1ecaa13d031c5b
> > 
> > Based on 49.0.1 build3 (rev: 2d931a5eaf8a, buildID: 20160922113459)
> > 
> > fc87: See comment #53
> > 
> > fc88:
> > https://archive.mozilla.org/pub/firefox/candidates/49.0.1-candidates/build3/
> > funnelcake88-testing/v7/win32/en-US/Firefox%20Setup%2049.0.1.exe (SHA1:
> > 16b86f4a4277468f98b2d9ef70f05fb28c1b7dd8)
> > 
> > cmore, could you please spot-check that before I push to the CDN.
> 
> Doing so now.

Checked:

Funnelcake 87 in comment 53.

* distribution ID looks right
* installed fine
* original migration prompt loads
* original first run loads

Funnelcake 88 in comment 63.

* distribution ID looks right
* installed fine
* auto imported
* First run modification loads
* add-on is v1.2.9 rc5 as planned

Looks good :nthomas!

Comment 66

[Chris More [:cmore]]

:nthomas: hang on a few minutes. Checking out something on the FxA side. don't push yet. pause until I update here.

Comment 67

[Chris More [:cmore]]

(In reply to Chris More [:cmore] from comment #66)
> :nthomas: hang on a few minutes. Checking out something on the FxA side.
> don't push yet. pause until I update here.

Did a full test and tested the FxA journey and all is well. All good from us. thanks

Comment 68

[Chris More [:cmore]]

We're set to launch this experiment today pending releng push to the CDN. If anyone has any red flags, please update this bug asap.

Comment 69

[Nick Thomas [:nthomas] (UTC+12)]

Builds are pushed to the CDN, bouncer links
 https://download.mozilla.org/?product=firefox-stub-f87&os=win&lang=en-US 
 https://download.mozilla.org/?product=firefox-stub-f88&os=win&lang=en-US 
are live, and SHA1 verified.

Comment 70

[Chris More [:cmore]]

(In reply to Nick Thomas [:nthomas] from comment #69)
> Builds are pushed to the CDN, bouncer links
>  https://download.mozilla.org/?product=firefox-stub-f87&os=win&lang=en-US 
>  https://download.mozilla.org/?product=firefox-stub-f88&os=win&lang=en-US 
> are live, and SHA1 verified.

woo hoo! We'll one more test of the funnel now before launch. thanks

Comment 71

[Chris More [:cmore]]

We're live!!

Comment 72

[Chris More [:cmore]]

Experiment stopped at midnight 10/16/2016. Retention results for week 1 should be here by 11/16/2016.