Closed Bug 1326252 Opened 7 years ago Closed 7 years ago

Funnelcake 98/99/100 v2.0 Onboarding (in-product)

Categories

(Release Engineering :: Release Requests, defect)

Product:
Release Engineering
Component:
Release Requests
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: cmore, Assigned: jlund)

References

(Blocks 1 open bug)

Details

Attachments

(1 file, 1 obsolete file)

Updated distribution.ini
864 bytes, text/plain
Details 
This is a placeholder bug for now to build two funnelcakes for the upcoming v2.0 onboarding experience that is being built with www.mozilla.org, stub installer, and in-product changes. Bug 1322718 is the overall tracking bug. More details will come later as all the details get worked out.
Attached file distribution.ini (obsolete) — 
I assembled the prefs from the various bugs for this funnelcake (thanks to Dao for putting them in the whiteboard, which make it easy!).

One can test with a .zip of beta12 using this. Just unzip, add this file as distribution\distribution.ini, and launch the beta12 firefox.exe.

From just a quick look, everything seems to basically be working.
Oooh nice. Could remove browser.usedOnWindows10, that got removed a little while ago.
Assignee: nobody → nthomas
We'll need stub installers for this funnelcake. The modified experience has a re-done stub (bug 1328445), pulling in the full installer with several changes (see deps of bug 1322718). The control is to use the current stub, presumably pulling in a minimally-changed funnelcake installer, so that's different from our previous workflow.

https://dxr.mozilla.org/mozilla-release/source/browser/installer/windows/nsis/defines.nsi.in#11 is the parameter to set to 98 & 99, so that the stub modifies the download.m.o url it uses.
Depends on: 1330371
browser.migrate.automigrate.ui.enabled should be left as true (the default), and browser.newtabpage.thumbnailPlaceholder should be set to true, so I've made those changes.

I've also removed the usedOnWindows10 thing which got removed in bug 1274633 and is no longer on beta.
Attachment #8824329 - Attachment is obsolete: true
Just to update here, we'll need 3 funnelcakes (98/99/100) as defined in the test plan: https://docs.google.com/document/d/1Kle7LlVQ_gCJ2IFpQOSg7XVDtrXGNYGPacKIaYO1i9A/edit#
Summary: Funnelcake 98/99 v2.0 Onboarding (in-product) → Funnelcake 98/99/100 v2.0 Onboarding (in-product)
When will these be needed?
(In reply to Chris AtLee [:catlee] from comment #6)
> When will these be needed?

We are estimating Jan 30th to give a week for QA.
:ritu: the Firefox team is working on an in-product onboarding experience and the MVP will be launched in Feb 201. Just want to make sure this is on your radar too given you mentioned before that you want to know about upcoming funnelcakes.
Flags: needinfo?(rkothari)
I've put together a pull request with the config for the full installers, 
  https://github.com/mozilla-partners/funnelcake/pull/47
except for an onboarding addon. How's that coming along ? I dropped the startup.homepage_override_url definition in 99 and 100, since we don't set that for vanilla builds, and it will pestering our test cohorts after every update.

I'm out next week - questions to jlund/catlee/coop - and I'll see how you're going when I get back on the 29th N.A. time. Or before if I'm weak and look at email.
(In reply to Chris More [:cmore] from comment #8)
> :ritu: the Firefox team is working on an in-product onboarding experience
> and the MVP will be launched in Feb 201. Just want to make sure this is on
> your radar too given you mentioned before that you want to know about
> upcoming funnelcakes.

Hi Chris, thanks for the heads-up. GChang (fyi) owns Fx51 which goes-live tomm. This experiment is planned to start on Jan 30th (based on comment 7). When does it end? Also, please monitor r-d ML in case we have a critical issue driving a dot release in which case we should build the FC build off of 51.x (dot release) and not 51.0
Flags: needinfo?(rkothari) → needinfo?(gchang)
(In reply to Ritu Kothari (:ritu) from comment #10)
> (In reply to Chris More [:cmore] from comment #8)
> > :ritu: the Firefox team is working on an in-product onboarding experience
> > and the MVP will be launched in Feb 201. Just want to make sure this is on
> > your radar too given you mentioned before that you want to know about
> > upcoming funnelcakes.
> 
> Hi Chris, thanks for the heads-up. GChang (fyi) owns Fx51 which goes-live
> tomm. This experiment is planned to start on Jan 30th (based on comment 7).
> When does it end? Also, please monitor r-d ML in case we have a critical
> issue driving a dot release in which case we should build the FC build off
> of 51.x (dot release) and not 51.0

Hi Ritu.

The specific audience and dates are in bug 1332130, which is the Optimizely experiment that controls the distribution of the funnelcake.

Planned dates:

Start date: 2/7/2017 
End date: 2/25/2017
:ritu: who from releng should we work with until catlee and nthomas return from pto?

We need to confirm that the funnelcakes builds are being created with funnelcake stub installers too and the funnelcake stub installers point specifically to the funnelcakes. By default, the stub installers will find the latest release channel installer. Almost all of the past funnelcakes have been created using the full installer because the stub installer wasn't part of the experiment itself, but that's not the case with the Firefox team's v2.0 MVP. 

How can we confirm this setup and if anyone else needs to be involved to make the needed changes to the funnelcake stub installer?
Flags: needinfo?(rkothari)
(In reply to Chris More [:cmore] from comment #12)
> :ritu: who from releng should we work with until catlee and nthomas return from pto?

nthomas had named jlund in comment 9 and elsewhere, Jordan will you be able to help us by setting up release branches and building a stub installer with patches from bug 1328445? (specifics are from https://bugzilla.mozilla.org/show_bug.cgi?id=1328445#c33 on)

The stub installer doesn't have to be built with the rest of the funnelcake build, it just needs a small patch as in comment 3 (aside from the changes to the stub itself) to tell d-m-o the funnelcake id it wants. I'll have those three patches (for 98/99/100) up in 1328445 probably tomorrow after some more testing tonight, then I think we should be good to go.
Flags: needinfo?(jlund)
I will be investigating this today. It's merge and release week and that has taken much of my time. I will reply to this bug regardless by EOD with status and where I think we are for the Jan 30th target.
Flags: needinfo?(jlund)
The funnelcake id patches are now on bug 1333873 (a separate bug to keep the streamlining work separate)
(In reply to Jordan Lund (:jlund) from comment #14)
> I will be investigating this today. It's merge and release week and that has
> taken much of my time. I will reply to this bug regardless by EOD with
> status and where I think we are for the Jan 30th target.

status update: getting familiar with state and procedure now, will report back tomorrow regardless, hopefully with something to show.
status:

I'm skipping 51.0. We released it last week briefly but immediately created 51.0.1. So I used that version as our 'base'.


on mozilla-release, we've branched off FIREFOX_51_0_1_RELEASE and created FUNNELCAKE_98_99_100_BRANCH

within that new branch, there is 3 tags, for the 3 funnelcake versions:

* https://hg.mozilla.org/releases/mozilla-release/rev/FUNNELCAKE_98
    * en-us win32 builds will show up here: https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=85b025063f469cb63b38e300b5a3218776c1214f

* https://hg.mozilla.org/releases/mozilla-release/rev/FUNNELCAKE_99
    * win32 builds will show up here: https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=236da98804885dc6872c855dd7eeb3d9d161d6f0
    * includes two revs for streamline stub installer: c3e4e07e53fa and 8fa090c703d2

* https://hg.mozilla.org/releases/mozilla-release/rev/FUNNELCAKE_100
    * en-us win32 builds will show up here: https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=c552c07a86219b16c9ff6cbc7867d37b22885186
    * includes two revs for streamline stub installer: c3e4e07e53fa and 8fa090c703d2

once the builds show up from the above treeherder urls, I can create unsigned funnelcake repacks and upload them to a candidates url on archive.m.o for sanity checking. I plan to have those repacks available by Sunday, 29th. Once approved I'll sign and push to releases (permanent bouncer location) for the 30th.
The stub installers in the win32 builds above look right. Until the releases are up they won't be able to download and install, but they're making the right requests. Thanks!
braindump:

agashlin: hey there. where can I find the onboarding addon v2? Preferably one that is signed by AMO
12:50:41 I think I'm going to make the three repacks without the addon for now
12:52:32 https://docs.google.com/spreadsheets/d/1gZ-GFyubAAlIxh-u-5SR52rqKKfBRLRuwrjSq97QkWA/edit#gid=0 suggests we want onboarding v2.0
12:52:41 I don't see that tag/version here: https://github.com/mozilla/all-aboard
13:34:41 hm, so because I am using CI and not a release build, partner-repacks.py source suggests I need to use a script like this:
13:34:43 ./partner-repacks.py --use-ci-builds --revision 85b025063f469cb63b38e300b5a3218776c1214f -d ../partners/funnelcake/desktop
13:35:19 though, that's not makign the taskcluster index happy. and reading further, I suspect I won't be able to upload to candidates dir
13:44:49 hm, https://index.taskcluster.net/v1/task/buildbot.revisions.85b025063f469cb63b38e300b5a3218776c1214f.mozilla-release.linux-i686
13:45:03 seems to be wrong index, should be s/linux-i686/linux
13:56:13 hm, since I only care about win32, maybe I can just ignore linux via --platform
13:57:06 \o/
13:59:26 boo https://irccloud.mozilla.com/pastebin/XNjgGbcJ/ 
14:01:54 oh, fun. partner-repack1's partner-repacks.py differs from git/hg tip

...

I then locally hacked partners/funnelcake/desktop/funnelcake{98,99,100}/repack.cfg to point upload_to_candidates to false. This seemed to get past the error about original_web_dir being undefined but the script still tried to upload to s3:

2017-01-29 06:07:15,765 - INFO - Done repacking win32 build firefox-51.0.1.en-US.win32.installer.exe
2017-01-29 06:07:15,766 - INFO - RepackWorker-4: Exiting
2017-01-29 06:07:15,766 - INFO - Uploading win32 build firefox-51.0.1.en-US.win32.installer.exe
2017-01-29 06:07:15,767 - INFO - S3 's3cmd -c ~/.s3cfg-mozilla put "/Users/cltbld/funnelcake/scripts/repacked_builds/unsigned/partner-repacks/f
unnelcake99-testing/v1/win32/en-US/firefox-51.0.1.en-US.win32.installer.exe" s3://net-mozaws-prod-delivery-firefox/partner-repacks/funnelcake99
-testing/v1/win32/en-US/'
WARNING: Module python-magic is not available. Guessing MIME types based on file extensions.
WARNING: Module python-magic is not available. Guessing MIME types based on file extensions.
WARNING: Module python-magic is not available. Guessing MIME types based on file extensions.
ERROR: S3 error: 403 (AccessDenied): Access Denied

anyway, I have three repacks for 98,99, and 100. 99 and 100 are missing the onboarding v2.0 mentiond in requirements here: https://docs.google.com/spreadsheets/d/1gZ-GFyubAAlIxh-u-5SR52rqKKfBRLRuwrjSq97QkWA/edit#gid=0

The repacks are currently local on partner-repacks1 under /Users/cltbld/funnelcake/scripts/repacked_builds/unsigned/partner-repacks (I think it defaults to pwd + output_dir if upload_to_candidates==false)

The artifacts look to be `firefox-51.0.1.en-US.win32.installer.exe` rather than `Firefox Setup` but I imagine that is due to non pretty release names.

partner-repack1:partner-repacks cltbld$ openssl dgst -sha1 funnelcake98-testing/v1/win32/en-US/firefox-51.0.1.en-US.win32.installer.exe
SHA1(funnelcake98-testing/v1/win32/en-US/firefox-51.0.1.en-US.win32.installer.exe)= 8659a5aa4d8aaec1a53d8eb633a13a17999a34f6
partner-repack1:partner-repacks cltbld$ openssl dgst -sha1 funnelcake99-testing/v1/win32/en-US/firefox-51.0.1.en-US.win32.installer.exe
SHA1(funnelcake99-testing/v1/win32/en-US/firefox-51.0.1.en-US.win32.installer.exe)= bf0a94dd60fd6dbc6c9093a352ddb67c42d320f8
partner-repack1:partner-repacks cltbld$ openssl dgst -sha1 funnelcake100-testing/v1/win32/en-US/firefox-51.0.1.en-US.win32.installer.exe
SHA1(funnelcake100-testing/v1/win32/en-US/firefox-51.0.1.en-US.win32.installer.exe)= 2d2ab64725adcb484f67d9d5cfaab7772675f182

I'll try uploading these to candidates or people outside the script later today.
cmore: do you know where I can find what version of the onboarding addon we want and where I can find a signed AMO one of it? Also, can you confirm we want that addon for funnelcake 99 and 100 only. 98 we don't require any addons correct?
Flags: needinfo?(chrismore.bugzilla)
agashlin: I uploaded the following so we can start verifying the repack without the addon:

funnelcake98 (current stub, control funnelcake):
   * location: http://archive.mozilla.org/pub/firefox/candidates/51.0.1-candidates/build3/funnelcake98-testing/v1/win32/en-US/firefox-51.0.1.en-US.win32.installer.exe
   * sha1 8659a5aa4d8aaec1a53d8eb633a13a17999a34f6

funnelcake99 (streamlined stub, auto-import, does not have onboarding addon yet):
   * location: http://archive.mozilla.org/pub/firefox/candidates/51.0.1-candidates/build3/funnelcake99-testing/v1/win32/en-US/firefox-51.0.1.en-US.win32.installer.exe
   * sha1 bf0a94dd60fd6dbc6c9093a352ddb67c42d320f8

funnelcake100 (streamlined stub, no auto-import, does not have onboarding addon yet):
   * http://archive.mozilla.org/pub/firefox/candidates/51.0.1-candidates/build3/funnelcake100-testing/v1/win32/en-US/firefox-51.0.1.en-US.win32.installer.exe
   * sha1 2d2ab64725adcb484f67d9d5cfaab7772675f182
(In reply to Jordan Lund (:jlund) from comment #21)
> agashlin: I uploaded the following so we can start verifying the repack
> without the addon:
> 
> funnelcake98 (current stub, control funnelcake):
>    * location:
> http://archive.mozilla.org/pub/firefox/candidates/51.0.1-candidates/build3/
> funnelcake98-testing/v1/win32/en-US/firefox-51.0.1.en-US.win32.installer.exe
>    * sha1 8659a5aa4d8aaec1a53d8eb633a13a17999a34f6
> 
> funnelcake99 (streamlined stub, auto-import, does not have onboarding addon
> yet):
>    * location:
> http://archive.mozilla.org/pub/firefox/candidates/51.0.1-candidates/build3/
> funnelcake99-testing/v1/win32/en-US/firefox-51.0.1.en-US.win32.installer.exe
>    * sha1 bf0a94dd60fd6dbc6c9093a352ddb67c42d320f8
> 
> funnelcake100 (streamlined stub, no auto-import, does not have onboarding
> addon yet):
>    *
> http://archive.mozilla.org/pub/firefox/candidates/51.0.1-candidates/build3/
> funnelcake100-testing/v1/win32/en-US/firefox-51.0.1.en-US.win32.installer.exe
>    * sha1 2d2ab64725adcb484f67d9d5cfaab7772675f182

I just tested funnelcake99 and funnelcake100, these don't seem to include the streamlined stub installer per bug 1328445?
Hi Jordan,

We're getting ready to test the Funnelcake builds this week. Are there any preferences or distribution.ini files that will need to be set into these builds in order to begin testing? Thanks.
Flags: needinfo?(jlund)
(In reply to Jordan Lund (:jlund) from comment #20)
> cmore: do you know where I can find what version of the onboarding addon we
> want and where I can find a signed AMO one of it? Also, can you confirm we
> want that addon for funnelcake 99 and 100 only. 98 we don't require any
> addons correct?

This test doesn't include an add-on. The work is all in Firefox.
Flags: needinfo?(chrismore.bugzilla)
(In reply to Grover Wimberly IV [:Grover-QA] from comment #24)
> Hi Jordan,
> 
> We're getting ready to test the Funnelcake builds this week. Are there any
> preferences or distribution.ini files that will need to be set into these
> builds in order to begin testing? Thanks.

no preferences I'm aware about. the installs should be good to go as is from our own distribution.ini and repack configs. I'll point you to where to find the funnelcakes that can be tested in the next comment.(In 

reply to Verdi [:verdi] from comment #25)
> (In reply to Jordan Lund (:jlund) from comment #20)
> > cmore: do you know where I can find what version of the onboarding addon we
> > want and where I can find a signed AMO one of it? Also, can you confirm we
> > want that addon for funnelcake 99 and 100 only. 98 we don't require any
> > addons correct?
> 
> This test doesn't include an add-on. The work is all in Firefox.

In addition to this reply, verdi, mhowell, and agashlin clarified some expectations for each funnelcake. I thought we were using a new addon for 99 and 100 but it turns out the only thing we need is stock firefox (what was released in 51.0.1) and new stubs for each.
Flags: needinfo?(jlund)
Okay, here are the new releases and stubs for 98, 99, and 100.

The following are signed (sigs can be found in the candidates dir below):

98:
http://archive.mozilla.org/pub/firefox/candidates/51.0.1-candidates/build3/partner-repacks/funnelcake98-testing/v1/win32/en-US/
SHA1(funnelcake98-testing/v1/win32/en-US/firefox-51.0.1.en-US.win32.installer-stub.exe)= 847dbc7b4a2e6eceecc9c0658b6f673c30093bf6
SHA1(funnelcake98-testing/v1/win32/en-US/Firefox Setup 51.0.1.exe = fb6cec433aef025ad85d9213eff0aa174c4afa00


99:
http://archive.mozilla.org/pub/firefox/candidates/51.0.1-candidates/build3/partner-repacks/funnelcake99-testing/v1/win32/en-US/
SHA1(funnelcake99-testing/v1/win32/en-US/firefox-51.0.1.en-US.win32.installer-stub.exe)= a70c92b95972ccf301b7f89545411891af2f4bec
SHA1(funnelcake99-testing/v1/win32/en-US/Firefox Setup 51.0.1.exe)= cacf44ce293c5a54482e7747489bee4c11a55fb9

100:
http://archive.mozilla.org/pub/firefox/candidates/51.0.1-candidates/build3/partner-repacks/funnelcake100-testing/v1/win32/en-US/
SHA1(funnelcake100-testing/v1/win32/en-US/Firefox Setup 51.0.1.exe)= 0b4713a01d8440af64366979ccbc0e7fed844193
SHA1(funnelcake100-testing/v1/win32/en-US/firefox-51.0.1.en-US.win32.installer-stub.exe)= c9ade98c97431bae5cec81bc95e2bcade265ad73

These are also available for download over bouncer / download.m.o. Note this is just for testing purposes as we iterate until it passes expectation of this bug and QA:

https://download.mozilla.org/?product=firefox-stub-f98&os=win&lang=en-US
https://download.mozilla.org/?product=firefox-latest-f98&os=win&lang=en-US

https://download.mozilla.org/?product=firefox-stub-f99&os=win&lang=en-US
https://download.mozilla.org/?product=firefox-latest-f99&os=win&lang=en-US

https://download.mozilla.org/?product=firefox-stub-f100&os=win&lang=en-US
https://download.mozilla.org/?product=firefox-latest-f100&os=win&lang=en-US

cmore, agashlin: I'll stand by for releng work once grover/QA or someone signs off on these.
Thanks jlund, the stubs all install as expected. Anyone testing can follow the ?product=firefox-stub-fXX links above to run through the whole installation procedure.

Someone familiar with the in-product features will have to report on those.
QA asked how these funnelcakes differ. To complement what was described in the test plan[1], I did the following: https://public.etherpad-mozilla.org/p/funnelcake_description

[1] https://docs.google.com/document/d/1Kle7LlVQ_gCJ2IFpQOSg7XVDtrXGNYGPacKIaYO1i9A/edit
Assignee: nthomas → jlund
Hi everyone. What's the status on the QA sign-off on these funnelcakes?
Preliminary Sign-Off coming within the hour.
Preliminary Sign-Off has been sent.
Hi Jordan, is there anything (besides bug 1336969) that needs to be done from the releng side before we can go live with these?
Flags: needinfo?(jlund)
We've live!
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Experiment started on 2/7/2017 @ 6:05pm PST.
Experiment update:

Split out Funnelcake 99 from 98 and 100, because of a Chrome import issue.

Funnelcake 99 = non-Firefox + non-Chrome + Windows + English + Desktop
Funnelcake 98/100 = non-Firefox + Windows + English + Desktop

Experiment restarted at 8:50pm pst on 2/11/2017.
Funnelcake stopped on 2/27/2017 at 8am PST.
Component: Custom Release Requests → Release Requests
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: