Closed Bug 892848 Opened 11 years ago Closed 11 years ago

spin up funnelcake build for SEM

Categories

(Release Engineering :: Release Requests, defect)

Product:
Release Engineering
Component:
Release Requests
Platform:
x86
macOS
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: aphadke, Assigned: nthomas)

References

Details

Attachments

(4 files, 1 obsolete file)

Notes for creating modified stub installers
5.64 KB, text/plain
Details
Notes for creating repacked installers
3.68 KB, text/plain
Details
Notes for creating repacked installers (FIrefox 23.0)
3.68 KB, text/plain
Details
[partner-repacks] Generate 23.0.1 funnelcake23 in automation
974 bytes, patch
mozilla
: review+
nthomas
: checked-in-
Details | Diff | Splinter Review 
Marketing is planning to run a SEM (Search Engine Marketing) campaign for users in Indonesia and Brazil. We need to spin up two funnelcake builds, one for each country, to understand the quality of users and long term value of users.

Please spin up a funnelcake build for pt-BR and id locale with the following details:
Channel - Release
Version - Latest Firefox Version (fx-22)
Platform - Windows
Locale - Indonesian (http://www.mozilla.org/id/firefox/new/)
Locale - Brazil (http://www.mozilla.org/pt-BR/firefox/new/)

Final builds should be similar to https://bugzilla.mozilla.org/show_bug.cgi?id=870464#c12
Assignee: nobody → nthomas
Is any QA needed here?
We will need QA to ensure:
1. The funnelcake build has the correct FirstRunPage (&sem=1 or equivalent as per nthomas)
2. User/s can successfully install the funnelcake build via stub.

-anurag
Thanks anurag, please let me know when this is ready for testing so we can qualify it.
We decided in the meeting today that this will be funnelcake23 rather than sem1, still id & pt-BR, win32 only. ?f=23 on first run etc.
Attached file Notes for creating repacked installers (obsolete) — 

    
    

    
  
Builds are staged in our 22.0 candidates directory for QA, and added to bouncer (will move them to firefox/releases/ after signoff). Bouncer will 404 if the lang is not id or pt-BR. I needed to add some 0-byte installer files in zh-TW to make bouncer's sentry happy.

ashughes, you can start with the stub installers at
 https://download.mozilla.org/?product=firefox-stub-f23&os=win&lang=id
 https://download.mozilla.org/?product=firefox-stub-f23&os=win&lang=pt-BR
You should get localised builds which are "Funnelcake July 2013" in the about dialog, query for updates on release-cck-mozilla23 (nb app.update.channel will still be release), and only differ from the vanilla builds by a distribution/distribution.ini file. If you want to try the full installers they're at 
 https://download.mozilla.org/?product=firefox-latest-f23&os=win&lang=id
 https://download.mozilla.org/?product=firefox-latest-f23&os=win&lang=pt-BR

anurag, I've checked the id build loads this on first run
 http://www.mozilla.org/id/firefox/22.0/firstrun/?f=23
and does this for blocklist:
https://addons.mozilla.org/blocklist/3/%7Bec8030f7-c20a-464f-9b0e-13a3a9e97384%7D/22.0/Firefox/20130618035212/WINNT_x86-msvc/id/release-cck-mozilla23/Windows_NT%205.1/mozilla23/1.0/invalid/invalid/0/
NB ..../id/release-cck-mozilla23/....
QA Contact: bhearsum → anthony.s.hughes

    
    

    
  
thx for the update nthomas.

-anurag

    
    

    
  
QA signs off these builds. For more details about what was tested please see our test plan[1]. Feel free to comment here or email me if you have any questions or concerns. Otherwise I think we are clear to push this live at Release Management's convenience.

1. https://wiki.mozilla.org/QA/Desktop_Firefox/Funnelcake/Firefox_22_SEM

    
    

    
  
Thanks for the QA.
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED

    
    

    
  
(In reply to Nick Thomas [:nthomas] from comment #7)
> Builds are staged in our 22.0 candidates directory for QA, and added to
> bouncer (will move them to firefox/releases/ after signoff). Bouncer will
> 404 if the lang is not id or pt-BR. I needed to add some 0-byte installer
> files in zh-TW to make bouncer's sentry happy.

I've moved the files into /pub/mozilla.org/firefox/releases/22.0-funnelcake23/ and updated bouncer. The bouncer links in comment #7 are unchanged.

    
    

    
  
Re-opening this bug because we need these same funnelcake 23 builds made for Mac and Linux because AdWords cannot filter by OS for desktop.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---

    
    

    
  
I'll start creating builds for the extra platforms straight away.

    
  
Blocks: 870469

    
    

    
  
Hi All,

I apologize for the confusion. My earlier confirmation was indeed a confirmation, but it turns out that this is something new in AdWords. 

JM

    
    

    
  
Those links are all working now. I've verified the mac builds start up OK and are customized as we expect.

    
    

    
  


  
Updated notes (+mac and linux32, signing server changes, handle different signing tokens and MOZ_SIGN_CMD for mac).

For completeness, the full set of bouncer configuration is:
* <product>               <os>
  <path>
* firefox-stub-f23        win
  /firefox/releases/22.0-funnelcake23/win32/:lang/Firefox%20Setup%20Stub%2022.0.exe
* firefox-latest-f23      win	
  /firefox/releases/22.0-funnelcake23/win32/:lang/Firefox%20Setup%2022.0.exe
* Firefox-22.0-f23        osx
  /firefox/releases/22.0-funnelcake23/mac/:lang/Firefox%2022.0.dmg
* Firefox-22.0-f23        linux
  /firefox/releases/22.0-funnelcake23/linux-i686/:lang/firefox-22.0.tar.bz2

The product names are case insensitive.

        Attachment #776925 -
        Attachment is obsolete: true

    
  
Status: REOPENED → RESOLVED
Closed: 11 years ago → 11 years ago
Resolution: --- → FIXED

    
    

    
  
(In reply to Nick Thomas [:nthomas] from comment #16)
> Those links are all working now. I've verified the mac builds start up OK
> and are customized as we expect.

Thanks, Nick! Such a quick response to this last minute emergency.

    
    

    
  
Please advise if you need verification of the Mac and Linux builds to the same degree we qualified win32 builds:
https://wiki.mozilla.org/QA/Desktop_Firefox/Funnelcake/Firefox_22_SEM

    
    

    
  
(In reply to Anthony Hughes, Mozilla QA (:ashughes) from comment #19)
> Please advise if you need verification of the Mac and Linux builds to the
> same degree we qualified win32 builds:
> https://wiki.mozilla.org/QA/Desktop_Firefox/Funnelcake/Firefox_22_SEM

The SEM campaign test is live now. Verifying the Mac/Linux build would be helpful if you think there is a risk.

    
    

    
  
I personally don't think it's at high risk, though I would defer to Nick Thomas before concluding as such.

    
    

    
  
I think we're OK with the spot checks of mac I did and miniscule usage of our linux builds.

    
    

    
  
Okay, thank you. Please add qawanted to this bug if more thorough testing is needed.

    
    

    
  
(In reply to a needinfo from Chris More [:cmore] at bug 870464 comment #16)
> Hey Nick: If you have seen in some of the email traffic, there was an
> incorrect character in the URL in AdWords and it caused invalid data to be
> collected. That was resolved a few days ago and data looks great now. The
> issue is that we don't have a full week of data and now Firefox 23 is coming
> out next Tuesday. Is it possible to Spin up Funnelcake with an ID of 24 that
> is built with Firefox 23 release? They would be the same as the funnelcake
> 23 you created, but make a funnelcake 24 using Firefox 23. This should be
> for windows, mac, and linux. How much time will you need and do you need to
> wait until Tuesday until Firefox 23 is released to do anything?

I see discussion has moved on, but yes we could make some more builds before Tuesday. The 23.0 builds are already created so we can repack a funnelcake build if needed.

A couple of things to consider if we do go ahead:
* unless there's some reason to go to funnelcake24 I would suggest reusing 23, just based on Firefox 23 instead. It's logically the same set of users so long as timing before/after release is not important, avoids having to recreate the stub installer (the slow part), and should be easier for metrics (f=23 everywhere etc)
* if we are literally talking about a day or two after Tuesday, perhaps it's not the end of the world to keep providing the Firefox 22 based funnelcake23 build. That would just be a little extra config in download.m.o, and users would get updated a few days later

    
    

    
  
Ok, so you could make another funnelcake 23 out of Firefox 23 instead of 22? That would probably be easiest on everyone since nothing would have to change except adding the product names in dmo. As for still using Firefox 22, that may not be a big deal since silent updates are usually throttled for a number of days.

The issue is that when the product_details is updated on mozilla.org tomorrow, the /new/ page will be pushing out firefox-23 and then we append -f23, it will throw a 404 given that the product is not in bouncer. I think it would still be easier to just create Firefox 23 builds of funnelcake and make them still funnelcake 23. Then adwords won't have to change and all of the reporting will still be tied to ?f=23.

    
    

    
  
nthomas: Given all of the above, how quickly could you have Firefox 23 funnelcake builds with ID of 23 created for Mac, Windows, and Linux available with the correct dmo product names?
Flags: needinfo?(nthomas)

    
    

    
  
I would guess two or three hours, assuming the requests comes during my work day.
Flags: needinfo?(nthomas)

    
    

    
  
(In reply to Nick Thomas [:nthomas] from comment #27)
> I would guess two or three hours, assuming the requests comes during my work
> day.

Ok, can this be the request? We need these as soon as possible so we don't have to pause the SEM test campaign too long. Let me know when the products are up and available to test. After mozilla.org's product details is updated tomorrow, we should be able to test if the ?f=23 links don't 404 on dmo. Thanks!

    
    

    
  
Anurag: nthomas is making the builds out of Firefox 23 and will be keeping the same funnelcake 23 ID. How long do you recommend waiting until after Firefox 23 that we can re-activate the campaign?
Flags: needinfo?(aphadke)

    
    

    
  
Assuming there's no chemspill, we should wait for a week after unthrottling. Assuming everything goes on track, we can resume on 8/19.
-anurag
Flags: needinfo?(aphadke)

    
    

    
  
Ok, the funnelcake23 based on Firefox 23 is ready.

Builds are in http://ftp.mozilla.org/pub/mozilla.org/firefox/releases/23.0-funnelcake23/ (ignore the zh-TW with empty files, that's just for bouncer's benefit). I've done some quick QA that windows and mac builds start OK, load the expected firstrun page, and query for updates as expected.

The following bouncer config is set up to handle linux & mac downloads:
* <product>               <os>
  <path>
* Firefox-23.0-f23        osx
  /firefox/releases/23.0-funnelcake23/mac/:lang/Firefox%2023.0.dmg
* Firefox-23.0-f23        linux
  /firefox/releases/23.0-funnelcake23/linux-i686/:lang/firefox-23.0.tar.bz2
The links in comment #14 work with product=firefox-23.0-f23 substituted.

I haven't changed these for windows yet
* firefox-stub-f23        win
  /firefox/releases/22.0-funnelcake23/win32/:lang/Firefox%20Setup%20Stub%2022.0.exe
* firefox-latest-f23      win	
  /firefox/releases/22.0-funnelcake23/win32/:lang/Firefox%20Setup%2022.0.exe
The locations should have s/22\.0/23.0/g done on them (two changes per location), but only after we ship 23.0.

    
    

    
  
I've tested these builds and they look fine to me except for one thing. The Stub Installer downloads the Funnelcake22 SEM builds. I receive an update to this Funnelcake23 build so I don't think it's a show stopper but may not be 100% expected.

Here's my AUS:SVC URL just in case:
> https://aus3.mozilla.org/update/3/Firefox/22.0/20130618035212/WINNT_x86-msvc/pt-BR/release-cck-mozilla23/Windows_NT%205.1.3.0%20(x86)/mozilla23/1.0/update.xml

    
    

    
  
(In reply to Anthony Hughes, Mozilla QA (:ashughes) from comment #33)
> I've tested these builds and they look fine to me except for one thing. The
> Stub Installer downloads the Funnelcake22 SEM builds. I receive an update to
> this Funnelcake23 build so I don't think it's a show stopper but may not be
> 100% expected.
> 
> Here's my AUS:SVC URL just in case:
> > https://aus3.mozilla.org/update/3/Firefox/22.0/20130618035212/WINNT_x86-msvc/pt-BR/release-cck-mozilla23/Windows_NT%205.1.3.0%20(x86)/mozilla23/1.0/update.xml

funnelcake22? We have used funnelcake23 this entire campaign and it first built with firefox 22 and now with firefox 23.

    
    

    
  
Maybe I'm using the terminology then.

When I download the Funnelcake23 stub installer from http://ftp.mozilla.org/pub/mozilla.org/firefox/releases/23.0-funnelcake23/ I get the following in the About dialog:

> Firefox
> 22.0
> Funnelcake July 2013
> mozilla23 - 1.0

Opening the about dialog triggers an update check which results in an update being found (see URL in comment 33). After restarting Firefox and opening the about dialog again I now see:

> Firefox
> 23.0
> Funnelcake July 2013
> mozilla23 - 1.0

The point of my comment wasn't terminology, it was the fact that the Stub Installer is serving the previous Funnelcake build, not this current one. If that's not a problem for you then please disregard.

    
    

    
  
I think ashughes meant Firefox 22.0 funnelcake23 in comment #333, that's what the AUS url says.  That the stub downloaded Firefox 22.0 is entirely expected given the last paragraph of comment #32. I've corrected that now, please retest windows.

    
    

    
  
Specifically, we now have
* <product>               <os>
  <path>
* firefox-stub-f23        win
  /firefox/releases/23.0-funnelcake23/win32/:lang/Firefox%20Setup%20Stub%2023.0.exe
* firefox-latest-f23      win	
  /firefox/releases/23.0-funnelcake23/win32/:lang/Firefox%20Setup%2023.0.exe

The stub installers in 23.0-funnelcake23/ were copied from 22.0-funnelcake23/.

    
    

    
  
I've almost messed myself up a few times when talking about funnelcake 23 built on either firefox 22 and 23.

Thanks, Nick and Anthony.

    
    

    
  
(In reply to Nick Thomas [:nthomas] from comment #36)
> I've corrected that now, please retest windows.

The funnelcake23 stub installer now gives me the correct funnelcake build. Calling this verified fixed.
Status: RESOLVED → VERIFIED
Product: mozilla.org → Release Engineering

    
    

    
  
Nick: We would like to wait for 23.0.1 to get out the door to make these funnelcake builds.

We need to make sure these products are up on dmo made from 23.0.1:

firefox-23.0.1-f23 (mac and linux)
firefox-stub-f23 (windows)

What is the earliest we can get these builds up on dmo?
Status: VERIFIED → RESOLVED
Closed: 11 years ago → 11 years ago

    
  
Status: RESOLVED → REOPENED
Flags: needinfo?(nthomas)
Resolution: FIXED → ---

    
    

    
  
I'll setup the funnelcake to be generated at the same time as the other repacks for 23.0.1 (rather than manual). Assuming we go to build on Wednesday I should have them up for you Thursday.
Flags: needinfo?(nthomas)

    
    

    
  


  
This is just committing the config used in earlier manual generation so that the release automation does the work for 23.0.1. Would get backed out again after that ships.

        Attachment #789817 -
        Flags: review?(aki)

    
    

    
  
Comment on attachment 789817 [details] [diff] [review]
[partner-repacks] Generate 23.0.1 funnelcake23 in automation

[14:24]	<aki>	i guess you don't need to differentiate between 23 and 23.0.1?
[14:25]	<aki>	or is that the dist_version

        Attachment #789817 -
        Flags: review?(aki) → review+

    
    

    
  
Comment on attachment 789817 [details] [diff] [review]
[partner-repacks] Generate 23.0.1 funnelcake23 in automation

https://hg.mozilla.org/build/partner-repacks/rev/a005a3606adb

Good point about the versioning, I changed it to 1.1 on landing.

Anurag, the older 22.0 builds are sending '1.0' for %DISTRIBUTION_VERSION% in the blocklist url (see below), while the 23.0.1 builds will send '1.1'.

https://addons.mozilla.org/blocklist/3/%APP_ID%/%APP_VERSION%/%PRODUCT%/%BUILD_ID%/%BUILD_TARGET%/%LOCALE%/%CHANNEL%/%OS_VERSION%/%DISTRIBUTION%/%DISTRIBUTION_VERSION%/%PING_COUNT%/%TOTAL_PING_COUNT%/%DAYS_SINCE_LAST_PING%/

        Attachment #789817 -
        Flags: checked-in+

    
    

    
  
(In reply to Nick Thomas [:nthomas] from comment #44)
> Comment on attachment 789817 [details] [diff] [review]
> [partner-repacks] Generate 23.0.1 funnelcake23 in automation
> 
> https://hg.mozilla.org/build/partner-repacks/rev/a005a3606adb
> 
> Good point about the versioning, I changed it to 1.1 on landing.
> 
> Anurag, the older 22.0 builds are sending '1.0' for %DISTRIBUTION_VERSION%
> in the blocklist url (see below), while the 23.0.1 builds will send '1.1'.
> 
> https://addons.mozilla.org/blocklist/3/%APP_ID%/%APP_VERSION%/%PRODUCT%/
> %BUILD_ID%/%BUILD_TARGET%/%LOCALE%/%CHANNEL%/%OS_VERSION%/%DISTRIBUTION%/
> %DISTRIBUTION_VERSION%/%PING_COUNT%/%TOTAL_PING_COUNT%/
> %DAYS_SINCE_LAST_PING%/

Anurag said that we won't be using the dist version in our reporting so whatever you change it to won't affect our reporting. Also, GA doesn't see it either. The unique identifier between the previous test and this upcoming one will be funnelcake build ID + Firefox version.

    
    

    
  
I spoke to Anurag before making the change, and he said a dist version difference would be useful if it was quick to do. It's only relevant to ADI in the long term, not hits on webpages.

    
    

    
  
just to close the loop, it's nice to be consistent by making the change to dist version, let's do it if the change isn't too difficult for nick.

    
    

    
  
Builds are created and basic QA done. eg verified first run pages and update query for mac id:
http://www.mozilla.org/id/firefox/23.0.1/firstrun/?f=23
https://aus3.mozilla.org/update/3/Firefox/23.0.1/20130814063812/Darwin_x86_64-gcc3-u-i386-x86_64/id/release-cck-mozilla23/Darwin%2012.4.0/mozilla23/1.1/update.xml

They're published into firefox/releases/ but hidden by an index.html. I haven't had a chance to do the bouncer config and testing yet.

    
    

    
  
I've done the basic smoketesting of these builds and everything looks fine.
 * Installer is localized
 * About dialog contains "Funnelcake July 2013 mozilla23 - 1.1" and is localized 
 * Update channel is "release-cck-mozilla23"
 * Browser is localized 
 * Browser passes basic smoketesting (browsing, bookmarks, history, downloads, etc)

    
    

    
  
Nick/Anthony: We are not going to do another SEM funnelcake test as we learned enough from the previous one and more data isn't "better data". Thanks for all the help!
Status: REOPENED → RESOLVED
Closed: 11 years ago → 11 years ago
Resolution: --- → FIXED

    
    

    
  
Comment on attachment 789817 [details] [diff] [review]
[partner-repacks] Generate 23.0.1 funnelcake23 in automation

Cleanup steps:
* backed out this patch (to avoid generating more funnelcake23 builds)
    https://hg.mozilla.org/build/partner-repacks/rev/da833632732d
* removed files firefox/releases/23.0.1-funnelcake23/ and 23.0-funnelcake23/ (never used)
* removed bouncer product Firefox-23.0-f23 (never used)
* set bouncer products firefox-latest-f23 and firefox-stub-f23 back to 22.0 locations, removed Check Now bit after sentry picked that up

        Attachment #789817 -
        Flags: checked-in+ → checked-in-

          You need to log in
          before you can comment on or make changes to this bug.
        






  

    
  







  

    

      
Attachment

      

      
      
      
      
    

    

      

        

          

            
General

            

              Creator: 



            

            
Created: 

            
Updated: 

            
Size: