Closed Bug 1538995 Opened 8 months ago Closed 5 months ago

Support for stub installers which download customized builds

Categories

(Release Engineering :: Release Automation: Other, enhancement)

enhancement
Not set

Tracking

(firefox-esr60 unaffected, firefox-esr68 fixed, firefox68 fixed, firefox69 fixed)

RESOLVED FIXED
Tracking Status
firefox-esr60 --- unaffected
firefox-esr68 --- fixed
firefox68 --- fixed
firefox69 --- fixed

People

(Reporter: nthomas, Assigned: nthomas)

References

Details

Attachments

(13 files)

58 bytes, text/x-github-pull-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
58 bytes, text/x-github-pull-request
Details | Review
55 bytes, text/x-github-pull-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
129.55 KB, image/png
Details
55 bytes, text/x-github-pull-request
Details | Review
58 bytes, text/x-github-pull-request
Details | Review

Bug 1498689 will add support to set/override download urls so that we can point at funnelcake or partner builds. This bug is for

  • adjusting tc-partner-repacks.py to support new config param(s) to create customized stub
  • download setup-stub.exe from parent vanilla build
  • create a stub-target.zip containing an ini file, which specifies the download urls (details TBD)
  • add those both as artifacts
  • adjust partner repackaging to create a stub installer, passing zip to the call
  • adjust partner repackaging signing to sign new stub
  • adjust partner beetmover to publish new stub
  • staging releases

Nick:

All the changes are in for this in code. How's this looking for Q2? (we have QA signed up for testing)

It's on my list for this quarter, and I've started working on it. Should be no problem.

Assignee: nobody → nthomas

Nick,

We're getting pinged on this by QA. Any updates?

Thanks!

Between other projects and time away I've not made as much progress as I'd like. It is one of my two highest priorities though, so hopefully more news in the next week.

For info we agreed to delay this bug and overall stub installer for partners feature QA to 69.

Hi Nick, any chance of an update on this - QA is now looking at covering as part of 69 QA cycle.

Flags: needinfo?(nthomas)

Work is progressing well. I spoke to mkaply recently and he'd like to investigate hosting the full installers ourselves, which is some extra work setting up download.m.o and moving files around on archive.m.o. I'm aiming to have patches up and test stubs for QA within the next week.

Flags: needinfo?(nthomas)

Pay attention to the new repack_stub_installer param, and fix config parsing to split on the first =.

On win32, where repack_stub_installer is enabled, extend the inputs to the partner repackage task so that it
donwloads the upstream artifacts and creates the stub installer.

Depends on D34945

Extends the upstream artifacts list to include the customized stub installer.

Depends on D34946

Appends the customized stub installer and it's gpg signature to the list of artifacts to beetmove to
candidates.

Depends on D34948

Adds a new release-partner-repack-bouncer-sub-firefox task to add the partner products. This is forked from the main bouncer submission because it diverges sufficiently that this is a cleaner approach. The get_partners_to_be_published helper is used by aliases too.

Depends on D34950

This passes partner data on to the beetmover scriptworker, so it can publish enabled partners. There is beetmover change at https://github.com/mozilla-releng/beetmoverscript/pull/227 to complete this support.

Depends on D35481

Pass partner aliases down to the bouncer scriptworker so it can update those too. Matching change to bouncerscript at https://github.com/mozilla-releng/bouncerscript/pull/57.

Depends on D35482

Attached image 64bit screenshot

We're ready for some QA testing.

Test method (based on a staging release):

  • Install certificates so the dep-signed files are accepted by Windows (once on each test machine/VM)
    certutil.exe -addstore Root MozRoot.cer
    certutil.exe -addstore Root MozFakeCA_2017-10-13.cer
    certutil.exe -addstore Root MozFakeCA.cer
  • Download and run the stub installer: en-US, ja

You should get a 69.0b13 build (staging!), with buildID 20190620214253, based on try rev 9cb6ab5. It should have a partner customization in Help > About Firefox as shown in the screenshot (see also distribution/distribution.ini in the install directory).

Testing on Windows 32 and Windows 64 would be appreciated, and ARM64 if available. The only expected change to the stub is the download URL used in the background. The full installers are available for win32, win64, and win64-aarch64, if those help for comparison.

Known issue - if the download fails we'll open the www.m.o page for vanilla Firefox. Bug 1498689 adds a way to override this, but we don't have anywhere to use with that.

Flags: needinfo?(oana.botisan)
Flags: needinfo?(anca.soncutean)

Hi Nick,

We managed to perform the investigation by using the steps provided by you in comment 18, using Windows 32, 64 and ARM64. Things look good, as expected. One question though, if running stub on a x32 architecture machine, the installation shouldn't ended up in Program Files (x86), instead of Program Files?

Flags: needinfo?(oana.botisan)
Flags: needinfo?(anca.soncutean)
Flags: needinfo?(nthomas)

Thanks for the testing. I think for 32-bit there is only Program Files; it's only 64-bit where the 32-bit apps end up in Program Files (x86).

Flags: needinfo?(nthomas)
See Also: → 1497097
Pushed by nthomas@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/20de5a5e5c55
support repacking stub installers for partners, r=aki
https://hg.mozilla.org/integration/autoland/rev/93f413cf1457
adjust repackage jobs to create partner stub installers, r=aki
https://hg.mozilla.org/integration/autoland/rev/8b151e2777fb
adjust repackage-signing jobs to sign partner stub installers, r=aki
https://hg.mozilla.org/integration/autoland/rev/aea787a5a4c6
adjust beetmover to copy partner stub installers into candidates dir, r=aki
https://hg.mozilla.org/integration/autoland/rev/fe29e66f80e2
submit partner products to bouncer when we'll host the files, r=aki
https://hg.mozilla.org/integration/autoland/rev/167db72d1987
pass list of partners to beetmover for pushing to releases/partners/, r=aki
https://hg.mozilla.org/integration/autoland/rev/37f02f5b2ad9
update bouncer aliases for partner products when shipping, r=aki

(In reply to Nick Thomas [:nthomas] (UTC+12) from comment #21)

Version bump for bouncerscript pending at https://github.com/mozilla-releng/bouncerscript/pull/60

https://github.com/mozilla-releng/bouncerscript/commit/ed5ff4e01bde358b044ea8475415104455478e2f

Puppet change at https://github.com/mozilla-releng/build-puppet/pull/529

https://github.com/mozilla-releng/build-puppet/commit/797fe7bb35b7594fddccea6c5778cd1a92e98301

That's everything landed.

It's not enabled until we hit 69.0b8, and we change one or more repack configurations by

  • adding this to the repack.cfg:
repack_stub_installer=true
publish_to_releases=true
  • adding a stub/partner.ini in the sub-config containing
[DownloadURL]
X86="https://download.mozilla.org/?product=%BOUNCER_PRODUCT%&os=win&lang=%LOCALE%"
AMD64="https://download.mozilla.org/?product=%BOUNCER_PRODUCT%&os=win64&lang=%LOCALE%"
AArch64="https://download.mozilla.org/?product=%BOUNCER_PRODUCT%&os=win64-aarch64&lang=%LOCALE%"

The AArch64 (ARM64) can be left out if we're not building that platform.

Thanks to tomprince.

See Also: → 1570078

Looks like we need to uplift at least some of this to m-r and m-esr68?

See Also: → 1573642
Flags: needinfo?(nthomas)

Looks like esr68 is unaffected. It's only partner-repacks on m-r. Eme-free is unaffected.
Because of this, I'm thinking we can:

  1. continue to push+ship 68.0.2 build 1
  2. uplift the appropriate patches to m-r
  3. promote-firefox-partners to re-promote just partner-repacks

(2)+(3) can be done at their own cadence, independent of (1).

See Also: → 1583685
You need to log in before you can comment on or make changes to this bug.