1498689 - Provide a way for partner builds (and funnelcakes) to use the stub installer

Status: RESOLVED FIXED

(Firefox :: Installer, enhancement)

Description

[Mike Kaply [:mkaply]]

Currently, whenever we do partner builds or funnelcakes, we have to use the full installer.

That has a couple disadvantages:

1. We lose the 64/32 bit Windows check.
2. Standard problem of abandonment on large download.

We should see if we can figure out a way to do stub installers for partner builds.

It would involve somehow binding the download URL to the stub externally so it could be done outside of a standard build process.

Comment 1

[Molly Howell (she/her) (mostly inactive)]

Would this require the ability to customize any branding strings/images as well?

Comment 2

[Mike Kaply [:mkaply]]

> Would this require the ability to customize any branding strings/images as well?

No. Our current full installer partner builds don't have any brand customizations.

Might be a nice to have thing at some point in the future, though.

Comment 3

[Molly Howell (she/her) (mostly inactive)]

Okay, so just the download URL, thanks.

Right now that gets baked into setup-stub.exe because it's an NSIS constant. But I'm not really sure how to meaningfully improve on that. Artifact builds already run the installer build, so a full build isn't necessary to change the URL. We could move it out into something like an INI file that gets packed up next to setup-stub.exe, but that doesn't really save a lot of steps; the packing process itself is nontrivial. The "dummy certificate" area that's used for attribution data is off-limits because this is not something that should be allowed to change after the package is signed. The only other place I can think of to put it is a string resource on the 7-zip stub, which is not exactly convenient (either to set or to read).

How far outside of a standard build process do you think we need to be able to get?

Comment 4

[Mike Kaply [:mkaply]]

> How far outside of a standard build process do you think we need to be able to get?

That's really up to releng. If they think we could rerun NSIS, that would be interesting.

Adding nthomas for his thoughts.

Comment 5

[Nick Thomas [:nthomas] (UTC+12)]

Storing the URL in an INI file inside the stub installer sounds like the most promising idea to me. We can likely use the existing 'mach repackage' code in-tree to handle calling NSIS. I think artifact builds are probably a bit heavier than doing that, and are currently only tier2 for linux on treeherder.

Experience says it's better for the vanilla builds to use the same mechanism as repacks, so any future bustage/regressions are visible asap instead of at some release in the future. This does make it more involved to transition vanilla builds because it requires some taskgraph work as well as NSIS. 


More details for future me - the mach repackage calls for full and stub installer differ by '--package-name firefox' and '--package ....fetches\target.zip' for the former, plus expected --tag, --setupexe, and --output. If we start passing a zip for the stub (which only contain the INI) it should work. The upstream build/partner repack job has to produce the target-stub.zip for this. The partner repackage jobs & script would need to be taught about stubs, probably dependent on a repack.cfg config value because external partners seem unlikely to need a stub. Downstream of repackage, the partner signing and beetmover tasks would also need to learn about stubs. Also fun, we build the vanilla stub in win32 builds, so win64=true && win32=false && winstub=true in a repack.cfg is a future footgun.

Relevant code:
@SubCommand('repackage', 'installer', https://dxr.mozilla.org/mozilla-central/source/python/mozbuild/mozbuild/mach_commands.py#2761
repackage_installer(): https://dxr.mozilla.org/mozilla-central/source/python/mozbuild/mozbuild/repackaging/installer.py#14

Comment 6

[Mike Kaply [:mkaply]]

So I guess there would be two parts here:

1. Getting installer to support an INI with download path.
2. Releng to do the repackaging

Comment 7

[Molly Howell (she/her) (mostly inactive)]

Okay, that was super helpful, thanks guys. It sounds like we're all agreed that the INI file is the way to go. That prevents having to run NSIS a second time and it should be a pretty simple patch on the installer side.

Just to make sure I have all the requirements straight: since you specifically mentioned the 32/64 bit selection feature, I assume this will need to support URL's for both of those. I think that's the only complication?

Comment 8

[Mike Kaply [:mkaply]]

> Just to make sure I have all the requirements straight: since you specifically mentioned the 32/64 bit selection feature, I assume this will need to support URL's for both of those. I think that's the only complication?

That's the only one I see.

The other interesting part is partner builds will always have to be public, but that's just life (and it's not your issue).

Comment 9

[Vasilisa]

Am I right, that this issue'll be released in the Firefox 64?

Comment 10

[Mike Kaply [:mkaply]]

(In reply to Vasilisa from comment #9)
> Am I right, that this issue'll be released in the Firefox 64?

No, there has been no work done here yet.

Comment 11

[Mike Kaply [:mkaply]]

See also bug 916666.

Comment 12

[Romain Testard [:RT]]

Nick, we're considering this as a Q2 activity with the 68 release train but there are releng dependencies, can you please confirm if this is something releng could support us on in Q2?

Comment 13

[Nick Thomas [:nthomas] (UTC+12)]

I'll talk to catlee and let you know.

Comment 14

[Nick Thomas [:nthomas] (UTC+12)]

Yes, we could work on this in Q2. I'll file a separate bug for RelEng code changes.

Comment 15

[Molly Howell (she/her) (mostly inactive)]

Attachment: Bug 1498689 - Support partner builds changing the stub installer download URL. r=agashlin!,mkaply!

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/1922e2efbb65
Support partner builds changing the stub installer download URL. r=agashlin,mkaply

Comment 17

[Stefan Hindli [:stefan_hindli]]

https://hg.mozilla.org/mozilla-central/rev/1922e2efbb65

Comment 18

[Hector Zhao [:hectorz]]

Hi,

IIUC, if the installation is somehow interrupted, the stub installer will still take the user to mozilla.org for a vanilla full installer.

Is this expected? Or should we make the URLManualDownload configurable, too?

Comment 19

[Mike Kaply [:mkaply]]

Hector:

Can you open a separate bug for that?

Comment 20

[Hector Zhao [:hectorz]]

(In reply to Mike Kaply [:mkaply] from comment #19)

Hector:

Can you open a separate bug for that?

Filed as bug 1558090.

Comment 21

[Romain Testard [:RT]]

Hi Nick, can you please update on status? We have only 2 days left to land to then allow for QA or this would be pushed to 70

Comment 22

[Nick Thomas [:nthomas] (UTC+12)]

This is almost ready to land and will make 69.

Comment 23

[Nick Thomas [:nthomas] (UTC+12)]

The release automation changes landed on central for 69, see bug 1538995 comment #24.