Closed Bug 696777 Opened 10 years ago Closed 10 years ago

Use pre-built updater.exe

Categories

(Firefox Build System :: General, defect, P1)

defect

Tracking

(Not tracked)

RESOLVED WONTFIX

People

(Reporter: catlee, Assigned: khuey)

References

Details

Attachments

(3 files)

In order to get bug 481815 shipped on nightly before we have fully automated signed nightly builds (bug 663055), we can use a pre-built signed version of updater.exe.

The build system needs to be instructed where to find this binary and include it before, or as part of, 'make package'. I think it's preferable if updater.exe still gets built from source so we can catch build problems, etc.

I propose setting an environment variable, e.g. MOZ_SIGNED_UPDATER_URL="https://.../updater.exe". If set, 'make package' could download this file (maybe a checksum too) and include it in the package instead of the locally built version. We would only set this variable for nightly builds.
Priority: -- → P1
11:05 < khuey> bhearsum: bbondy: assign it to me, I'll do it this week
Assignee: nobody → khuey
You left out the part about the maple cream cookies!
We need a new bribe/reward input element in the bugzilla header.
Attached patch First passSplinter Review
This verifies that the binary is signed, but we really want to verify the binary is signed by us.  bbondy has some code in Bug 481815 that will prove quite useful for that.
Attachment #570271 - Flags: review?(ted.mielczarek)
Comment on attachment 570271 [details] [diff] [review]
First pass

Review of attachment 570271 [details] [diff] [review]:
-----------------------------------------------------------------

::: toolkit/mozapps/installer/packager.mk
@@ +710,5 @@
>  endif # MOZ_PKG_MANIFEST
>  endif # UNIVERSAL_BINARY
> +ifdef MOCO_SIGNED_UPDATER_EXE
> +	@echo "Replacing in-tree updater.exe with external signed binary from $(MOCO_SIGNED_UPDATER_EXE)"
> +	(cd $(DIST)/$(MOZ_PKG_DIR) && $(WGET) -nv -N  "$(MOCO_SIGNED_UPDATER_EXE)")

It would probably be good to use -O updater.exe here, to make sure it ends up with the right name.
I tried the patch in my elm branch but updater.exe isn't signed.

bhearsum: 
Regarding your suggestion, would it look like htis?
 +	(cd $(DIST)/$(MOZ_PKG_DIR) && $(WGET) -nv -N  "$(MOCO_SIGNED_UPDATER_EXE)" -O updater.exe)
(In reply to Brian R. Bondy [:bbondy] from comment #6)
> I tried the patch in my elm branch but updater.exe isn't signed.
> 
> bhearsum: 
> Regarding your suggestion, would it look like htis?
>  +	(cd $(DIST)/$(MOZ_PKG_DIR) && $(WGET) -nv -N 
> "$(MOCO_SIGNED_UPDATER_EXE)" -O updater.exe)

Probably need to put the '-O' before $(MOCO_SIGNED_UPDATER_EXE). It shouldn't matter here, though, because the file on the server is also called 'updater.exe'.

In any case, the problem seems to be that MOCO_SIGNED_UPDATER_EXE isn't in the environment, which means there's a problem with my patch from bug 698837.
Comment on attachment 570271 [details] [diff] [review]
First pass

Review of attachment 570271 [details] [diff] [review]:
-----------------------------------------------------------------

::: toolkit/mozapps/installer/packager.mk
@@ +708,5 @@
>  	@($(XPIDL_LINK) $(DIST)/xpt/$(MOZ_PKG_APPNAME).xpt $(DIST)/$(STAGEPATH)$(MOZ_PKG_DIR)/components/*.xpt && rm -f $(DIST)/$(STAGEPATH)$(MOZ_PKG_DIR)/components/*.xpt && cp $(DIST)/xpt/$(MOZ_PKG_APPNAME).xpt $(DIST)/$(STAGEPATH)$(MOZ_PKG_DIR)/components && printf "interfaces $(MOZ_PKG_APPNAME).xpt\n" >$(DIST)/$(STAGEPATH)$(MOZ_PKG_DIR)/components/interfaces.manifest) || echo No *.xpt files found in: $(DIST)/$(STAGEPATH)$(MOZ_PKG_DIR)/components/.  Continuing...
>  endif # DMG
>  endif # MOZ_PKG_MANIFEST
>  endif # UNIVERSAL_BINARY
> +ifdef MOCO_SIGNED_UPDATER_EXE

Does this really need to have MOCO_ in the name? Maybe just call it PRE_SIGNED_UPDATER_EXE?

@@ +713,5 @@
> +	@echo "Replacing in-tree updater.exe with external signed binary from $(MOCO_SIGNED_UPDATER_EXE)"
> +	(cd $(DIST)/$(MOZ_PKG_DIR) && $(WGET) -nv -N  "$(MOCO_SIGNED_UPDATER_EXE)")
> +# Verify that the updater is signed.
> +	signtool verify -pa -v $(DIST)/$(MOZ_PKG_DIR)/updater.exe
> +  $(shell false)

I assume this is leftover from something else? Seems like a no-op...
Attachment #570271 - Flags: review?(ted.mielczarek) → review+
I think this is causing elm to not build. (My bad for trying it early but just letting you know)

http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/elm-win32/1320721571/elm-win32-nightly-build5.txt.gz

/e/builds/moz2_slave/elm-w32-ntly/build/toolkit/mozapps/installer/packager.mk:719: *** commands commence before first target.  Stop.
make[6]: *** [locales_export] Error 2
make[5]: *** [export_tier_app] Error 2
make[4]: *** [tier_app] Error 2
make[3]: *** [default] Error 2
make[2]: Leaving directory `/e/builds/moz2_slave/elm-w32-ntly/build'
make[2]: *** [realbuild] Error 2
make[1]: Leaving directory `/e/builds/moz2_slave/elm-w32-ntly/build'
make[1]: *** [profiledbuild] Error 2
make: *** [build] Error 2
program finished with exit code 2
elapsedTime=4378.985000
Oh, you want to remove the $(shell false) line.  That was debugging stuff ....
ok thanks will try
Getting a new error after fixing that $(shell false) line:
http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/elm-win32/1320874142/

elm change log:
http://hg.mozilla.org/projects/elm/


printf "manifest components/interfaces.manifest\nmanifest components/components.manifest\nmanifest chrome/nonlocalized.manifest\nmanifest chrome/localized.manifest\n" > ../../dist/firefox//chrome.manifest
Warning: trying to link manifests in missing directory '../../dist/manifests/xpcom/components'
Warning: trying to link manifests in missing directory '../../dist/manifests/xpcom/chrome'
Replacing in-tree updater.exe with external signed binary from https://build.mozilla.org/signed-binaries/updater.exe
(cd ../../dist/firefox && wget -nv -N  "https://build.mozilla.org/signed-binaries/updater.exe")
make[4]: Leaving directory `/e/builds/moz2_slave/elm-w32-ntly/build/obj-firefox/browser/installer'
make[3]: Leaving directory `/e/builds/moz2_slave/elm-w32-ntly/build/obj-firefox/browser/installer'
make[2]: Leaving directory `/e/builds/moz2_slave/elm-w32-ntly/build/obj-firefox'
make[1]: Leaving directory `/e/builds/moz2_slave/elm-w32-ntly/build'
Authorization failed.
make[4]: *** [stage-package] Error 1
make[3]: *** [all] Error 2
make[2]: *** [package] Error 2
make[1]: *** [profiledbuild] Error 2
make: *** [build] Error 2
program finished with exit code 2
elapsedTime=4396.234000
That sounds like a releng problem to me.
Thought so too, bhearsum is CC'ed :)
I'll try to get that fixed up today.
Actually, catlee already filed bug 701266 on it.
Work around bug 701266 not being done by abusing the talos directory.
Attachment #573672 - Flags: review?(nrthomas)
Attachment #573672 - Flags: review?(nrthomas) → review+
(From bug 701266:)
(In reply to Brian R. Bondy [:bbondy] from comment #7)
> Builds are succeeding but updater.exe isn't signed in the elm build:
> 
> Latest build and logs: Nov 11th
> http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/elm-win32/
> 1321018207/
> 
> History:
> http://hg.mozilla.org/projects/elm/
> 
> This might be a comment for khuey's bug, not sure.

And from the log, showing the code is getting updater.exe and verifying it:

Replacing in-tree updater.exe with external signed binary from https://build.mozilla.org/clobberer/updater.exe
(cd ../../dist/firefox && wget -nv -N  "https://build.mozilla.org/clobberer/updater.exe")
signtool verify -pa -v ../../dist/firefox/updater.exe
07:19:40 URL:https://build.mozilla.org/clobberer/updater.exe [269288/269288] -> "updater.exe" [1]


Verifying: ../../dist/firefox/updater.exe

Hash of file (sha1): 1D6A9E29C0914F06D6DAAF5F29840AB41F5D3F2F

Signing Certificate Chain:
    Issued to: Thawte Premium Server CA

    Issued by: Thawte Premium Server CA

    Expires:   Thu Dec 31 15:59:59 2020

    SHA1 hash: 627F8D7827656399D27D7F9044C9FEB3F33EFA9A


        Issued to: thawte Primary Root CA

        Issued by: Thawte Premium Server CA

        Expires:   Wed Dec 30 15:59:59 2020

        SHA1 hash: 1FA490D1D4957942CD23545F6E823D0000796EA2


            Issued to: Thawte Code Signing CA - G2

            Issued by: thawte Primary Root CA

            Expires:   Fri Feb 07 15:59:59 2020

            SHA1 hash: 808D62642B7D1C4A9A83FD667F7A2A9D243FB1C7


                Issued to: Mozilla Corporation

                Issued by: Thawte Code Signing CA - G2

                Expires:   Wed Oct 31 15:59:59 2012

                SHA1 hash: 3C9F63F58DAF9499122604006D9E78F0AE67C801


The signature is timestamped: Tue Nov 01 10:54:42 2011

Timestamp Verified by:
    Issued to: Thawte Timestamping CA

    Issued by: Thawte Timestamping CA

    Expires:   Thu Dec 31 15:59:59 2020

    SHA1 hash: BE36A4562FB2EE05DBB3D32323ADF445084ED656


        Issued to: VeriSign Time Stamping Services CA

        Issued by: Thawte Timestamping CA

        Expires:   Tue Dec 03 15:59:59 2013

        SHA1 hash: F46AC0C6EFBB8C6A14F55F09E2D37DF4C0DE012D


            Issued to: VeriSign Time Stamping Services Signer - G2

            Issued by: VeriSign Time Stamping Services CA

            Expires:   Thu Jun 14 15:59:59 2012

            SHA1 hash: ADA8AAA643FF7DC38DD40FA4C97AD559FF4846DE


Successfully verified: ../../dist/firefox/updater.exe


Number of files successfully Verified: 1

Number of warnings: 0

Number of errors: 0
Just verified with latest build and it is being signed on elm, sorry about that.  Looks great!
Attachment #577616 - Flags: review?(catlee) → review+
Comment on attachment 577616 [details] [diff] [review]
use final location for updater.exe

After the next reconfig we'll be pulling from http://runtime-binaries.pvt.build.mozilla.org/updater.exe and I'll delete the one in clobberer/
Attachment #577616 - Flags: checkin+
I think this can be marked as fixed or obsolete since you guys have automatic signing implemented now.  I don't think this will land anywhere else.
Glad you found it useful during development.

(I still get my cookies, right?)
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → WONTFIX
Of course :) And yes it was very useful, thanks to everyone involved in this task.
Was support for this backed out of m-c?
I don't think it ever made it to m-c, but we should back it out of elm (and ash and oak if it landed there).
I reverted the patch from khuey on elm.
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.