556222 - Firefox 3.0.19 Windows installers are not digitally-signed

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Kohei Yoshino]

A Firefox user has reported that Firefox 3.0.19 installers are not digitally-signed. I just confirmed the issue at least en-US and ja builds.

Comment 1

[Kohei Yoshino]

Do you have a QA step to verify signing? chktrust can do that...

Microsoft:
http://msdn.microsoft.com/en-us/library/z045761b%28VS.80%29.aspx

Mono: (I prefer this 'cause it works on Mac!)
https://developer.mozilla.org/en/Signing_an_executable_with_Authenticode#Verify

Comment 2

[Nick Thomas [:nthomas] (UTC+12)]

Further confirmed by the modification dates in firefox/releases/win32/ all being March 15 around 18:45 instead of in the last couple of days, and
 # stage
 $ cd ~cltbld/firefox-3.0.19/batch1
 $ diff -x *.asc -rq stage-unsigned/ stage-signed/
 $ 
All the Windows executables should differ at this point, so something went wrong on the signing box.

Any idea what happened here Lukas ? Did you possibly skip right over the 'Sign win32 EXEs' section in the wiki, distracted by the 'Skip this for XULRunner' message ? (now removed) That's the same place where the verification happens Kohei.

This is not great, but not a blocker as I don't think we link to these builds anywhere on mozilla.com. And all the internals to the installers are signed or QA would have run into problems running on Vista.

Comment 3

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to comment #2)
> Further confirmed by the modification dates in firefox/releases/win32/ 

Make that firefox/releases/3.0.19/win32/

Comment 4

[Carsten Book [:Tomcat]]

Attachment: screenshot of the user warning

Ok thats the screenshot what users will see (windows 7):

Only the setup exe is not signed. The Files after installation in the Firefox Directory like firefox.exe - updater.exe - crashreporter.exe etc have a digital signature and are signed as expected.

Beside this i did some testing and:
-> Installing works (after you accept this security warning)
-> Firefox 3.0.19 runs as expected 
-> Major Update to 3.6.2 works
-> Uninstalling works

Comment 5

[Mike Beltzner [:beltzner, not reading bugmail]]

(In reply to comment #2)
> This is not great, but not a blocker as I don't think we link to these builds
> anywhere on mozilla.com. And all the internals to the installers are signed or
> QA would have run into problems running on Vista.

Well, I'd agree that they don't require a respin, but can we get a signed version dropped into the FTP site as a replacement? We shouldn't go all archival with unsigned builds if we can avoid it.

And yes, I'd like to understand how this happened, and whether or not it's possible to insert an automated check to ensure that it doesn't happen again.

Comment 6

[Chris AtLee [:catlee]]

(In reply to comment #5)
> (In reply to comment #2)
> > This is not great, but not a blocker as I don't think we link to these builds
> > anywhere on mozilla.com. And all the internals to the installers are signed or
> > QA would have run into problems running on Vista.
> 
> Well, I'd agree that they don't require a respin, but can we get a signed
> version dropped into the FTP site as a replacement? We shouldn't go all
> archival with unsigned builds if we can avoid it.
> 
> And yes, I'd like to understand how this happened, and whether or not it's
> possible to insert an automated check to ensure that it doesn't happen again.

We do have automated checks in place for this for 3.5.x and onwards.

Comment 7

[[e-mail] (:bhearsum)]

Lukas, can you fix up the installers? Feel free to ping me if you need a hand.

Comment 8

[Al Billings [:abillings - ex-MoCo]]

Beltzner was rightly curious as to why this wasn't caught during normal release testing. I was curious as well as I was under the impression that we had this in the BFT tests, which we ran on Windows. I've checked for signed bits on more than one occasion during releases.

It turns out that we do have testcases for it (such as this one for 3.0: https://litmus.mozilla.org/show_test.cgi?id=4370) but they are in the FFTs now, not in the BFTs, which are what we normally run for releases. See this Litmus query: https://litmus.mozilla.org/show_test.cgi?searchType=fulltext&text_snippet=windows+builds+are+signed&relevance_threshold=1&match_limit=25.

My memory is either mistaken (which I doubt) or they were moved by someone during a litmus reorganization at some point. Someone with better Litmus access than me could probably say.

I'm going to move these into the smoketests unless some objects. It is important that we check these before releasing windows builds and that is the only way to guarantee it. If there is a programatic way to check, we should be able to have a Mozmill case for this for 3.5 and 3.6 (and beyond).

Comment 9

[Al Billings [:abillings - ex-MoCo]]

It has been pointed out that we only final sign the installer AFTER the beta period just before release. This could only be caught in the releasetest and release channels just before we ship. 

Build has automated checks in place for 3.5 and 3.6 per e-mail.

Comment 10

[Lukas Blakk [:lsblakk] use ?needinfo]

Signed win32 installers have now replaced the unsigned ones.  Waiting on QA to confirm before closing.

Comment 11

[Al Billings [:abillings - ex-MoCo]]

QA has looked at these. I see the signatures from 10:30ish this morning on all of the exes that I've examined (ru, en-US, de).