Closed Bug 632627 Opened 9 years ago Closed 9 years ago

3.6.14build2 EUBallot build internal DLLs/EXEs don't match non-EUBallot versions


(Release Engineering :: General, defect)

Not set


(Not tracked)



(Reporter: armenzg, Unassigned)



(1 file)

For 3.6.14 build2 I had to trigger the EUBallot separately.
verify-signature failed.

I am not sure if this would be reached on the next release since I did not follow an standard order of operations but I will file it to determine if it requires further investigation.

When we reach "Verifying checksums of MARs and installers match" [1] we create a list of every unsigned file and then create a dictionary with the locale as the key.

The problem comes that we have partner-repacks and win32-EUBallot showing up for some locales with that logic.

Then we do a comparison that fails:
> if not sums_are_equal(packages[mar[0]], [packages[s] for s in packages.keys()]):

When I look at that function it seems that it disregards partner-repacks because they are not signed [3], and therefore, they don't take part of the comparison.

I get quite lost on the logic but I believe that this verification is not doing what it is supposed to when there are EUBallots.
I bet I am wrong here and there please feel free to correct me.

This can be reproduced with:
> ./ --product firefox unsigned-build2 signed-build2

>Comparing packages for locale: el
> - unsigned-build2/unsigned/partner-repacks/google/win32/el/Firefox Setup 3.6.14.exe
> - unsigned-build2/unsigned/update/win32/el/firefox-3.6.14.complete.mar
> - unsigned-build2/unsigned/win32/el/Firefox Setup 3.6.14.exe
> - unsigned-build2/unsigned/win32-EUBallot/el/Firefox Setup 3.6.14.exe
>helper.exe differs!
>Error: MARs and installer contents do not match

This does not happen when there is only partner repacks:
>Comparing packages for locale: ar
> - unsigned-build2/unsigned/update/win32/ar/firefox-3.6.14.complete.mar 
> - unsigned-build2/unsigned/partner-repacks/google/win32/ar/Firefox Setup 3.6.14.exe 
> - unsigned-build2/unsigned/win32/ar/Firefox Setup 3.6.14.exe

(In reply to comment #1)
> Possible dup of bug 632412.

Not a dupe.

This bug is tracking the acute issue of 3.6.14build2 EUBallot builds being invalid. I filed bug 632793 on the root issue.
No longer blocks: 627271
Summary: Verify signature fails with EUBallots → 3.6.14build2 EUBallot build internal DLLs/EXEs don't match non-EUBallot versions
Fixed these with the following:
cd ~/signing-work/firefox-3.6.14
rm -rf cache
# Create a new directory to plop signed ballot builds in, just in case we screw something up
rsync -av --exclude=win32-EUBallot signed-build2/ signed-build2-euballot-attempt2/
# Populate the cache by running w/ --keep-cache and --first-locale against the original directory. This causes it to unpack the first locale's build, cache it, and then exit because there's nothing to sign
for l in `ls unsigned-build2/unsigned/win32-EUBallot`; do ./ --keydir=${KEYDIR} -j4 --product Firefox -o signed-build2-euballot-attempt2 -p --first-locale ${l%\/} --keep-cache unsigned-build2/; done
# Now, run on the directory with only the EUBallot builds in it. This will have a 100% hit rate, because all of the files from these locales are in the cache.
./ --keydir=${KEYDIR} -j4 --product Firefox -o signed-build2-euballot-attempt2 -p --keep-cache unsigned-build2-euballot-fix/
# Generate detached sigs for them
cd signed-build2-euballot-attempt2
../signing/sign-files .
# Run verify-signatures to ensure everything is OK
./ --product firefox unsigned-build2 signed-build2-euballot-attempt2 > verify-signatures-build2-euballot.fix.log
# Upload!
rsync -e "ssh -i /home/cltsign/.ssh/ffxbld_dsa" -av signed-build2-euballot-attempt2/

Also added these in the 3.6.14 build notes.
Closed: 9 years ago
Resolution: --- → FIXED
Product: → Release Engineering
You need to log in before you can comment on or make changes to this bug.