755114 - mar.py throws exceptions on signed mars

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P3)

Description

[Nick Thomas [:nthomas] (UTC+12)]

Eg in http://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/13.0b3-candidates/build1/logs/release-mozilla-beta-antivirus-bm13-build1-build0.txt.gz

Exception IOError: 'File not open for writing' in <bound method BZ2MarFile.__del__ of <mar.BZ2MarFile instance at 0x18615a8>> ignored
no key specified to validate 1 signature
no key specified to validate 1 signature
START 13:12:53: /pub/mozilla.org/firefox/nightly/13.0b3-candidates/build1/update/linux-x86_64/en-US/firefox-13.0b2-13.0b3.partial.mar
/pub/mozilla.org/firefox/nightly/13.0b3-candidates/build1/update/linux-x86_64/en-US/firefox-13.0b2-13.0b3.partial.mar: OK
/tmp/tmpyxijuS/pub/mozilla.org/firefox/nightly/13.0b3-candidates/build1/update/linux-x86_64/en-US/firefox-13.0b2-13.0b3.partial.mar/platform.ini.patch: OK
/tmp/tmpyxijuS/pub/mozilla.org/firefox/nightly/13.0b3-candidates/build1/update/linux-x86_64/en-US/firefox-13.0b2-13.0b3.partial.mar/libnssckbi.so.patch: OK
/tmp/tmpyxijuS/pub/mozilla.org/firefox/nightly/13.0b3-candidates/build1/update/linux-x86_64/en-US/firefox-13.0b2-13.0b3.partial.mar/libnssutil3.so.patch: OK
/tmp/tmpyxijuS/pub/mozilla.org/firefox/nightly/13.0b3-candidates/build1/update/linux-x86_64/en-US/firefox-13.0b2-13.0b3.partial.mar/libnss3.so.patch: OK
/tmp/tmpyxijuS/pub/mozilla.org/firefox/nightly/13.0b3-candidates/build1/update/linux-x86_64/en-US/firefox-13.0b2-13.0b3.partial.mar/update.manifest: OK
/tmp/tmpyxijuS/pub/mozilla.org/firefox/nightly/13.0b3-candidates/build1/update/linux-x86_64/en-US/firefox-13.0b2-13.0b3.partial.mar/application.ini.patch: OK
/tmp/tmpyxijuS/pub/mozilla.org/firefox/nightly/13.0b3-candidates/build1/update/linux-x86_64/en-US/firefox-13.0b2-13.0b3.partial.mar/libxul.so.patch: OK
/tmp/tmpyxijuS/pub/mozilla.org/firefox/nightly/13.0b3-candidates/build1/update/linux-x86_64/en-US/firefox-13.0b2-13.0b3.partial.mar/updatev2.manifest: OK
/tmp/tmpyxijuS/pub/mozilla.org/firefox/nightly/13.0b3-candidates/build1/update/linux-x86_64/en-US/firefox-13.0b2-13.0b3.partial.mar/omni.ja.patch: OK
/tmp/tmpyxijuS/pub/mozilla.org/firefox/nightly/13.0b3-candidates/build1/update/linux-x86_64/en-US/firefox-13.0b2-13.0b3.partial.mar/libplc4.so.patch: OK
/tmp/tmpyxijuS/pub/mozilla.org/firefox/nightly/13.0b3-candidates/build1/update/linux-x86_64/en-US/firefox-13.0b2-13.0b3.partial.mar/libmozsqlite3.so.patch: OK
/tmp/tmpyxijuS/pub/mozilla.org/firefox/nightly/13.0b3-candidates/build1/update/linux-x86_64/en-US/firefox-13.0b2-13.0b3.partial.mar/updater.patch: OK
/tmp/tmpyxijuS/pub/mozilla.org/firefox/nightly/13.0b3-candidates/build1/update/linux-x86_64/en-US/firefox-13.0b2-13.0b3.partial.mar/precomplete: OK
/tmp/tmpyxijuS/pub/mozilla.org/firefox/nightly/13.0b3-candidates/build1/update/linux-x86_64/en-US/firefox-13.0b2-13.0b3.partial.mar/libnssdbm3.so.patch: OK
/tmp/tmpyxijuS/pub/mozilla.org/firefox/nightly/13.0b3-candidates/build1/update/linux-x86_64/en-US/firefox-13.0b2-13.0b3.partial.mar/libplds4.so.patch: OK
/tmp/tmpyxijuS/pub/mozilla.org/firefox/nightly/13.0b3-candidates/build1/update/linux-x86_64/en-US/firefox-13.0b2-13.0b3.partial.mar/libfreebl3.so.patch: OK
/tmp/tmpyxijuS/pub/mozilla.org/firefox/nightly/13.0b3-candidates/build1/update/linux-x86_64/en-US/firefox-13.0b2-13.0b3.partial.mar/libssl3.so.patch: OK
/tmp/tmpyxijuS/pub/mozilla.org/firefox/nightly/13.0b3-candidates/build1/update/linux-x86_64/en-US/firefox-13.0b2-13.0b3.partial.mar/libfreebl3.chk: OK
/tmp/tmpyxijuS/pub/mozilla.org/firefox/nightly/13.0b3-candidates/build1/update/linux-x86_64/en-US/firefox-13.0b2-13.0b3.partial.mar/libnssdbm3.chk: OK
/tmp/tmpyxijuS/pub/mozilla.org/firefox/nightly/13.0b3-candidates/build1/update/linux-x86_64/en-US/firefox-13.0b2-13.0b3.partial.mar/libsoftokn3.so.patch: OK
/tmp/tmpyxijuS/pub/mozilla.org/firefox/nightly/13.0b3-candidates/build1/update/linux-x86_64/en-US/firefox-13.0b2-13.0b3.partial.mar/libsmime3.so.patch: OK
/tmp/tmpyxijuS/pub/mozilla.org/firefox/nightly/13.0b3-candidates/build1/update/linux-x86_64/en-US/firefox-13.0b2-13.0b3.partial.mar/firefox.patch: OK
/tmp/tmpyxijuS/pub/mozilla.org/firefox/nightly/13.0b3-candidates/build1/update/linux-x86_64/en-US/firefox-13.0b2-13.0b3.partial.mar/libnspr4.so.patch: OK
/tmp/tmpyxijuS/pub/mozilla.org/firefox/nightly/13.0b3-candidates/build1/update/linux-x86_64/en-US/firefox-13.0b2-13.0b3.partial.mar/libsoftokn3.chk: OK
/tmp/tmpyxijuS/pub/mozilla.org/firefox/nightly/13.0b3-candidates/build1/update/linux-x86_64/en-US/firefox-13.0b2-13.0b3.partial.mar/firefox-bin.patch: OK
/tmp/tmpyxijuS/pub/mozilla.org/firefox/nightly/13.0b3-candidates/build1/update/linux-x86_64/en-US/firefox-13.0b2-13.0b3.partial.mar/distribution/extensions/[e-mail]: OK
END 13:12:55 (2 seconds elapsed): /pub/mozilla.org/firefox/nightly/13.0b3-candidates/build1/update/linux-x86_64/en-US/firefox-13.0b2-13.0b3.partial.mar

The IOError exception seems to be harmless but adds noise to the log. The 'no key specified to validate 1 signature' I don't know about, are we doing that validation elsewhere ?

Comment 1

[[e-mail] (:bhearsum)]

It sounds like mar.py might not be updated to handle signed MARs? Though, bug 699700 appears to have only landed in beta for 12 and I see the "no key specified" error in the 11.0b8 antivirus log (http://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/11.0b8-candidates/build1/logs/release-mozilla-beta-antivirus-build39.txt.gz), so that doesn't quite line up.

Comment 2

[[e-mail] (:bhearsum)]

Any idea about this, Chris?

Comment 3

[Chris AtLee [:catlee]]

the IOError comes from a __del__ method, which is harmless but annoying. Most likely it can be fixed by doing "if self.fileobj and self.mode == 'w'" here:
http://hg.mozilla.org/build/tools/file/default/buildfarm/utils/mar.py#l378

The "no key specified" warning is generated when it detects that it's working with a signed mar, but you haven't told it what key to use to validate it. I'm not 100% sure if this code was updated to match the latest signing format, so we can either update it and specify the public key to validate the mar, or rip out the signing code completely.

Comment 4

[[e-mail] (:bhearsum)]

Given that it's not hurting anything, I don't think this is high priority.

Comment 5

[[e-mail] (:bhearsum)]

Updating summary/component - this isn't release automation specific.

Comment 6

[Rail Aliiev [:rail]]

Fixed in upstream in https://github.com/rail/build-mar/commit/d719736f21e9da4511d4e50bc0ada557afcd5b46