error uploading file for signing: File too large uploading Thunderbird Daily builds

RESOLVED FIXED

Status

Release Engineering
General
--
blocker
RESOLVED FIXED
6 years ago
a month ago

People

(Reporter: standard8, Assigned: kmoir)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Reporter)

Description

6 years ago
Since sometime yesterday, we've been getting this on Mac builds:

2012-10-08 02:43:47,443 - 3395fa97544068444c86a330eb5bf9407569e3e8: processing Daily.app.tar.gz on https://mac-signing2.srv.releng.scl3.mozilla.com:9110
2012-10-08 02:43:47,463 - 3395fa97544068444c86a330eb5bf9407569e3e8: uploading for signing
2012-10-08 02:43:54,232 - 3395fa97544068444c86a330eb5bf9407569e3e8: error uploading file for signing: File too large
2012-10-08 02:43:55,234 - 3395fa97544068444c86a330eb5bf9407569e3e8: giving up after 20 tries

It might be good to also extend the "File too large" message with size & limit.
Found in triage.
Component: Release Engineering → Release Engineering: Automation (General)
QA Contact: catlee
(Reporter)

Comment 2

6 years ago
This is currently stopping us getting Mac Nightlies out on Daily & Earlybird. Sorry I wasn't explicit about that before.
Severity: normal → blocker
Someone needs to verify that this isn't a packaging problem or something else artificially inflating the package. If that's not the case, then this value needs to be bumped up: https://hg.mozilla.org/build/puppet-manifests/file/default/modules/signingserver/templates/signing.ini.erb#l15

master-puppet1 will need to be updated with the change, and the signing servers will automatically reload themselves for it when they pick it up.
(Reporter)

Comment 4

6 years ago
(In reply to Ben Hearsum [:bhearsum] from comment #3)
> Someone needs to verify that this isn't a packaging problem or something
> else artificially inflating the package.

According to FTP:

Firefox:     2012-10-07 = 46583640, 2012-10-08 = 48476361.
Thunderbird: 2012-10-07 = 48079959, 2012-10-08 = ?

According to my calculations, the next day should be around 50000000 for Thunderbird, which should still be a little shy of 52428800.

Can someone log onto a builder and confirm the size of the package?

It should be around build/objdir/mozilla/dist/ iirc.
find . -name Daily.app.tar.gz -ls
237209661   106592 -rw-rw-rw-    1 cltbld           admin            54573314 10 Oct 07:39 ./objdir-tb/i386/mozilla/dist/universal/thunderbird/Daily.app.tar.gz
(Assignee)

Comment 6

6 years ago
It looks like the XUL file alone is 144MB unzipped, is that expected?

bld-lion-r5-058:MacOS cltbld$ pwd
/builds/slave/tb-c-cen-osx64/build/objdir-tb/i386/mozilla/dist/universal/thunderbird/Daily.app/Contents/MacOS
bld-lion-r5-058:thunderbird cltbld$ cd Daily.app/Contents/MacOS/
bld-lion-r5-058:MacOS cltbld$ du -sh XUL
144M	XUL
(Reporter)

Comment 7

6 years ago
Quite possibly, we've don't strip symbols in nightlies, and the last Daily I have had a XUL of 139804480.
(Reporter)

Comment 8

6 years ago
Ok, I think this is down to bug 790517 which added a whole load of webrtc stuff - about 10M of object files on my local 64-bit only directory.

I suspect, multiply that by two and compress into XUL, and you get the increase that pushed us over the limit.

So I think we can safely bump the limit knowing this is an increase in code size.
Blocks: 790517
(Assignee)

Updated

6 years ago
Assignee: nobody → kmoir
We ran into this briefly on Saturday on one Try push we did of our staging patch-queue for the Webrtc/signaling landing, and we had a bunch of traffic with Callek and others trying to figure out what the limit was and what our options were.  But a later Try (with a few corrections for a bug in a Makefile) went green, as did all the other builds and our landing.

If there isn't a real reason for the limit of ~52MB, I agree with bumping it.
(Assignee)

Comment 10

6 years ago
Created attachment 670394 [details] [diff] [review]
patch
Attachment #670394 - Flags: review?(rail)
Attachment #670394 - Flags: review?(rail) → review+
(Assignee)

Updated

6 years ago
Attachment #670394 - Flags: checked-in+
(Assignee)

Comment 11

6 years ago
deployed to puppet masters
(Reporter)

Comment 12

6 years ago
This is now fixed, thanks.
Status: NEW → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
Component: General Automation → General
Product: Release Engineering → Release Engineering
You need to log in before you can comment on or make changes to this bug.