760566 - Timeout during 'uploading for signing' on OS X 10.7 builds

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Ed Morley [:emorley]]

OS X 10.7 mozilla-central build on 2012-06-01 07:31:08 PDT for push 12ab69851e05

slave: bld-lion-r5-054

{
2012-06-01 08:43:09,350 - c3f226ad4e9a4122ab0b8ff2bcc5c1da17bd7583: processing FirefoxNightly.app.tar.gz on https://mac-signing1.srv.releng.scl3.mozilla.com:9110
2012-06-01 08:43:09,356 - c3f226ad4e9a4122ab0b8ff2bcc5c1da17bd7583: connection error; trying again soon
2012-06-01 08:43:10,360 - c3f226ad4e9a4122ab0b8ff2bcc5c1da17bd7583: connection error; trying again soon
2012-06-01 08:43:11,364 - c3f226ad4e9a4122ab0b8ff2bcc5c1da17bd7583: connection error; trying again soon
2012-06-01 08:43:12,368 - c3f226ad4e9a4122ab0b8ff2bcc5c1da17bd7583: connection error; trying again soon
2012-06-01 08:43:13,372 - c3f226ad4e9a4122ab0b8ff2bcc5c1da17bd7583: connection error; trying again soon
2012-06-01 08:43:14,376 - c3f226ad4e9a4122ab0b8ff2bcc5c1da17bd7583: connection error; trying again soon
2012-06-01 08:43:15,379 - c3f226ad4e9a4122ab0b8ff2bcc5c1da17bd7583: connection error; trying again soon
2012-06-01 08:43:16,383 - c3f226ad4e9a4122ab0b8ff2bcc5c1da17bd7583: connection error; trying again soon
2012-06-01 08:43:17,387 - c3f226ad4e9a4122ab0b8ff2bcc5c1da17bd7583: connection error; trying again soon
2012-06-01 08:43:18,391 - c3f226ad4e9a4122ab0b8ff2bcc5c1da17bd7583: connection error; trying again soon
2012-06-01 08:43:19,395 - c3f226ad4e9a4122ab0b8ff2bcc5c1da17bd7583: connection error; trying again soon
2012-06-01 08:43:20,399 - c3f226ad4e9a4122ab0b8ff2bcc5c1da17bd7583: connection error; trying again soon
2012-06-01 08:43:21,403 - c3f226ad4e9a4122ab0b8ff2bcc5c1da17bd7583: connection error; trying again soon
2012-06-01 08:43:22,407 - c3f226ad4e9a4122ab0b8ff2bcc5c1da17bd7583: connection error; trying again soon
2012-06-01 08:43:23,411 - c3f226ad4e9a4122ab0b8ff2bcc5c1da17bd7583: connection error; trying again soon
2012-06-01 08:43:24,414 - c3f226ad4e9a4122ab0b8ff2bcc5c1da17bd7583: connection error; trying again soon
2012-06-01 08:43:25,418 - c3f226ad4e9a4122ab0b8ff2bcc5c1da17bd7583: connection error; trying again soon
2012-06-01 08:43:26,422 - c3f226ad4e9a4122ab0b8ff2bcc5c1da17bd7583: connection error; trying again soon
2012-06-01 08:43:27,426 - c3f226ad4e9a4122ab0b8ff2bcc5c1da17bd7583: connection error; trying again soon
2012-06-01 08:43:28,430 - c3f226ad4e9a4122ab0b8ff2bcc5c1da17bd7583: connection error; trying again soon
2012-06-01 08:43:29,431 - c3f226ad4e9a4122ab0b8ff2bcc5c1da17bd7583: giving up after 20 tries
2012-06-01 08:43:29,608 - c3f226ad4e9a4122ab0b8ff2bcc5c1da17bd7583: processing FirefoxNightly.app.tar.gz on https://mac-signing2.srv.releng.scl3.mozilla.com:9110
2012-06-01 08:45:04,091 - c3f226ad4e9a4122ab0b8ff2bcc5c1da17bd7583: uploading for signing

command timed out: 1200 seconds without output, attempting to kill
process killed by signal 9
program finished with exit code -1
elapsedTime=1340.460315
}

Comment 1

[Ed Morley [:emorley]]

https://tbpl.mozilla.org/php/getParsedLog.php?id=12277534&tree=Mozilla-Inbound
https://tbpl.mozilla.org/php/getParsedLog.php?id=12277196&tree=Mozilla-Inbound

Comment 2

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

this bug is now ~10hours old, and I see other, later osx10.7 builds posting on mozilla-inbound just fine. Is this still an issue?

Comment 3

[Chris AtLee [:catlee]]

it's dependent on load - mostly due to losing one of our mac signing machines earlier (bug 760077)

Comment 4

[[e-mail] (:bhearsum)]

We're in better shape now. I think the releases tipped us over the edge, and we had to bring up the only half-working mac-signing1 to compensate. We're bringing up two more in bug 759759 and bug 760077 tracks fixing mac-signing1. Once all that work is done we'll have 4 signing servers and be able to handle one going down easily. The acute issue is solved for now, so this is FIXED.