Closed Bug 592490 Opened 15 years ago Closed 15 years ago

mw32-ix-slave* slaves lost connection to master

Categories

(mozilla.org Graveyard :: Server Operations, task, P1)

Product:
mozilla.org Graveyard
Component:
Server Operations
Platform:
x86
macOS
Type:
task

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: ehsan.akhgari, Assigned: phong)

Details

I see 8 failures, across several mozilla-central and jaegermonkey. The common factor is win32 ix hardware, mw32-ix-slave*.
Assignee: nobody → nrthomas
Priority: -- → P1
Summary: mozilla-central jobs failing with type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'callRemote' → mw32-ix-slave* slaves lost connection to master
3 were at 15:44, 5 more at 14:35; two on pm03, 6 on pm01 - can't find anything wrong with the maters. Seems like this would be a switch or network fault if it affects such a specific group of machines.
From twistd.log on mw32-ix-slave19: 2010/08/31 15:34 -0700 [Broker,client] SlaveBuilder._ackFailed: SlaveBuilder.sendUpdate [repeats 6 more times] 2010/08/31 15:34 -0700 [Broker,client] lost remote [repeats many more times] 2010/08/31 15:34 -0700 [Broker,client] lost remote step 2010/08/31 15:34 -0700 [Broker,client] stopCommand: halting current command <buildbot.slave.commands.base.SlaveShellCommand instance at 0x014521C0> dmoore says the network between Castro and MPT is very busy at the moment, so the latency is causing the slave to drop the connection to the master, or we're dropping connections in some network hardware. --> IT. I have no problem at all with throttling the xserve archiving that is chewing bandwidth.
Assignee: nrthomas → server-ops
Component: Release Engineering → Server Operations
QA Contact: release → mrz
(In reply to comment #3) > I have no problem at all with throttling the xserve archiving that is chewing > bandwidth. OK, can you do that? If yes, then will you reopen the tree please?
Phong, can you restart the rsync with a cap of around 10 Mbps or so?
We've throttled down my copy. This should be fixed now.
Assignee: server-ops → phong
Status: NEW → RESOLVED
Closed: 15 years ago
Resolution: --- → FIXED
Thanks Phong. Tree reopened.
Product: mozilla.org → mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.