1013975 - please loan jmaher an entire IX machine (4) of linux64 talos testers

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: CIDuty, task)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

I am working on a new talos test that fails sometimes.  One theory I have is that we are power (or some other resource) starved which is causing the media tests to crash.  When running on try about 50% of the time we crash with no rhyme or reason.  Reproducing these crashes on loaners are difficult. 

So the linux64 ix machines are 4 to a box.  I want an entire set of those machines.

Comment 1

[Pete Moore [:pmoore][:pete]]

OK, I'm working on this

Comment 2

[Pete Moore [:pmoore][:pete]]

BTW, in the last couple of days, we've had several nagios alerts, such as those in bug 1013885.

From looking at inventory, it seems the talos-linux64-ix-* machines have their OOB switch assigned to devices such as these.

Therefore if we were having these switch issues, and these switches are connected to these talos linux 64 machines, this could match your theory about a resource loss starving the machines during testing.

Do you think it might be related?

Pete

Comment 3

[Pete Moore [:pmoore][:pete]]

Hey Joel,

Based on the discussions we had yesterday on IRC with dcurado etc, would you still like to go ahead with this loaner?

Thanks,
Pete

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

Pete,

Yes, I would still like to see this.  I was not sure how long we have had Juniper problems, and until I have this working, I need to keep debugging!

Comment 5

[Pete Moore [:pmoore][:pete]]

From IRC:

jmaher
15:50:56 pmoore: my tests are working much much better now
pmoore
15:51:06 jmaher: ah that's great!
jmaher
15:51:13 pmoore: so maybe we should cancel that loaner request