Closed Bug 1531239 Opened 6 years ago Closed 5 years ago

[MDC2] t-yosemite-r7-031 is running multiple tasks in the same time

Categories

(Infrastructure & Operations :: RelOps: Hardware, task)

task
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: riman, Assigned: dividehex)

References

Details

I've rebooted and re-imaged this machine but it still runs multiple tasks in the same time and resolves them as exception.

e.g
running test-macosx64-qr/opt-reftest-e10s-1 Uuz2KM7-TKacbU3F3bsJBQ a minute ago 2019-02-28T05:17:27.906Z -
running test-macosx64/debug-test-verify-e10s YTOHf8TQRfqSBzQRdEYts 4 minutes ago 2019-02-28T05:15:17.005Z -

There are no logs 11 hours ago. Link from Papertrail: https://papertrailapp.com/systems/t-yosemite-r7-031/events?focus=1037647191134593026&selected=1037647191134593026

The machine is quarantined.

I have reimaged the machine and removed it from quarantine.
We will keep on monitoring it to see if it will still run multiple tasks in the same time.

still in bad state, it's running multiple tasks at the same time. added it back to quarantine.
Dave, could you have a look please?

Flags: needinfo?(dhouse)
No longer blocks: t-yosemite-r7-310

I cycled power and the machine came up. The last log shows repeated crashes until the machine stopped:

[root@t-yosemite-r7-031.test.releng.mdc2.mozilla.com ~]# last -20
root      ttys000  rejh2.srv.releng.mdc1.mozilla.com Tue Apr  2 05:21   still logged in
cltbld    console                   Tue Apr  2 05:21   still logged in
reboot    ~                         Tue Apr  2 05:20 
cltbld    console                   Mon Mar 18 15:06 - crash (14+14:14)
reboot    ~                         Mon Mar 18 15:06 
cltbld    console                   Mon Mar 18 15:04 - crash  (00:01)
reboot    ~                         Mon Mar 18 15:04 
cltbld    console                   Mon Mar 18 13:27 - crash  (01:37)
reboot    ~                         Mon Mar 18 13:27 
cltbld    console                   Mon Mar 18 13:16 - crash  (00:10)
reboot    ~                         Mon Mar 18 13:16 
cltbld    console                   Mon Mar 18 13:15 - crash  (00:01)
reboot    ~                         Mon Mar 18 13:15 
cltbld    console                   Mon Mar 18 12:39 - crash  (00:35)
reboot    ~                         Mon Mar 18 12:39 
cltbld    console                   Mon Mar 18 11:46 - crash  (00:53)
reboot    ~                         Mon Mar 18 11:46 
cltbld    console                   Mon Mar 18 08:52 - crash  (02:54)
reboot    ~                         Mon Mar 18 08:52 
cltbld    console                   Mon Mar 18 08:21 - crash  (00:30)

I've powered it off for a few days to see if it comes up after that.

the machine is still down, following :

Pinging t-yosemite-r7-031.test.releng.mdc2.mozilla.com [10.51.56.190] with 32 bytes of data:
Request timed out.
Request timed out.
Request timed out.
Request timed out.

Ping statistics for 10.51.56.190:
Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

Dave, can you please power it on ? Then, lets check if the issue still persists..

Assignee: nobody → jwatkins
Status: NEW → RESOLVED
Closed: 5 years ago
Flags: needinfo?(dhouse)
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.