1089252 - docker-worker: Shutdown invoked but node still appears running on AWS?

Status: RESOLVED WORKSFORME

(Taskcluster :: Workers, defect)

Description

[James Lal [:lightsofapollo]]

https://papertrailapp.com/systems/55744643/events

This one is really strange... AWS Shows the node as running but our node is clearly idle in the logs.

Comment 1

[Jonas Finnemann Jensen (:jonasfj)]

When you see strange things like this you should try to SSH into the machine, ping it... Figure out if truly is still alive :)
AWS is eventually consistent and may report machines as running after they are shutdown.
(I've heard stories about things like this that was reported incorrectly for days/weeks)

Of course, it still more likely that it didn't shutdown correctly :)

Comment 2

[Greg Arndt [:garndt]]

I have not seen this happen in many, many months.  I think some of our work for updating the upstart scripts might have taken care of this.  Closing for now.  Reopen if still an issue.