Closed Bug 1001416 Opened 10 years ago Closed 10 years ago

Monitor aws_stop_idle.py hungs

Categories

(Release Engineering :: General, defect)

x86_64
Linux
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: rail, Assigned: rail)

References

Details

Attachments

(2 files)

aws_stop_idle.py was hung this morning what caused a lot of builders running. We should monitor aws_stop_idle.py and kill it/alert if it stays alive too long.
Attachment #8434944 - Flags: review?(dustin)
Comment on attachment 8434944 [details] [diff] [review]
nagios-puppet.diff

you do fine work!
Attachment #8434944 - Flags: review?(dustin) → review+
Comment on attachment 8434944 [details] [diff] [review]
nagios-puppet.diff

remote:   https://hg.mozilla.org/build/puppet/rev/51607523ebff
remote:   https://hg.mozilla.org/build/puppet/rev/68b291568001


(In reply to Dustin J. Mitchell [:dustin] from comment #2)
> you do fine work!

Thank you! :)
Attachment #8434944 - Flags: checked-in+
Depends on: 1021003
Sounds like this has some impact on yesterday's tree closure - I just checked the log and the latest timestamp was 2014-06-18 13:33:30,084...
Blocks: 1027437
This should help with long running stop idle processes...
Attachment #8443426 - Flags: review?(catlee)
Comment on attachment 8443426 [details] [diff] [review]
signal-cloud-tools.diff

On IRC we decided to use something external to watch this.
Attachment #8443426 - Flags: review?(catlee)
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
See Also: → 1055600
Component: General Automation → General
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: