Closed Bug 1001416 Opened 11 years ago Closed 10 years ago

Monitor aws_stop_idle.py hungs

Categories

(Release Engineering :: General, defect)

Product:
Release Engineering
Component:
General
Platform:
x86_64
Linux
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: rail, Assigned: rail)

References

Details

Attachments

(2 files)

nagios-puppet.diff
1.33 KB, patch
dustin
: review+
rail
: checked-in+
Details | Diff | Splinter Review
signal-cloud-tools.diff
2.09 KB, patch
Details | Diff | Splinter Review
aws_stop_idle.py was hung this morning what caused a lot of builders running. We should monitor aws_stop_idle.py and kill it/alert if it stays alive too long.
Attachment #8434944 - Flags: review?(dustin)
Comment on attachment 8434944 [details] [diff] [review] nagios-puppet.diff you do fine work!
Attachment #8434944 - Flags: review?(dustin) → review+
Comment on attachment 8434944 [details] [diff] [review] nagios-puppet.diff remote: https://hg.mozilla.org/build/puppet/rev/51607523ebff remote: https://hg.mozilla.org/build/puppet/rev/68b291568001 (In reply to Dustin J. Mitchell [:dustin] from comment #2) > you do fine work! Thank you! :)
Attachment #8434944 - Flags: checked-in+
Depends on: 1021003
Sounds like this has some impact on yesterday's tree closure - I just checked the log and the latest timestamp was 2014-06-18 13:33:30,084...
Blocks: 1027437
This should help with long running stop idle processes...
Attachment #8443426 - Flags: review?(catlee)
Comment on attachment 8443426 [details] [diff] [review] signal-cloud-tools.diff On IRC we decided to use something external to watch this.
Attachment #8443426 - Flags: review?(catlee)
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
See Also: → 1055600
Component: General Automation → General
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: