Downtime notifications from nagiosbot need not have index numbers

RESOLVED FIXED

Status

mozilla.org Graveyard
Server Operations
--
minor
RESOLVED FIXED
5 years ago
3 years ago

People

(Reporter: ashish, Assigned: rtucker)

Tracking

Details

(Reporter)

Description

5 years ago
Downtime notifications from the bots aren't actionable. So they don't need an index number. Any action on a downtime start/end notification doesn't make sense, so the index introduces some ambiguity and can be done away with.
(Assignee)

Updated

5 years ago
Assignee: server-ops → rtucker
(Assignee)

Comment 1

5 years ago
I Committed revision 68159 which will now ignore service and host notifications for DOWNTIME[START|END] from being added to the indexed list.

This should resolve this, I will test/confirm and then close this out.
(Assignee)

Comment 2

5 years ago
I ended up fixing another issue here where the default color was always being set to blue instead of following the conditionals.

I've tested and confirmed the core of this bug, in that there are no indexes being assigned for the downtime notifications.
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
(Reporter)

Comment 3

5 years ago
Did this get reverted somehow?

< nagios-phx1> ashish: Downtime for celery1.addons.phx1.mozilla.com:Load scheduled for 8:00:00 18:27:46 < nagios-phx1> Mon 18:27:46 PDT [190] celery1.addons.phx1.mozilla.com:Load is DOWNTIMESTART (CRITICAL): CRITICAL - load average: 25.10, 23.41, 20.73 (http://m.allizom.org/Load)
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(Assignee)

Comment 4

5 years ago
Yes,
This ended up being a regression from some other changes. I've now set these so that they shouldn't get indexed anymore. This is a bit complicated due to regex searches and these messages also contain the indexable statuses like UP/DOWN/UNREACHABLE/OK/CRITICAL etc

I'm pretty confident that the code I just pushed should resolve this.

I'll keep an eye on it and close this out if it's resolved again.
(Assignee)

Comment 5

5 years ago
I ran a test by downtiming a service for 1 minute. No indexes were set to the alerts

I did the same for a host, and this checks out as well. No indexes set for the DOWNTIMESTART or DOWNTIMEEND notifications.
Status: REOPENED → RESOLVED
Last Resolved: 5 years ago5 years ago
Resolution: --- → FIXED
Product: mozilla.org → mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.