1245591 - Monitor coalesce service

Status: RESOLVED FIXED

(Infrastructure & Operations :: RelOps: General, task)

Description

[Jake Watkins [:dividehex]]

The coalescing service will need to be monitored and alert if it falls over.  My inclination is to simply add a https ping check to nagios.  But since we are moving to hosting webapps in external PaaS container products such as heroku, the options are wide open.

For instance, taskcluster uses uptimerobot.com with a simple static page that displays based on json output from uptimerobot.  https://deadmanssnitch.com/ has also been recommended.

The coalesce service itself is a noncritical component.  If it goes offine, tasks simply stop coalescing. So a monitoring service for coalescing doesn't need to be of high time resolution.

Comment 1

[Jake Watkins [:dividehex]]

I've added a host and service check this app.  Unfortunately, heroku doesn't allow icmp traffic to their lb endpoints so I had to change the host check and service ping check to a tcp port 443 check in addition to the https endpoint (app) response check of /v1/ping.  Checks are all green now.