662973 - Nagios can and should check https://support.mozilla.com/services/monitor

Status: RESOLVED FIXED

(mozilla.org Graveyard :: Server Operations, task)

Description

[James Socol [:jsocol, :james]]

We monitor several site services and status info and display it on a pretty page:

https://support.mozilla.com/services/monitor

Most importantly, if everything is fine, we return HTTP 200, but if anything fails, it returns a 500. That's specifically for nagios' benefit, and now that we're showing green across the board, we should have nagios pay attention.

Comment 1

[Shyam Mani [:fox2mike]]

oremj does this for addons, so might want to touch base with his stuff to see if we can just use that. (it's already setup and live on nagios)

Comment 2

[Rob Tucker [:rtucker]]

Are you looking to monitor each of the supportX.webapp.phx1 webheads or just a general check that looks at the url through the load balancer?

Comment 3

[James Socol [:jsocol, :james]]

Most things on that are cluster-wide services, but the "Libraries" section checks locally-installed software, so if it's not a ton more work to monitor them individually, we should.

Comment 4

[Rob Tucker [:rtucker]]

James,
You'll need to provide to me a list of the systems and the respective urls. I get a 403 when trying to access the url directly

Comment 5

[Corey Shields [:cshields]]

this page will need to be allowed on the http :80 side if we are going to bypass the lb and hit each node directly.

Comment 6

[James Socol [:jsocol, :james]]

It doesn't require HTTPS. You'll probably need to send a "Host: support.mozilla.com" header to them, though, then the URL should just be

support{1-5}.webapp.phx1.mozilla.com/services/monitor

Comment 7

[Rob Tucker [:rtucker]]

James,
That's exactly what I needed. This will be completed momentarily.

Comment 8

[Rob Tucker [:rtucker]]

The checks are in place across all 5 webheads and are now green. Closing.

Comment 9

[James Socol [:jsocol, :james]]

Great, thanks Rob!