1356380 - Deploy the new main ping monitor

Status: RESOLVED FIXED

(Cloud Services Graveyard :: Metrics: Pipeline, defect, P1)

Description

[Mike Trinkala [:trink]]

1) remove moz_telemetry_doctype_error_ratio_monitor
2) remove moz_telemetry_doctype_sax_monitor
3) add main_ping_monitor with the following cfg (update alert recipient list)

filename = 'moz_telemetry_doctype_monitor.lua'
docType = "main"
message_matcher = 'Fields[docType] == "'.. docType .. '" && (Type == "telemetry" || Type == "telemetry.error")'
ticker_interval = 60
preserve_data = true

alert = {
  disabled = false,
  prefix = true,
  throttle = 90,
  modules = {
    email = {recipients = {"[e-mail]"}},
  },

  thresholds = { -- map of normalized channel names
    release = {
      volume = {
        inactivity_timeout = 5,
        percent_delta = 10, -- optional
      },
      size = {
        average = 15482,
        percent_delta = 10,
      },
      ingestion_error = {
        percent = 0.5,
      },
      duplicate = {
        percent = 2,
      },
      shape = {
        mindist = 0,
      },
      latency = {
        hours = 24,     -- 1 - 192
                        -- everything in the future is added to the first bucket
                        -- everything in the past is added to the last bucket
        percent = 33,   -- alerts if more than 33% of submission are greater than 24 hours latent
      }
    }
  }
}

Comment 1

[Thomas Huelbert]

Trinks update to come

Comment 2

[Mike Trinkala [:trink]]

Attachment: updated cfgs

Comment 3

[Wesley Dawson [:whd]]

Deployed: https://github.com/mozilla-services/puppet-config/pull/2568.