974130 - [monitoring-story] count users who are failing to create or sign into fxa

Status: RESOLVED FIXED

(Cloud Services :: Server: Firefox Accounts, defect)

Description

[Edwin Wong [:edwong]]

1. As a SRE/QA manager, I want to know the number of users who are failing to create or sign into the service.
Acceptance criteria:
- SRE/QA manager has a dashboard showing count and percentage of failed create and sign ins for custom time spans that can be filtered by browser, OS, and region.
- An email alert is sent when create or sign in exceeds a set threshold.
Timeframe: Fx29 Deploy

Comment 1

[Edwin Wong [:edwong]]

B feature would be to count the types of errors: bad pw, account exists, etc...

Comment 2

[Katie Parlante]

With the exception of the region segmentation, we should be able to get this information from the request.summary log: https://github.com/mozilla/fxa-auth-server/pull/565#issuecomment-34934322

In particular, we want to look at create and login:
- POST /v1/account/create
- POST /v1/account/login

Look at errno: 0 is success and >0 for errors

For the user agent segmentations, we should align with https://bugzilla.mozilla.org/show_bug.cgi?id=969003, pulling out the fields:
- OS
- Browser
- Version

It doesn't make sense to have heka dashboard/views for all possible segmentations, the segmentations should be available for perusing in kibana.

Comment 3

[Edwin Wong [:edwong]]

this is fixed by: https://kibana.fxa.us-west-2.prod.mozaws.net/#/dashboard/elasticsearch/Endpoint%20Counts
I'll segment with other dashboards

Comment 4

[James Bonacci [:jbonacci]]

:edwong - if you have a growing list of specific kibana dashboards, let's document and post them somewhere...
I keep seeing references to various dashboards in IRC, but we should collect them somewhere to
1. make it easier for everyone to find and use
2. keep some consistent baseline/easy-to-use dashboards
3. work up a more general site for creating specific dashboard views

thanks for the one in Comment 3!

Comment 5

[Wesley Dawson [:whd]]

:jbonacci that is one of the intended purposes of this mana page, but I haven't been keeping up with curating it: https://mana.mozilla.org/wiki/display/SVCOPS/Firefox+Accounts+Logging+and+Monitoring+Services

We can also make the default dashboard at https://kibana.fxa.us-west-2.prod.mozaws.net/ link to all the other dashboards that have been created. Right now the landing page makes the instance look like it's unconfigured.

Comment 6

[Katie Parlante]

Agree with your points :jbonacci

(In reply to Wesley Dawson [:whd] from comment #5)
> We can also make the default dashboard at
> https://kibana.fxa.us-west-2.prod.mozaws.net/ link to all the other
> dashboards that have been created. Right now the landing page makes the
> instance look like it's unconfigured.

I'll take on this task: https://bugzilla.mozilla.org/show_bug.cgi?id=986612

Comment 7

[Edwin Wong [:edwong]]

We should use :whd's doc as the main page.  You can also copy and paste the table of saved views by click on the folder on top right.  Generally, if you make a dashboard you should export the scheme to a gist or something.

Good idea on default.