972105 - [Metrics Story] FxAccounts created total

Status: RESOLVED FIXED

(Cloud Services :: Server: Firefox Accounts, defect)

Description

[Katie Parlante]

User Story:

As VP of Services, I want to know the total number of FxAccounts (tracked over time), so that I can show the company how close we are to meeting our goals.

Acceptance criteria:

mmayo (and all of Mozilla) have access to a web page with a graph of # FxAccounts/time (or even just one big number). The goal should be available on the web page as well (on the graph, for example).

Comment 1

[Katie Parlante]

Attachment: FxAOMTM-12.jpg

Comment 2

[Katie Parlante]

This metric is our One Metric That Matters for 2014; we need a simple, static page for it (not a link to kibana). We can then do things like display it on flat panel screens hanging in Mozilla offices. :)

I've attached a sketch. Crystal, John, Ryan: feel free to improve upon it!

Comment 3

[Katie Parlante]

Related to ADU: https://bugzilla.mozilla.org/show_bug.cgi?id=968983, another contender for "OMTM". We'll instrument both and can wire up either one (or both) to a static page.

Comment 4

[Katie Parlante]

Bug for logging the right data in fxa-auth-server: https://github.com/mozilla/fxa-auth-server/issues/569

Comment 5

[Benson Wong [:mostlygeek]]

This stat is probably best slurped out of the database once a day and logged into elastic search as a daily count document (schema TBD). Then with kibana a gauge and a line chart (or whatever) can be created to show: 

* total accounts (gauge) 
* growth of accounts (chart) 

The big thing to figure out is *where* to put this in a reliable fashion. A pattern that we've adopted is to create an admin/misc server. There will be one of these with the proper access that does things like run the cronjob to make this happen. 

Another option is to a bit more involved and that includes: 

* cronjobs run on all the app servers
* we leverage Amazon's SQS so only one server gets the stats jobs at a time
* the job is processed and a new item is added to the SQS queue

I prefer the admin server approach as the implementation is a lot simpler and requires little coordination of network services.

Comment 6

[Katie Parlante]

(In reply to Ben Wong [:mostlygeek] from comment #5)
> This stat is probably best slurped out of the database once a day and logged
> into elastic search as a daily count document (schema TBD). 

Agreed. I was assuming fxa-auth-server would do the slurping, but your proposal (admin server) sounds better. I'll close the auth server bug, where should I open a bug for the admin server?

> Then with kibana a gauge and a line chart (or whatever) can be created to show: 

For this particular use case, we're going to want a simple web display in addition to kibana. (We can do a first pass in kibana though). I'm assuming all metrics "dashboards", kibana or otherwise, will pull from the same elasticsearch db.

Comment 7

[Benson Wong [:mostlygeek]]

(In reply to Katie Parlante from comment #6)

> where should I open a bug for the admin server?

https://github.com/mozilla-services/puppet-config/issues/141

> 
> > Then with kibana a gauge and a line chart (or whatever) can be created to show: 
> 
> For this particular use case, we're going to want a simple web display in
> addition to kibana. (We can do a first pass in kibana though). I'm assuming
> all metrics "dashboards", kibana or otherwise, will pull from the same
> elasticsearch db.

ES seems like the right place for it right now. These metrics are just JSON docs. 
Can you design the JSON doc schema for this? 

Querying the data should be fairly trivial.

Comment 8

[Mike Trinkala [:trink]]

https://heka.fxa.us-west-2.dev.mozaws.net/#sandboxes/FxaSandboxManager-FxaAccountCreated/outputs/FxaSandboxManager-FxaAccountCreated.DailyRunningTotal.cbuf

And all the other derivatives (the user agents will be cleaned up shortly https://github.com/mozilla-services/heka/issues/722)
https://heka.fxa.us-west-2.dev.mozaws.net/#plugins/filters/FxaSandboxManager-FxaAccountCreated

Comment 9

[Katie Parlante]

Good enough for go live, but still need to automate this.

Comment 10

[Mike Trinkala [:trink]]

Heka no longer keeps a daily running total, it just monitors the current day for anomalies.  This was part of the cleanup to only produce Heka output that is actually used for monitoring (this is a guideline, not a rule, but it does prevent a lot of output bloat)

Comment 11

[Katie Parlante]

Story accomplished: https://metrics.fxa.us-west-2.prod.mozaws.net/accounts-dashboard/

Git repo: https://github.com/mozilla/accounts-dashboard