1194031 - Add an account age field to Token server application logs.

Status: RESOLVED FIXED

(Cloud Services Graveyard :: Server: Token, defect, P1)

Description

[Bob Micheletto [:bobm]]

Add an account age field to Token server application logs along with the unique, and anonymous account identifier.  The age should be measured in days since account creation.  This will allow us to easily calculate the account retention curve for the Sync 1.5 Service needed for Bug 1173407.

Comment 1

[Ryan Kelly [:rfkelly]]

/cc :ckarlof.  Chris, this is the bug I mentioned to you on IRC.  The tokenserver already logs an "activity event" for MAU purposes, if we add "account age" then we get retention almost for free.

Comment 2

[Ryan Kelly [:rfkelly]]

Attachment: ts-days-since-first-seen.patch

Here's a simple attempt at this.  It uses the "created_at" data from the tokenserver node assignments table to emits a "uid.first_seen_at" field in the request metrics.  This gives the millisecond timestamp at which we first saw the user, which some downstream filter can use to calculate the age of the account when this event occurred.

(I tried having the tokenserver emit "uid.days_since_first_seen" directly, but doing so assumes certain things about what counts as a day.  Does a new day start at midnight UTC for all users?  Emiting the timestamp directly will let us centralize that decision in one place, the downstream processing filter.)

Note the some events can cause the user to get a new node-assignment record with a new created_at timestamp, including:

* explicit node-reassignments on our side
* the user doing a password reset

These events would make them appear to start again at day zero.  I think that's an OK approximation for our first attempt at this.  If we like the result, we should revisit this and add an explicit "first_seen_at" field to the tokenserver db that properly persists this information.

The next step then, is that this value should be picked up by a HLLs-in-a-circular-buffer counting lua filter along the lines of [1], to calculate the D0/D1/D3/D7/D14/D30 active user counts.

Bob, do you have much experience with making such lua filters?  It would be great if you, I and Phil could collaborate on the details of making that go, because we want to apply a similar technique for Firefox Accounts as a whole.

[1] https://github.com/mozilla-services/puppet-config/blob/master/shared/modules/shared/files/hekad/lua_filters/sync_active_daily_users.lua

Comment 3

[Bob Micheletto [:bobm]]

Comment on attachment 8649700 [details] [diff] [review]
ts-days-since-first-seen.patch

This approach sounds good to me.  I like the idea of using this to calculate account age (24hr periods starting at timestamp X) instead of number of calendar days (UTC or otherwise) since timestamp X.  It seems closer to what we're trying to measure, since it's age from the point of view of the user.

Comment 4

[Bob Micheletto [:bobm]]

(In reply to Ryan Kelly [:rfkelly] from comment #2)

> Bob, do you have much experience with making such lua filters?  It would be
> great if you, I and Phil could collaborate on the details of making that go,
> because we want to apply a similar technique for Firefox Accounts as a whole.

Just enough to be dangerous.  We could pull in :whd or some other folks into the metrics team.

Comment 5

[Chris Karlof [:ckarlof]]

I like the direction.

I like the idea of adding this to all relying services (possibly including Pocket), so we can consolidate and break out activity/age information by relier in a central place.

Comment 6

[Ryan Kelly [:rfkelly]]

https://github.com/mozilla-services/tokenserver/commit/5c9f41b6b0d94d660f180e32328513026d716bcb