1535632 - Document behavior/shortcomings of (sub)session lengths and active_ticks

Status: NEW

(Toolkit :: Telemetry, enhancement, P2)

Description

[Jan-Erik Rediger [:janerik]]

We do know about some shortcomings in how we measure (sub)session lengths.
We see some wildly inaccurate values in our data, some of which might be explained by the fact that on some platforms our time measurement includes sleep times and on others it does not, some of which we can't explain.
We also have active_ticks as a measurement of actual user activity intervals, which might or might not be more reliable.

For proper analysis and reference we should have a single place where this behavior/shortcomings of either method are clearly explained.
This should probably go into docs.telemetry.mozilla.org

Comment 1

[Jan-Erik Rediger [:janerik]]

Subsession lengths: measured as monotonic time between two generated pings (subsessions)
Session lengths: measured as monotonic time since start of session
Active ticks: count of the number of five-second intervals (‘ticks’) the user was considered ‘active’ (sending UI events to the window)

Known issues with (sub)session lengths:

• Might include sleep time (Windows) or not (Mac, Linux)
  • (Windows) Subsessions might be longer than 24 hours when machine was asleep before ping was generated
    • Theoretically no limit on sleep time
• Huge values are seen. Unclear if another issue.

Comment 2

[Tim Smith (inactive) 👨‍🔬 [:tdsmith]]

The Smoot v1 report covers some of this ground in the chapter on engagement metrics: https://docs.google.com/document/d/1zgv_Ebdp1-AGIqBE-PlkQi7F3MPujexknLlqhM0cIdQ/edit

We'll intend to publish that more widely sometime early in H2 and could reshape it into a longer-lived metrics-behavior reference site; I'm not sure if that fully addresses this need or not.