1301820 - Create writeup explaining telemetry date/times, delays, etc.

Status: RESOLVED FIXED

(Data Platform and Tools :: Documentation and Knowledge Repo (RTMO), defect, P3)

Description

[Katie Parlante]

Cover:
- submission date vs user reported activity date
- fields associated with above
- what we know about clock skew (may require analysis prep)
- what we know about submission lag (may require analysis prep)
- current mau/dau strategy: https://bugzilla.mozilla.org/show_bug.cgi?id=1240849
- any guidelines for analysis

Plan is to have this for the telemetry info meeting ~9/22

Comment 1

[Mark Reid [:mreid]]

Chris HC has added some relevant analysis recently:

http://reports.telemetry.mozilla.org/post/projects/ping_delays.kp
http://reports.telemetry.mozilla.org/post/projects/crash_ping_delays.kp

Comment 2

[Ryan Harter [:harter]]

Hey Mark - does it look like you'll be able to handle this in Q2 or should we move this to P3?

Comment 3

[Chris H-C :chutten]

How much of this was covered by similar topics in https://github.com/mozilla/firefox-data-docs/blob/master/concepts/analysis_gotchas.md

Comment 4

[Mark Reid [:mreid]]

I don't think I'll get to this in Q2, bumping priority accordingly. Thanks.

Comment 5

[Mark Reid [:mreid]]

(In reply to Chris H-C :chutten from comment #3)
> How much of this was covered by similar topics in
> https://github.com/mozilla/firefox-data-docs/blob/master/concepts/
> analysis_gotchas.md

There's definitely some overlap with information in there, in particular I think it fully covers these items:
- fields associated with [submission date vs user reported activity date]
- any guidelines for analysis

It also partially covers what we know about submission lag.

We also have some ongoing monitoring of submission latency by document type via the CEP[1].

I'd still like to see the distribution of clock skew.


[1] For example, for 'main' pings: https://pipeline-cep.prod.mozaws.net/dashboard_output/graphs/analysis.moz_telemetry_doctype_monitor_main.latency.html

Comment 6

[Mark Reid [:mreid]]

I'm calling this done per :chutten's investigations.