1522363 - Document stub installer pings on dtmo

Status: RESOLVED FIXED

(Data Science :: Documentation, task)

Description

[Tim Smith (inactive) 👨‍🔬 [:tdsmith]]

Stub installer pings presently aren't documented on dtmo.

The canonical documentation is in the tree: https://searchfox.org/mozilla-central/source/browser/installer/windows/docs/StubPing.rst

They are formed and sent from NSIS code (!) in the stub installer, in the SendPing subroutine: https://searchfox.org/mozilla-central/source/browser/installer/windows/nsis/stub.nsi

They are processed into Redshift by https://github.com/whd/dsmo_load (I think).

The Redshift tables are accessible from the DSMO-RS data source in STMO.

In Redshift, it looks like a table is created for each calendar day; download_stats and download_stats_year are views that union all (or a year's worth) of the per-day tables together, which makes e.g. SELECT * LIMIT 10 operations on them quite slow.

Comment 1

[Tim Smith (inactive) 👨‍🔬 [:tdsmith]]

Confirmed via Bug 1520794 that whd/dsmo_load is the repo that's used in production.

Comment 2

[Emily Thompson]

Started some comments in this doc (which for now only really amalgamates your comments above). Was thinking this could live under "Dataset Specific " on DTMO. Thoughts? Comments?

Comment 3

[Tim Smith (inactive) 👨‍🔬 [:tdsmith]]

I added comments with a couple of references to answer some of the questions you had; it's probably worth linking out to the Funnelcake wiki page. Otherwise, you know as much as I do 🙂

I think I would put this under 6.5, Dataset Reference - Other Datasets.

I think this would be a valuable contribution to dtmo as-is. Opening a DTMO PR and flagging robotblake and/or whd for review, and for assistance fleshing out anything you think is missing, is probably a good next step.