1526267 - Quantify the number of full installer downloads happening on our servers

Status: RESOLVED FIXED

(Data Science :: Investigation, task)

Description

[Romain Testard [:RT]]

Brief description of the request:
We recently shipped full installer telemetry (see https://sql.telemetry.mozilla.org/dashboard/full-installer-telemetry) and we were surprised to see that full install numbers are equivalent to stub install numbers.
We assume this may relate to either (1) third party websites pointing downloads to our servers or (2) third party websites copying and serving the full installer from their own CDN (the build_id of the installs are right and the fact that we get full installer pings tell us that they must be using our installer directly).
What is needed: provider the number of daily downloads of the full installer from our servers (to include downloads from https://www.mozilla.org/en-US/firefox/all/ , https://www.mozilla.org/en-US/firefox/organizations/all/ and any other known location where third parties can access our full installers (currently assuming the only such location is archive.mozilla.org).
After discussion with oremj we have access logs with the fields listed in this format: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/AccessLogs.html, which we can query against using a few different tools.

Link to any assets:
Full installer telemetry dashboard: https://sql.telemetry.mozilla.org/dashboard/full-installer-telemetry

Is there a specific data scientist you would like or someone who has helped to triage this request:
No

Comment 1

[Jess Mokrzecki [:jmok]]

Hi Romain, Could you give us your desired timeline for this request?

Comment 2

[Romain Testard [:RT]]

This is not critical - 6 weeks timeline would be OK

FYI oremj confirmed that the logs are in sitting in S3 in their raw format and can be queried with Athena.
These logs only cover the cloudfront part of downloads, Akamai download volumes will be provided separately through the Akamai interface.

Comment 3

[Jesse McCrosky]

Hi, I'm doing some background work on new-user measures and would like to understand this better. Can someone provide more context?

Why do we need to look at CloudFront and Akamai? Is the full installer ping coming through CDNs?

And can you give some context on why your proposed reasons explain why stub numbers would match full install numbers? "(1) third party websites pointing downloads to our servers or (2) third party websites copying and serving the full installer from their own CDN"

Thanks!

Comment 4

[Corey Dow-Hygelund [:ccd]]

Waiting for access to Cloudfront and Akamai installer logs.

Comment 5

[Corey Dow-Hygelund [:ccd]]

Product Delivery Mana page: https://mana.mozilla.org/wiki/display/SVCOPS/Product+Delivery

Comment 6

[Corey Dow-Hygelund [:ccd]]

As per Jeremy Orem, the Akamai full-installer downloads are initiated from the stub installer. The initial full installer requests go through Cloudfront. Awaiting access to Cloudfront logs from Athena.

Comment 7

[Corey Dow-Hygelund [:ccd]]

Further documentation: https://docs.google.com/document/d/1yVyM5Q9MrjzYkPKccYfjv0l60AL93uiirXeDXoggeSw/edit

Comment 8

[Corey Dow-Hygelund [:ccd]]

Initial dashboard implemented from Cloudfront Installer logs: https://sql.telemetry.mozilla.org/dashboard/cloundfront-downloads-full-installers

It appears that the full installer downloads accountable on Cloudfront are ~20% what is observed in full installer ping.

Comment 9

[Corey Dow-Hygelund [:ccd]]

The report regarding the rationale behind the queries used in the dashboard: https://docs.google.com/document/d/1pjFqAA_NkbkWlGBgRZpdFxYvJHmeD-X8vLnzaWsMsHg/edit?usp=sharing

Comment 10

[Corey Dow-Hygelund [:ccd]]

Work complete for request. Additional bug filed for next steps in the determination of the source of excessive installers.