1519911 - Assess whether the moz.org change that introduced signing-up to a newsletter post Firefox download negatively impacts install/download ratio

Status: RESOLVED FIXED

(Data Science :: General, task, P1)

Description

[Romain Testard [:RT]]

Brief description of the request:
A change in the page being displayed post download of the installer was made on Sept 11th, showing a form to sign-up the newsletter.
Looking at downloads on google analytics VS installs (stub installer ping collected on DSMO-RS), a down trend starts after the change was pushed around September 11th
Concern: the change introduced got users to focus on the form rather than opening the installer after a download, lowering the install/download ratio.

Link to any assets:

• Analysis of download VS installs: https://docs.google.com/spreadsheets/d/1YfRspr9h7EeqlcGjXWp8MXvNLj84ue-TBF2TyCWzDYg/edit#gid=1232960230
• Moz.org change: https://github.com/mozilla/bedrock/pull/6092
• Moz.org asset: https://www.mozilla.org/en-US/firefox/download/thanks/

Is there a specific data scientist you would like or someone who has helped to triage this request: no

Comment 1

[Romain Testard [:RT]]

Hi Rebecca, we fear this is loosing us installs, could we please help get someone assigned to validate this is a real issue?

Comment 2

[Daniel Zielaski]

Hi all,

Please feel free to reach out to MWarther or me (DZielaski) directly w/ questions about this experimental change. We are in the process of comprehensively evaluating the results and will share more as soon as the analysis is available.

RWeiss, let's connect so as not to duplicate efforts.

DZ

Comment 3

[Tim Smith (inactive) 👨‍🔬 [:tdsmith]]

Hello; I'm picking this up for the data science team.

Daniel, thanks for reaching out. Do you have an estimate for when your analysis might be ready to share?

Comment 4

[Tim Smith (inactive) 👨‍🔬 [:tdsmith]]

I spoke with Romain and he gave me some additional context, and I've started familiarizing myself with the stub installer ping and the DSMO-RS dataset. I'm looking to see how I can get access to Google Analytics. I'm curious about whether there are seasonal effects driving fluctuations in install completion and whether we can detect and estimate the size in a change after Sep 11.

Comment 5

[Tim Smith (inactive) 👨‍🔬 [:tdsmith]]

Handing this back to Rebecca to discuss how to collaborate around this kind of question.

Comment 6

[Daniel Zielaski]

Rebecca - I think we have this covered. Might make sense for my team to take the lead and better socialize our analysis.

All - please reach out to DZielaski or SGhose for any questions.

DZ