1573317 - [Pocket] Newtab Experiments - Impact of Personalization on SPOC Performance

Status: RESOLVED FIXED

(Firefox :: New Tab Page, task, P1)

Description

[Tawanda Kanhema]

Brief Description of the request (required):

Test impact of personalization on SPOC performance in Discovery Stream variants.

• Personalization ON - Discovery Stream
• Personalization OFF - Discovery Stream

Business purpose for this request (required):

Determine whether personalization should be turned ON/OFF for Discovery Stream.

Requested timelines for the request or how this fits into roadmaps or critical decisions (required):

Firefox 69. Personalization is currently turned off on Sponsored Content items. Measuring its impact on performance will inform decisions on whether to keep it off or turn it on in Discovery Stream for Firefox 70.

Links to any assets (e.g Start of a PHD, BRD; any document that helps describe the project):

Experiment Design Doc - https://docs.google.com/document/d/1tbpgKgKCMbgWKHUsCg82Y_KZceNF7fFarPjm7IXdZ-Q/edit

Experimenter Ticket - https://experimenter.services.mozilla.com/experiments/activity-stream-pktnewtabh22019-70-personalization-test/

Name of Data Scientist (If Applicable):

Please note if it is found that not enough information has been given this will delay the triage of this request.

Comment 1

[Su-Young Hong]

hey Tawanda, not sure what the needinfo is for, going to clear.

Comment 2

[Jess Mokrzecki [:jmok]]

The request within this ticket is not within the scope of work that the Data Science team. We do not have access to the data so we cannot run the analysis. As such I will be closing this ticket.

Comment 3

[Jessilyn Davis]

Thanks Jess! Rerouting this to our team to see if we can solve for it internally.

Comment 4

[Tawanda Kanhema]

:kirill Experimenter ticket here for your review

Comment 5

[Kirill Demtchouk]

Reviewed and looks good to me!

Comment 6

[Jessilyn Davis]

[Tracking Requested - why for this release]: Personalization is currently turned off on Sponsored Content items in Fx 69. We'd like to measure the impact of when personalization is on/off to decide whether to keep it off or turn it on in Discovery Stream for Firefox 71.

Comment 7

[Experimenter Bot]

[Experiment]activity-stream-pktnewtabH22019-70-personalization-test status has been changed to: Live
            url:https://experimenter.services.mozilla.com/experiments/activity-stream-pktnewtabh22019-70-personalization-test/

Comment 8

[Kirill Demtchouk]

Experiment results are here:
https://sql.telemetry.mozilla.org/dashboard/pocket-new-tab-personalization-test---pocket-metrics

Closing this ticket out.