1531604 - Fix "position" field on Pocket story clicks in Discovery Stream variants

Status: VERIFIED FIXED

(Firefox :: New Tab Page, defect, P1)

Description

[Kirill Demtchouk]

In non-control variants of the pref-flip-study-activity-stream-pkt-newtab-beta66complexandperformance2-1529327 shield study, we are not recording the position of click events and impression events consistently.

This makes it impossible to calculate CTR and # of clicks for a given position in the new tab, which hinders Pocket's effort to add additional sponsored content to the Firefox New Tab page.

Impressions are currently assigned position in the following manner. Let us use the example of 2 rows of 4 Pocket stories:
0-1-2-3
4-5-6-7

Clicks are currently assigned position in the follow manner. Let us again use the same example of 2 rows of 4 Pocket stories:
0-1-2-3
0-1-2-3

Can you please change the "position" parameter on clicks to be consistent with impressions before 66 release?

Comment 1

[Ed Lee :Mardak]

[Tracking Requested - why for this release]: Correct numbers needed for experiment

Comment 3

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/activity-stream/pull/4823

Comment 4

[Liz Henry (:lizzard) (relman/hg->git project)]

Can you verify this once it lands in nightly? Thanks!

Comment 5

[Ed Lee :Mardak]

https://hg.mozilla.org/mozilla-central/rev/3d9f5f3d386b

Comment 6

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/activity-stream/pull/4826

Comment 7

[Scott [:thecount] Downe]

Attachment: Bug 1531604 - Fixing impression pos

Comment 8

[Scott [:thecount] Downe]

Comment on attachment 9048636 [details]
Bug 1531604 - Fixing impression pos

Beta/Release Uplift Approval Request

• Feature/Bug causing the regression: Bug 1522832
• User impact if declined: Experiment data issues not valuable.
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: Yes
• If yes, steps to reproduce: We need to verify this on the server, that the metrics position are coming in as expected in beta/nightly.
• List of other uplifts needed: None
• Risk to taking this patch: Medium
• Why is the change risky/not risky? (and alternatives if risky): Feature is preffed off, should be verified in nightly shortly.
• String changes made/needed: none

Comment 9

[Kirill Demtchouk]

Verified! Position is coming in correctly for impressions and clicks in latest Nightly build.

Comment 10

[Liz Henry (:lizzard) (relman/hg->git project)]

Comment on attachment 9048636 [details]
Bug 1531604 - Fixing impression pos

Uplift all the things!

Comment 11

[Bogdan Maris, Desktop Test Engineering]

Based on the above comments this was verified in Nightly (Fx67) and waiting for the fix to hit Beta (Fx66). Changing the flags accordingly.

Comment 12

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/releases/mozilla-beta/rev/37e058d76eca

Comment 13

[Bogdan Maris, Desktop Test Engineering]

(In reply to Kirill Demtchouk from comment #9)

Verified! Position is coming in correctly for impressions and clicks in latest Nightly build.

Could you do the same verification for Beta if you don't mind?

Comment 14

[Kirill Demtchouk]

Verified that this is working as expected in latest Beta!