1754957 - Users are unenrolled from the A/A Picture in Picture study after PiP has been used

Status: VERIFIED FIXED

(Toolkit :: Picture-in-Picture, defect, P2)

Description

[Maria Heres, inactive, contact Ecosystem QA]

Attachment: targeting mismatch PiP.gif

[Affected Versions]:

• Firefox Nightly 99.0a1 (Build ID: 20220210065747)

[Affected Platforms]:

• Windows 10
• Linux Mint 20.2
• macOS 11.6

[Prerequisites]:

• Have Firefox Nightly 99 en-US / en-GB / en-CA installed and open.
• Be enrolled in the Picture-in-Picture - First-time Toggle Text (A/A) study. You can enroll by settings the following preferences and restarting the browser:
  messaging-system.rsexperimentloader.collection_id set to nimbus-preview.
  browser.search.region to one of: 'AU', 'CA', 'GB', 'NZ', 'US'

[Steps to reproduce]:

1. Play a video from a streaming video website (e.g. https://youtu.be/fcjR93Jin4Y).
2. Click on the "Watch in Picture-in-Picture" button displayed on the right-hand side of the video.
3. Close and open the browser again.
4. Observe the “about:studies” page.

[Expected result]:

• You are still enrolled in the study after using the Picture in Picture feature.

[Actual result]:

• You are unenrolled from the study after using the Picture in Picture feature for the first time.

[Notes]:

• The filter set for media.videocontrols.picture-in-picture.video-toggle.has-used in advanced targeting seems to be non-sticky. See #2326 for an explanation of sticky filters.
• Attached is a recording of the issue.

Comment 1

[Ania]

Hi Maria,

I have created a request for a new sticky filter that should resolve the problem.
As soon as the filter is deployed, I will change the experiment targeting and will update this issue.

Comment 2

[Maria Heres, inactive, contact Ecosystem QA]

The issue was fixed here https://github.com/mozilla/experimenter/pull/7088.

Comment 3

[Maria Heres, inactive, contact Ecosystem QA]

We have verified that the issue is no longer reproducible. The following scenarios were verified:

• Users are not enrolled in the study if they have used PiP before.
• Users remain enrolled in the study even after using PiP.

Tested using Firefox Nightly 99.0a1 (Build ID: 20220218090822) on Windows 10, macOS 11.6 and Linux Mint 20.2.