1873469 - [Experiment] [Intermittent]The Picture-in-picture callout message is not displayed on YouTube videos

Status: RESOLVED WONTFIX

(Firefox :: Messaging System, defect, P1)

Description

[Patriciu Pop [:ppop], Ecosystem QA]

Attachment: pip_callout_flash.mp4

[Affected versions]:

• Firefox Beta 122.0b6 (Build #2015981850)

[Affected Platforms]:

• Windows 10 x64
• Windows 11 x64
• macOS 13.5
• Ubuntu 22.04 x64

[Prerequisites]:

• Have the latest Firefox Beta 122.0b6 installed.
• Have the browser's channel set to "release" in the channel-prefs.js file.
• Have a browser client enrolled in a Nimbus experiment that enables the PiP callout. https://experimenter.services.mozilla.com/nimbus/picture-in-picture-first-time-use-callout/summary

[Steps to reproduce]:

1. Open the browser with the profile from prerequisites.
2. Navigate to youtube.com and open any video.
3. Observe the addressbar.

[Expected result]:

• The PiP address bar icon and callout are displayed.

[Actual result]:

• Only the icon is displayed and the callout is not triggered.

[Notes]:

• The issue is reproducible when using an adblocker as well (Ublock Origin)
• In some cases, the callout was briefly displayed before the video's ads loaded.
• The issue is also reproducible on amazon.com videos.
• The following error is displayed in the browser console: Unknown event: ["messaging_experiemnts", "reach", "featureCallout"]
• Attached a screen recording of the issue:

Comment 1

[Punam Dahiya [:pdahiya]]

NI product to help triage and determine if issue here is blocker for experiment.

Comment 2

[Shane Hughes [:aminomancer]]

Unfortunately there's just not a lot that can be done when we're trying to show a popup pointing to a certain element, but that element is very erratic and transitory in its presentation. It's not like we can make the feature callout more permanent than the PiP button, since the PiP button is what we're trying to get the user to interact with. If the button's not present when the feature callout is, then it would just be confusing.

That said, I've thought of 2 possible mitigations.

1. Remove the 1-per-lifetime frequency cap, and instead have the CTA block the feature callout. This way, if it disappears because the PiP icon disappeared, it will still be able to reappear 24hrs later. But if the user explicitly clicks the button to dismiss it, then it will never reappear again.
2. Delay the experiment and land changes allowing us to filter the experiment to certain websites. This isn't perfect, since those websites may also have scenarios where the PiP icon disappears. But that was the original intent of the experiment, and they're probably less problematic than youtube or websites with a scrolling feed.

The following error is displayed in the browser console: Unknown event: ["messaging_experiemnts", "reach", "featureCallout"]

Filed bug 1874264 for this. It's unrelated to the experiment.

Comment 3

[Vtay]

My preference is to go with Option 1 - removing the 1-per-lifetime frequency cap.

In QA, can we make sure we QA that they dont see it again when they dismiss it.

Comment 4

[Shane Hughes [:aminomancer]]

We launched the experiment without changes