1765907 - PBM experiment message should override default promo message

Status: VERIFIED FIXED

(Firefox :: Messaging System, enhancement, P1)

Description

[Punam Dahiya [:pdahiya]]

With Fx100, about:privatebrowsing experiment user falls back to default promo message when experiment message frequency cap weekly or lifetime is met.

https://searchfox.org/mozilla-central/source/browser/components/privatebrowsing/content/aboutPrivateBrowsing.js#228
https://searchfox.org/mozilla-central/source/browser/components/newtab/lib/ASRouter.jsm#1620

Ideally when a user is enrolled in an experiment , promo shown in experiment should override default promo message even after frequency cap of experiment message is met.

Comment 1

[Punam Dahiya [:pdahiya]]

As per product, in future there could be use cases where default promo message should be shown after experiment promo. To cover those usecases, instead of blanket override of default , a flag in experiment message (e.g. hideDefault) should determine if default promo should be visible or hidden.

To support getting telemetry from default promo message spotlight modal, metrics flag that controls telemetry should be computed and passed dynamically.

In Fx100 metric flag is set inside action.data of spotlight message https://searchfox.org/mozilla-central/rev/5204ea3d3dac5832075fb61d9689dd7879ed91b2/browser/components/newtab/lib/OnboardingMessageProvider.jsm#125

In future to support collecting telemetry from default message shown in an experiment, we should compute and set this flag dynamically before passing from aboutPrivateBrowsing.js
https://searchfox.org/mozilla-central/rev/5204ea3d3dac5832075fb61d9689dd7879ed91b2/browser/components/privatebrowsing/content/aboutPrivateBrowsing.js#123

Comment 2

[Punam Dahiya [:pdahiya]]

One option to implement override default promo messages via flag set in experiments is

• Have default promo messages id in OnboardingMessageProvider suffix with '*_DEFAULT' or a separate field type: 'default'
  https://searchfox.org/mozilla-central/source/browser/components/newtab/lib/OnboardingMessageProvider.jsm#98

• Have experiment messages configure with a field e.g. hideDefault: true or false

• Inside sendPbNewTabMessage before handleMessageRequest call filter out default messages from state.messages if a 'pb_newtab' experiment message has hideDefault as true

https://searchfox.org/mozilla-central/rev/5204ea3d3dac5832075fb61d9689dd7879ed91b2/browser/components/newtab/lib/ASRouter.jsm#1613

Comment 3

[Dan Mosedale (:dmosedale, :dmose)]

separate field type: 'default'

This seems like the cleaner of the two choices.

Comment 4

[Punam Dahiya [:pdahiya]]

Attachment: Bug 1765907 - PBM experiment message should override default promo message

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/297265ab08f0
PBM experiment message should override default promo message r=mviar

Comment 7

[Cristian Tuns]

Backed out for causing mochitest failures on browser_privatebrowsing_protocolhandler.js

• Backout link
• Push with failures
• Failure Log
• Log 2: https://treeherder.mozilla.org/logviewer?job_id=377221418&repo=autoland

Comment 9

[Punam Dahiya [:pdahiya]]

Trying re-land after intermittent failure fix thanks

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/eb12cdc5d1e8
PBM experiment message should override default promo message r=mviar

Comment 11

[Cristian Tuns]

Backed out for causing mochitest failures on browser_privatebrowsing_focus_promo.js

• Backout link
• Push with failures
• Failure log
• Failure line: TEST-UNEXPECTED-FAIL | browser/components/privatebrowsing/test/browser/browser_privatebrowsing_focus_promo.js | Focus promo is shown for allowed region - null == true - got null, expected true (operator ==)

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/851e8c15869e
PBM experiment message should override default promo message r=mviar

Comment 13

[Punam Dahiya [:pdahiya]]

Updated patch by adding extra cleanups and removing test opening spotlight from PBM that seems to be cause . Filed followup to cover opening spotlight test flow

https://bugzilla.mozilla.org/show_bug.cgi?id=1769085

Comment 14

[bszekely]

https://hg.mozilla.org/mozilla-central/rev/851e8c15869e

Comment 15

[Punam Dahiya [:pdahiya]]

Comment on attachment 9274113 [details]
Bug 1765907 - PBM experiment message should override default promo message

Beta/Release Uplift Approval Request

• User impact if declined: PBM experiment in Fx101 will not have capability to configure show/hide default promo from experiment JSON
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: Yes
• If yes, steps to reproduce: See Test Plan in https://phabricator.services.mozilla.com/D144871
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): This enhancement is used in experiments only
• String changes made/needed: None
• Is Android affected?: No

Comment 16

[Simona Rosu [:srosu], Ecosystem QA]

I‘ve verified this enhancement using the latest Firefox Nightly 102.0a1 (Build ID: 20220515214519) on Windows 10 x64, macOS 12.3.1, and Ubuntu 20.04 x64.

• PBM experiment message overrides the default promo message.
• The experiment metrics are displayed in the “Browser Console”.
• The PBM experiment message is no longer displayed on a new private tab.

Comment 17

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9274113 [details]
Bug 1765907 - PBM experiment message should override default promo message

Approved for 101.0b8.

Comment 18

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/e2df93963a7e

Comment 19

[Simona Rosu [:srosu], Ecosystem QA]

I‘ve verified this enhancement using Firefox Beta 101.0b8 (Build ID: 20220517185920) on Windows 10 x64, macOS 12.3.1, and Ubuntu 20.04 x64.

• PBM experiment message overrides the default promo message.
• The experiment metrics are displayed in the “Browser Console”.
• The PBM experiment message is no longer displayed on a new private tab.

Comment 20

[Alex Finder]

Hello! We encountered the following regression for talos on mozilla-beta. Would it be possible for this push to have triggered that regression ?

Comment 21

[Punam Dahiya [:pdahiya]]

Fix in this bug touches only about:privatebrowsing . Regressing Talos test doesn’t seem to indicate that its testing page load in private browsing mode and IMO doesn’t appear to be the cause.

https://searchfox.org/mozilla-central/source/testing/talos/talos/tests/perf-reftest-singletons/window-named-property-get.html

https://firefox-source-docs.mozilla.org/testing/perfdocs/talos.html#perf_reftest_singletons

https://firefox-source-docs.mozilla.org/testing/perfdocs/talos.html#page-load