1724108 - (vpm-promo-login-press) target vpn promo experiment messages ONLY to those who have pressed login button

Status: RESOLVED WONTFIX

(Firefox :: Messaging System, enhancement, P1)

Description

[Dan Mosedale (:dmosedale, :dmose)]

User story: as someone who has connected to the internet through a captive portal, I should be shown a VPN promo only when I have pressed the "Login button" in the captive portal infobar, so that it's likely that I have a sufficient mental context around why the VPN promo has been shown.

Acceptance criteria:

• VPN promo is not shown when the login button has not been pressed, and the captive-portal-login-success has been sent.
• VPN promo is shown when the login button has been pressed and the captive-portal-login-success notification has been sent.
• (Nice to have, but not required for an experiment, could totally be a spinoff bug): If the captive-portal-login-success state is received, and it has been more than N minutes since the login button was pressed, the VPN promo is not shown, since we don't know if they're really associated.

In bug 1721178, code was landed to target experimental VPN promo messages to users when captive-portal-login-success observer notification was received.

If a user has logged into a captive portal through the OS user interface, or another browser, and not through the Firefox UI, the captive portal detection code can sometimes (but not always) detect when their Internet connection is going through a captive portal. In that case, there is a some chance that a captive-portal-login-success notification will be sent despite no captive portal UI being shown. This means that the user will have less or little mental context around why the VPN promo is being shown at that moment, and may have a hard time making sense of it.

As a way to avoid this, we can make that promo trigger only when a captive-portal-login-success is received after the "login" button has been pressed.

This will be useful regardless of what form the promo takes, infobar, modal, or whatever.

Comment 1

[Dan Mosedale (:dmosedale, :dmose)]

Added these acceptance criteria to comment 0:

• VPN promo is not shown when the login button has not been pressed, and the captive-portal-login-success has been sent.
• VPN promo is shown when the login button has been pressed and the captive-portal-login-success notification has been sent.
• (Nice to have, but not required for an experiment. Could totally be a spinoff bug.): If the captive-portal-login-success state is received, and it has been more than N minutes since the login button was pressed, the VPN promo is not shown, since we don't know if they're really associated.

Comment 2

[Dan Mosedale (:dmosedale, :dmose)]

Attachment: Bug 1724108 - Only show captive portal VPN promo after button press, r?valentin!

Comment 3

[Dan Mosedale (:dmosedale, :dmose)]

Attachment: Bug 1724108 - Don't send captive-portal-login-success-after-button-pressed in some cases, r=gijs

Depends on D121935

Comment 4

[Dan Mosedale (:dmosedale, :dmose)]

There's still some more work to do here, but I'd love any comments on the WIP patches...

Comment 5

[Valentin Gosu [:valentin] (he/him)]

At a high level it all looks good to me, but a Firefox peer might be able to provide more on-point feedback with regards to the implementation.

Comment 6

[Dan Mosedale (:dmosedale, :dmose)]

Attachment: Bug 1724108 - switched browser_trigger_listeners to new vpn promo event, r?andreio!

Depends on D121936

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/5e57386eae33
Only show captive portal VPN promo after button press, r=valentin

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/91aa3dd65624
Don't send captive-portal-login-success-after-button-pressed in some cases, r=Gijs

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/0bdbea37f84f
switched browser_trigger_listeners to new vpn promo event, r=andreio

Comment 10

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/5e57386eae33
https://hg.mozilla.org/mozilla-central/rev/91aa3dd65624
https://hg.mozilla.org/mozilla-central/rev/0bdbea37f84f

Comment 11

[Dan Mosedale (:dmosedale, :dmose)]

Comment on attachment 9235001 [details]
Bug 1724108 - Only show captive portal VPN promo after button press, r?valentin!

Beta/Release Uplift Approval Request

• User impact if declined: User Journey team unable to validate metrics and telemetry in experiment on 92 beta & release.
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): UI changes triggered by experiment enrollment; low risk to roll back.
• String changes made/needed:

Comment 12

[Ryan VanderMeulen [:RyanVM]]

In the future, please try to set the dependencies in Phabricator so all the related commits land together in one push.

Comment 13

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9235001 [details]
Bug 1724108 - Only show captive portal VPN promo after button press, r?valentin!

Approved for 92.0b4.

Comment 14

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/0466e29b71bb
https://hg.mozilla.org/releases/mozilla-beta/rev/acfe08471ff9
https://hg.mozilla.org/releases/mozilla-beta/rev/3b7f74051f4a

Comment 15

[Dan Mosedale (:dmosedale, :dmose)]

Will do; thanks!

Comment 16

[Atila Butkovits]

Backed out as requested.

Backout link: https://hg.mozilla.org/integration/autoland/rev/7c80102d91b03c587948977b9372aeb5a132bc52

Comment 17

[Dan Mosedale (:dmosedale, :dmose)]

Thank you! This was backed out because after a discussion with product, we decided to change direction and use a slightly different strategy.