1592671 - [Experiment] Add static triplet branch to support 72 dynamic triplets experiment

Status: VERIFIED FIXED

(Firefox :: Messaging System, enhancement, P1)

Description

[Punam Dahiya [:pdahiya]]

Follow up of https://bugzilla.mozilla.org/show_bug.cgi?id=1591584#c2, scope of this bug is to rename 'supercharge' to 'static' branch, we have interrupts experiments that assumes supercharge as default triplet and we should change that to 'static' triplet branch.

Comment 1

[Punam Dahiya [:pdahiya]]

Attachment: Bug 1592671 - Remove messaging router custom code to run firstrun experiments

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c4e2033deb99
Remove messaging router custom code to run firstrun experiments r=k88hudson

Comment 3

[Arthur Iakab [arthur_iakab]]

Backed out changeset c4e2033deb99 (bug 1592671) for node-newtab test failures.

Backout revision https://hg.mozilla.org/integration/autoland/rev/9e73f5ee80529475d8ead57c9ca3859efe40109d

Failed push https://treeherder.mozilla.org/#/jobs?repo=autoland&selectedJob=278525797&revision=c4e2033deb992527e2835b221dda18d1672475da&searchStr=linux%2Cx64%2Copt%2Cnode%2Ctests%2Csource-test-node-newtab-unit-tests%2Cnode%28newtab%29

Failure logs https://treeherder.mozilla.org/logviewer.html#?job_id=278525797&repo=autoland

Punam can you please take a look?

Comment 4

[Punam Dahiya [:pdahiya]]

Updated patch with tests fixed. Thanks

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/9c5390930437
Remove messaging router custom code to run firstrun experiments r=k88hudson

Comment 6

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/9c5390930437

Comment 7

[Punam Dahiya [:pdahiya]]

Steps for QA
A). Create a new profile
B). Open browser. By default if trailhead.firstrun.branches is empty , default values ‘join’ and ‘supercharge’ are picked for interrupt and triplet.
C). verify below fixes

1. Triplet name is sent as part of utm_term when clicked on a triplet card

e.g. For ‘join-supercharge’,
a) click of card in extended triplets sets utm_term as - trailhead-cards-supercharge-card
b) click of card in second stage triplet (accessed via about:welcome -> continue browsing) sets utm_term as - trailhead-join-supercharge-card

2. Change in pref value 'trailhead.firstrun.branches' picks new interrupt and triplet value on refresh or open of new tab and doesn’t need restart

3. Removed Messaging Router custom code. Pref such as ‘trailhead.firstrun.interruptsExperiment’, ‘trailhead.firstrun.tripletsEnrolled’, ‘trailhead.extendedTriplets.experiment’ are no longer supported and should not be seen in 72 for a new profile

Pref “trailhead.firstrun.didSeeAboutWelcome” continues to be supported and gets set to true if a user has seen about:welcome

C) Set trailhead.firstrun.branches value as 'join-static'. Verify that static brach behaves same as 'supercharge'

Comment 8

[Andreea Cupsa [:acupsa], Experiments QA]

I have verified this issue on Latest Firefox Nightly 73.0a1 (Build ID: 20191205094649) and Firefox Beta 72.0b2 (Build ID: 20191203141951) on Windows 10 x64, Mac 10.14.6 and Debian Gnome 3.22.2.

With the trailhead.firstrun.branches preference set to the join-supercharge value:

• The utm_term has the trailhead-cards-supercharge-card value after clicking on any card from the new tab or from the about:welcome page.

With the trailhead.firstrun.branches preference set to the join-static value:

• The utm_term has the trailhead-cards-static-card value after clicking on any card from the new tab or from the about:welcome page.

3. The ‘trailhead.firstrun.interruptsExperiment’, ‘trailhead.firstrun.tripletsEnrolled’, ‘trailhead.extendedTriplets.experiment’ preferences are no longer displayed in the about:config page on Firefox Beta 72.0b2.

4. The trailhead.firstrun.didSeeAboutWelcome is set to true after a user has seen the about:welcome page.

I have noticed the following behavior for case 2:

• The new interrupt and triplet value is set after a refresh of a new tab page, on a new window, but is not set on the first new tab opened, only on the second one.

@Punam, is it possible that this is a preloading issue? Should we file a bug to track this?

Comment 9

[Punam Dahiya [:pdahiya]]

I have noticed the following behavior for case 2:

• The new interrupt and triplet value is set after a refresh of a new tab page, on a new window, but is not set on the first new tab opened, only on the second one.

@Punam, is it possible that this is a preloading issue? Should we file a bug to track this?

Yes, triplet cards not picking update on first new tab opened is preloading issue and is being tracked in bug https://bugzilla.mozilla.org/show_bug.cgi?id=1595074. Thanks!

Comment 10

[Andreea Cupsa [:acupsa], Experiments QA]

Thanks, Punam.
Considering the comments above (Comment 8 and Comment 9) and the fact that the preloading issue is already being tracked I am marking this issue as verified.