1897804 - Fox Doodle Spotlight Rollout enrollment targeting should exclude First Run

Status: RESOLVED FIXED

(Firefox :: Messaging System, defect, P1)

Description

[Punam Dahiya [:pdahiya]]

Scope of this bug is to ensure we are not showing spotlight fox doodle on about:welcome, asking users to set default on first run right after Firefox install. For windows this should be possible by including !isFirstStartup in enrollment targeting and to be extra safe profileRestartCount > 0.

https://experimenter.services.mozilla.com/nimbus/fox-doodle-set-to-default-early-day-user-en-treatment-a-rollout/summary

Anothe option is to end rollout in favor of message landed in Fx124 with Bug 1873793

Comment 1

[Punam Dahiya [:pdahiya]]

Attachment: Fox Doodle Spotlight on First Run

Comment 2

[Punam Dahiya [:pdahiya]]

NI Jason to help confirm if spotlight rollout is intended to enroll profile age > 7 as commented here Thanks

Comment 3

[Jason Prickett]

There is no enrollment for this as the message has landed in tree.

The targeting includes (currentDate|date - profileAgeCreated|date) / 86400000 <= 28 && (currentDate|date - profileAgeCreated|date) / 86400000 >= 7 which should prevent profiles <= 7 days old and >= 28 days old from seeing the message. Adding targeting to exclude first run users would be beneficial to prevent the edge case of a user still seeing the message when they never closed their browser after starting up firefox for the first time.

Comment 4

[Punam Dahiya [:pdahiya]]

(In reply to Jason Prickett from comment #3)

There is no enrollment for this as the message has landed in tree.

The targeting includes (currentDate|date - profileAgeCreated|date) / 86400000 <= 28 && (currentDate|date - profileAgeCreated|date) / 86400000 >= 7 which should prevent profiles <= 7 days old and >= 28 days old from seeing the message. Adding targeting to exclude first run users would be beneficial to prevent the edge case of a user still seeing the message when they never closed their browser after starting up firefox for the first time.

Unfortunately profileAge is not a reliable check for avoiding collision with about:welcome as its possible for users coming from stub installer after reinstall with restore default checked to have profile age between 7 and 28 days.

As discussed in triage let's re-purpose bug to adjust message targeting here for default message to include profileRestartCount > 0

Steps to replicate another edge case of Fox Doodle Spotlight showing up for returning users
a) Install Firefox Release with new profile. Adjust profile created, firstuse date under times.json in profile folder to be a week old
b) Delete and reinstall of Firefox (with restore default checkbox checked) picks last opened profile for the channel ( same profile as one used for step a) and shows spotlight on about:welcome

Comment 5

[Meg Viar [:mviar]]

There's some delay in processing my VMware license, so I'll remove myself until I have a Windows testing environment set up.

Comment 6

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:aminomancer, could you have a look please?

For more information, please visit BugBot documentation.

Comment 7

[Meg Viar [:mviar]]

I explored using profileRestartCount > 0 in the targeting, but the value appears to always be 0 on startup, then jumps up the expected value after browsing.

I found a note in this patch adding the profileRestartCount targeting which might explain this behavior: 'The targeting attribute doesn't get it's value until "browsing has started" and there is an "idle time window".'

Unless there's a workaround for this, we'll need to find (or create) another reliable way of avoiding collisions with about:welcome. I'm guessing this message that also includes source == 'startup' in the targeting could collide for the same reasons as the message featured in this bug. If so, its targeting should also be updated.

I'm going to look into using previousSessionEnd > 0 for this instead of profileRestartCount.

Comment 8

[Meg Viar [:mviar]]

Attachment: Bug 1897804 - Fox Doodle Spotlight and Fox Doodle Set Default enrollment targeting should exclude First Run

Comment 9

[Punam Dahiya [:pdahiya]]

(In reply to Meg Viar [:mviar] from comment #7)

I explored using profileRestartCount > 0 in the targeting, but the value appears to always be 0 on startup, then jumps up the expected value after browsing.

Clarifying you mean value is 0 on first start up and increments to > 0 on subsequent startup. My understanding we want spotlight to not show on first session/first run and only subsequent sessions which should work with profileRestartCount > 0

Comment 10

[Meg Viar [:mviar]]

(In reply to Punam Dahiya [:pdahiya] from comment #9)

(In reply to Meg Viar [:mviar] from comment #7)

I explored using profileRestartCount > 0 in the targeting, but the value appears to always be 0 on startup, then jumps up the expected value after browsing.

Clarifying you mean value is 0 on first start up and increments to > 0 on subsequent startup. My understanding we want spotlight to not show on first session/first run and only subsequent sessions which should work with profileRestartCount > 0

AFAICT, the value of profileRestartCount doesn't automatically increment on subsequent startup until you actually start browsing. You can see in ASRouter targeting that it appears as 0 first, then updates after you start browsing (visible by refreshing the targeting page). Let me know if I'm missing some context here, though.

Comment 11

[Punam Dahiya [:pdahiya]]

(In reply to Meg Viar [:mviar] from comment #10)

(In reply to Punam Dahiya [:pdahiya] from comment #9)

(In reply to Meg Viar [:mviar] from comment #7)

I explored using profileRestartCount > 0 in the targeting, but the value appears to always be 0 on startup, then jumps up the expected value after browsing.

Clarifying you mean value is 0 on first start up and increments to > 0 on subsequent startup. My understanding we want spotlight to not show on first session/first run and only subsequent sessions which should work with profileRestartCount > 0

AFAICT, the value of profileRestartCount doesn't automatically increment on subsequent startup until you actually start browsing. You can see in ASRouter targeting that it appears as 0 first, then updates after you start browsing (visible by refreshing the targeting page). Let me know if I'm missing some context here, though.

+1 profileRestartCount is not behaving reliably for me at least in Nightly and staying 0 unclear if its some TelemetrySession configurations.

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d463493adf25
Fox Doodle Spotlight and Fox Doodle Set Default enrollment targeting should exclude First Run r=omc-reviewers,pdahiya

Comment 13

[Cristian Tuns]

https://hg.mozilla.org/mozilla-central/rev/d463493adf25