1770745 - [Experiment] The Import ECO spotlight is automatically dismissed after ~20s even if there is no interaction with it

Status: RESOLVED WONTFIX

(Firefox :: Messaging System, defect)

Description

[Cosmin Muntean [:cmuntean], Ecosystem QA]

Attachment: Import Spotlight auto dismiss.mp4

[Affected versions]:

• Firefox Release 100.0.2
• Beta 101.0bx
• Nightly 102.0a1

[Affected Platforms]:

• Mac 10.15.7
• Windows 10 x64

[Prerequisites]:

• Have the latest Firefox Release 100.0.2 installed.
• Have a new Firefox profile.
• Be enrolled in Treatment A, B or C.
• The "Import Bookmarks" spotlight was not previously displayed.

[Steps to reproduce]:

1. Open the Firefox browser with the profile from prerequisites.
2. After the Import ECO spotlight is displayed wait ~20s.
3. Observe the behavior.

[Expected result]:

• The Import ECO spotlight correctly remains displayed.

[Actual result]:

• The Import ECO spotlight is automatically dismissed even if there is no interaction with it.

[Notes]:

• Also, if you manage to import from another browser in less than 20s, the "Import" spotlight remains displayed even if you completed the import flow.
• The following event is registered: "event":"AUTO_ADVANCE","event_context":"{\"source\":\"secondary_button\",\"page\":\"about:welcome\"}","message_id":"COVERED-PIN-INFREQUENT-ECO2205-EN:TREATMENT-B_0_Bring"
• Attached is a screen recording of the issue.

Comment 1

[Dan Mosedale (:dmosedale, :dmose)]

This is a side-effect of the hack we used so that the user doesn't have to click the "dismiss" button after completing (or aborting?) the import.
It also means that some number of people that get presented with the dialog, if they happen to get distracted at the wrong time (ie go off to answer a phone call), they'll never really see it at all, and our telemetry will say that we dismissed it, which won't be true. Then again, if we're only going off the "Import" button being clicked, this won't make any difference to that.

We could revert that hack, and then people would need to manually dismiss after completing or aborting the dialog.

Akash, Ed, any opinions here?

As I said in bug 1770737, unless data science pulls a rabbit out of their hat, I'm leaning more and more towards just trying to land the right fix for 102 (and uplift for 101 if we get really lucky), and delay this experiment....

Comment 2

[Akash Mathur [:akashmathur]]

Looks like an interesting tradeoff in both cases - it would be a poor user experience for the user to have to click 'dismiss' after importing, and on the flip side we don't want to incorrectly count dismisses with the auto-dismiss after 20s.

I agree with Dan - let's hear from data science and go from there. We can push this to 102 in case data science doesn't have anything we can use.

Comment 3

[Ed Lee :Mardak]

Telemetry is different for dismiss vs auto-dismiss with bug 1764070.

Comment 4

[Punam Dahiya [:pdahiya]]

We can differentiate in telemetry how many users encounter auto dismiss, its a known issue for the experiment and can be marked as won't fix.