Closed Bug 1665934 Opened 4 years ago Closed 4 years ago

Onboarding intervention for tab-to-search

Categories

(Firefox :: Address Bar, enhancement, P1)

enhancement
Points:
5

Tracking

()

RESOLVED FIXED
83 Branch
Iteration:
83.2 - Oct 5 - Oct 18
Tracking Status
firefox83 --- fixed

People

(Reporter: verdi, Assigned: bugzilla)

References

()

Details

Attachments

(6 files)

Instead of the regular tab-to-search result, the first N times we should show an intervention style result. After you encounter this intervention result N times, you will see the regular tab-to-search result from then on.

Severity: -- → S3
Priority: -- → P2
Points: --- → 5
Blocks: 1647921
No longer blocks: urlbar-searchshortcuts
Assignee: nobody → htwyford
Status: NEW → ASSIGNED
Iteration: --- → 83.1 - Sept 21 - Oct 4

There are a few unknowns that need to be clarified:

  • How many times should we show tab-to-search as a tip and not as a normal result?
  • Should we show the tip version at most once per session, like Search Tips?
  • Product suggested adding an "Okay, Got It" button to dismiss the tip-sized tab-to-search results. If we do this, then we will need some other kind of target on the tip to enter search mode. Or we make the entire rest of the tip a target?
Keywords: blocked-ux
Priority: P2 → P1
Iteration: 83.1 - Sept 21 - Oct 4 → 83.2 - Oct 5 - Oct 18
Attached image unselected.png

For the onboarding intervention, we'll use a two-row intervention. It will have the same animation as the regular result (see Bug 1647924). The icon (coming soon) should be 32x32.

Attached image selected.png

The selected state will use a double high highlight to cover the entire result.

More details from UX:

  • We will show the large-size result a maximum of 10 times total and at most twice per session. These maximums should be configurable to allow for experimentation.
  • Once the large result is selected, we should never show it again.
Keywords: blocked-ux

This is nearly ready for review. Barring review feedback, I don't plan to make significant changes to the patch, so it's safe to start reading it over if you feel like it. I plan on splitting it up into more digestible parts before actually tagging anyone for review. There are a couple of issues that still need to be resolved:

  • This patch does not use the icon in the UX spec. I'm waiting on that icon and just using the engine icon in the meantime.
  • We also don't have a spec for how the result should wrap at narrow window widths, so I put some placeholder behaviour in.
  • It appears dynamic results are incapable of overriding the CSS for normal Urlbar classes unless !important is used, regardless of specificity. Thus a few rules in dynamic-tabToSearch.css use !important, which isn't ideal. It appears this is because dynamic result stylesheets are loaded as AGENT_SHEETs. Changing that value to AUTHOR_SHEET allows the dynamic result CSS to override the normal Urlbar classes without !important. This strikes me as a tough issue. dynamic-tabToSearch.css is an agent sheet since it's provided by the browser. I do wonder if its defensible to load it as an author sheet since it will usually be loaded by an extension/external developer (this might be a stretch :). I'm not well-versed enough in the technicalities here to really make a call, so I'm eager to hear if @dao or @adw have any thoughts on how to resolve this issue.

Try run for my WIP patch, just to check what needs to be fixed: https://treeherder.mozilla.org/#/jobs?repo=try&revision=b75ccc78271ca3a9d9600171a059421c16945832

Attachment #9180554 - Attachment description: Bug 1665934 - WIP - Add onboarding intervention for tab-to-search. → Bug 1665934 - Part 2 - Add onboarding intervention for tab-to-search. r?adw!,dao

Comment on attachment 9180744 [details]
Bug 1665934 - Part 3 - Add tab-to-search onboarding telemetry. r?adw!

  1. What questions will you answer with this data?

How the new tab-to-search feature in the Urlbar is use. I'm adding telemetry for tab-to-search onboarding results. These are large tab-to-search results that appear the first few times the user sees one of those results.

  1. Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements? Some example responses:

We want to know if users see and interact with the onboarding results to know if they successfully get people to use tab-to-search results.

  1. What alternative methods did you consider to answer these questions? Why were they not sufficient?

We plan on also doing user studies with this feature, but we want aggregate data to understand how it is used more broadly.

  1. Can current instrumentation answer these questions?

No, this entry point would currently record in a urlbar.searchmode.other probe, which does not allow us to distinguish it from other search mode entry points.

  1. List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories found on the Mozilla wiki.

Measurements

urlbar.searchmode.tabtosearch_onboard

Category 2: The keys for this scalar are strings describing what kind of search mode was entered after the user selects a tab-to-search onboarding result. This will be the name of a search engine, like "Google", or "DuckDuckGo". We only collect the names of engines that are bundled with Firefox. If the user enters search mode with an engine they installed themselves, we record "other" as the key.

  1. Please provide a link to the documentation for this data collection which describes the ultimate data set in a public, complete, and accurate way.

The documentation on Urlbar telemetry will be updated with this patch to describe the new probes.

  1. How long will this data be collected? Choose one of the following:

The search team will permanently monitor this data. Teon will look after it.

  1. What populations will you measure?

All Desktop users, countries, and locales.

  1. If this data collection is default on, what is the opt-out mechanism for users?

Standard telemetry opt-out.

  1. Please provide a general description of how you will analyze this data.

We will use the measurement dashboard and STMO.

  1. Where do you intend to share the results of your analysis?

Internally, with the search team and Firefox Desktop leadership.

  1. Is there a third-party tool (i.e. not Telemetry) that you are proposing to use for this data collection?

No.

Attachment #9180744 - Flags: data-review?(mmccorquodale)
Pushed by htwyford@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/8c1edb639aa6
Part 1 - Allow results to set their own resultSpan. r=adw
https://hg.mozilla.org/integration/autoland/rev/1b03418a14f6
Part 1.5 - Split _addDynamicStylesheet and _removeDynamicStylesheet out from UrlbarView into the file body. r=adw
https://hg.mozilla.org/integration/autoland/rev/56946d59bf0a
Part 2 - Add onboarding intervention for tab-to-search. r=fluent-reviewers,adw,flod
https://hg.mozilla.org/integration/autoland/rev/342354dfd594
Part 3 - Add tab-to-search onboarding telemetry. r=adw

Ah sorry, forgot about the data-review. I'm going to request backout.

Please leave this open until Part 3 of the patch lands, covering telemetry.

Keywords: leave-open

Comment on attachment 9180744 [details]
Bug 1665934 - Part 3 - Add tab-to-search onboarding telemetry. r?adw!

  1. Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?
    Yes, this will be documented in the urlbar telemetry docs.

  2. Is there a control mechanism that allows the user to turn the data collection on and off?
    Yes, users can opt out of telemetry.

  3. If the request is for permanent data collection, is there someone who will monitor the data over time?
    Teon Brooks and the search team will monitor.

  4. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?
    Category 2, interaction data.

  5. Is the data collection request for default-on or default-off?
    Default on.

  6. Does the instrumentation include the addition of any new identifiers?
    No new identifiers.

  7. Is the data collection covered by the existing Firefox privacy notice?
    Yes.

  8. Does there need to be a check-in in the future to determine whether to renew the data?
    No.

  9. Does the data collection use a third-party collection tool?
    No.


data-review +

Attachment #9180744 - Flags: data-review?(mmccorquodale) → data-review+
Pushed by htwyford@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/c297c729eaa2
Part 1 - Allow results to set their own resultSpan. r=adw
https://hg.mozilla.org/integration/autoland/rev/b256f36594db
Part 1.5 - Split _addDynamicStylesheet and _removeDynamicStylesheet out from UrlbarView into the file body. r=adw
https://hg.mozilla.org/integration/autoland/rev/0cd1d314471b
Part 2 - Add onboarding intervention for tab-to-search. r=fluent-reviewers,adw,flod
Pushed by htwyford@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/e9bece1d1360
Part 3 - Add tab-to-search onboarding telemetry. r=adw

All the parts landed afaict.

Status: ASSIGNED → RESOLVED
Closed: 4 years ago
Keywords: leave-open
Resolution: --- → FIXED
Target Milestone: --- → 83 Branch
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: