Closed Bug 1626897 Opened 1 year ago Closed 1 year ago

Support and expose tail search suggestions

Categories

(Firefox :: Search, enhancement, P1)

enhancement
Points:
5

Tracking

()

VERIFIED FIXED
Firefox 78
Iteration:
78.1 - May 4 - May 17
Tracking Status
firefox78 --- verified

People

(Reporter: mak, Assigned: harry)

References

(Blocks 1 open bug)

Details

Attachments

(4 files)

We should add support for tail suggestions in the SearchSuggestionsController, the format is pretty much the official one, for a search of "hobbit holes for sale in l" we'd get

[
  "hobbit holes for sale in l",
  [
    "hobbit holes for sale in london",
    "hobbit holes for sale in laguna",
    "hobbit hole for sale in lotr mod",
    "hobbit hole for sale in lake district",
    "hobbit hole for sale in lake district tripadvisor",
    "hobbit hole for sale in lego",
    "hobbit hole for sale in lord of the rings",
    "hobbit hole for sale in layout",
    "hobbit hole for sale in lego set",
    "hobbit hole for sale in location rdr2"
  ],
  [],
  {
    "google:suggestdetail": [
      {
        "mp": "… ",
        "t": "holes for sale in london"
      },
      {
        "mp": "… ",
        "t": "holes for sale in laguna"
      },
      {
        "mp": "… ",
        "t": "hole for sale in lotr mod"
      },
      {
        "mp": "… ",
        "t": "hole for sale in lake district"
      },
      {
        "mp": "… ",
        "t": "hole for sale in lake district tripadvisor"
      },
      {
        "mp": "… ",
        "t": "hole for sale in lego"
      },
      {
        "mp": "… ",
        "t": "hole for sale in lord of the rings"
      },
      {
        "mp": "… ",
        "t": "hole for sale in layout"
      },
      {
        "mp": "… ",
        "t": "hole for sale in lego set"
      },
      {
        "mp": "… ",
        "t": "hole for sale in location rdr2"
      }
    ]
  }
]

Note this differs slightly from https://github.com/dewitt/opensearch/blob/master/mediawiki/Specifications/OpenSearch/Extensions/Suggestions/1.1/Draft%201.wiki#JSONformatted_search_suggestion_responses

we should probably be flexible enough to support both cases.

Also note Google sends more info to Chrome atm, in the future we may want something similar, so the code should be flexible about properties and their values:

[
  "1 cad ",
  [
    "1 cad to eur",
    "1 cad to usd",
    "1 cad",
    "1 cad to inr",
    "1 cad to pkr",
    "1 cad to aud",
    "1 cad to php",
    "1 cad to gbp",
    "1 cad to aed",
    "1 cad to yen",
    "1 cad to mxn",
    "1 cad to hkd",
    "1 cad to brl",
    "1 cad to peso",
    "1 cad to us",
    "1 cad to cop",
    "1 cad to mexican peso",
    "1 cad to pound",
    "1 cad to try",
    "1 cad to idr"
  ],
  [
    "",
    "",
    "",
    "",
    "",
    "",
    "",
    "",
    "",
    "",
    "",
    "",
    "",
    "",
    "",
    "",
    "",
    "",
    "",
    ""
  ],
  [],
  {
    "google:clientdata": {
      "bpc": false,
      "tlw": false
    },
    "google:fieldtrialtriggered": true,
    "google:suggestdetail": [
      {
        "ansa": {
          "l": [
            {
              "il": {
                "t": [
                  {
                    "t": "1 cad<b> to eur</b>",
                    "tt": 8
                  }
                ]
              }
            },
            {
              "il": {
                "t": [
                  {
                    "t": "1 dollaro canadese = 0.69 euro",
                    "tt": 17
                  }
                ]
              }
            }
          ]
        },
        "ansb": "10",
        "ansc": "1582800179255"
      },
      {},
      {},
      {},
      {},
      {},
      {},
      {},
      {},
      {},
      {},
      {},
      {},
      {},
      {},
      {},
      {},
      {},
      {},
      {}
    ],
    "google:suggestrelevance": [
      601,
      600,
      567,
      566,
      565,
      564,
      563,
      562,
      561,
      560,
      559,
      558,
      557,
      556,
      555,
      554,
      553,
      552,
      551,
      550
    ],
    "google:suggesttype": [
      "QUERY",
      "QUERY",
      "QUERY",
      "QUERY",
      "QUERY",
      "QUERY",
      "QUERY",
      "QUERY",
      "QUERY",
      "QUERY",
      "QUERY",
      "QUERY",
      "QUERY",
      "QUERY",
      "QUERY",
      "QUERY",
      "QUERY",
      "QUERY",
      "QUERY",
      "QUERY"
    ],
    "google:verbatimrelevance": 851
  }
]

The search suggestions controller should be able to expose the additional information we get, and then consumers will optionally consume it.

I'm requesting some technical documentation about the "google:suggestdetail" array, stay tuned.

There is no specific technical documentation, but more or less we can use https://cs.chromium.org/chromium/src/components/omnibox/browser/search_suggestion_parser.cc?l=394 as a reference, when parsing the Google proprietary fields.
"mp" is match prefix
"ansa" is answer in suggest

Depends on: 1634173
Assignee: nobody → htwyford
Status: NEW → ASSIGNED
Iteration: --- → 77.2 - Apr 20 - May 3
Iteration: 77.2 - Apr 20 - May 3 → 78.1 - May 4 - May 17
Flags: qe-verify?

This patch does not add support to SearchSuggestionsController's consumers. Do not use this patch without the subsequent Part 3 patch as well.

Depends on D74117

Blocks: 1636696
Blocks: 1637060
Pushed by htwyford@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/fcbe77e9a61f
Part 1 - Move SuggestionsFetch from PlacesSearchAutocompleteProvider to UrlbarProviderSearchSuggestions. r=mak
https://hg.mozilla.org/integration/autoland/rev/61ebd356f31b
Part 2 - Add support for tail suggestions to SearchSuggestionsController.jsm. r=mak
https://hg.mozilla.org/integration/autoland/rev/44e1d9d65379
Part 3 - Add support for tail suggestions to SearchSuggestionController's consumers. r=mak
https://hg.mozilla.org/integration/autoland/rev/97855f04050a
Part 4 - Add tail suggestion tests. r=mak
Depends on: 1639917
Blocks: 1623848

We're still waiting on Google to enable this feature. However, QA can get a headstart by installing this add-on: https://github.com/htwyford/urlbar-richsuggestion-tester. Installation instructions in the repo. It will install a new search engine called Google (firefox-dev) that will return tail suggestions.

Blocks: 1645059
Flags: qe-verify? → qe-verify+

We verified this issue following the instructions from the link in comment 9.
We used Fx 78.0b8 on Windows 10 x64, Ubuntu 18.04 LTS and macOS 10.13.

Flags: qe-verify+
Status: RESOLVED → VERIFIED

:harry, I am running tests on hardware and find an error creating a network port on 9001:
https://firefoxci.taskcluster-artifacts.net/HmPJzbk1Q7OhUrrfFl58aw/0/public/logs/live_backing.log

this works for me locally on my windows 10 laptop, but not in a datacenter (what we were going to use to help with load/cost on our tests for a while).

Is port 9001 a requirement? Can we make it dynamic or in a different range?

Flags: needinfo?(htwyford)

Not a requirement. I just picked 9001 because we already have some search engines in automation on port 9000. Port 9001 is referenced here. It can probably be updated to anything other than 9000. Is there a particular port that you would prefer?

Flags: needinfo?(htwyford)
You need to log in before you can comment on or make changes to this bug.