1623160 - Create a hash for each result to make detecting duplicates easier

Status: NEW

(Firefox :: Address Bar, enhancement, P3)

Description

[Harry Twyford]

From this comment:

each result could have a "key" property that autogenerates internally (an hash likely), and then it would be even easier to find dupes. Generating the key would consider or ignore www depending on the pref

We should add this key and rework UnifiedComplete's deduping code.

Comment 1

[Drew Willcoxon :adw]

Not sure what priority this should be but it seems like a P3. Please update if that's incorrect.

Comment 2

[Marco Bonardo [:mak]]

Anyway this is not as trivial as it sounds, it requires to move all the deduping code from unifiedcomplete to the muxer, that means the muxer must have more results returned than maxRichResults, and also handle bucketing. but long term that'd be the best outcome.

Comment 3

[Mike de Boer [:mikedeboer]]

(In reply to Marco Bonardo [:mak] from comment #2)

Anyway this is not as trivial as it sounds, it requires to move all the deduping code from unifiedcomplete to the muxer, that means the muxer must have more results returned than maxRichResults, and also handle bucketing. but long term that'd be the best outcome.

This sounds like a good improvement, so you've got my vote to invest in this! Raising to a P2 to reflect that.
Marco, could you add a points-estimate for planning purposes? Thanks!

Comment 4

[Marco Bonardo [:mak]]

It's a large effort, we should start splitting UnifiedComplete into single providers, the first step I'd suggest would be to split out a search suggestions provider, that should help us setting up the basics.
Then we should decide how to handle the heuristic result: should we have a provider for it, or allow any provider to return one? Probably the former, but I'm not sold on a solution yet.
Then there's other things to split out like remote tabs provider.
While splitting the providers we'll start having to handle bucketing and deduping in the Muxer, because that's where all the results go through. This is where hashes to dedupe would be handy. This also means each provider should start returning more than maxRichResults results, so the Muxer has ways to discard some (like dupes).

There are a few things that may end up being problematic:

1. we have a ton of tests for unifiedcomplete, once we start splitting things out those tests will have to be ported to the urlbar component, many may start breaking, fixing test will take a huge amount of time
2. once results arrive from many more sources than today, we could hit again the results flickering problem, and then we should start looking into some kind of ui buffering (return results in chunks every 16 ms?). We already have some problem with the multiple providers today, where a result arrives late and pushes down another one (for example the Search in a private window result, some users complains it appeared just when they were about to click on the previous result in that position)

In practice this is a 13 that requires breakdown, but it's hard to breakdown everything from scratch because roadblocks are to be found along the way.