517081 - [faceted search] message result display should coalesce messages with the same message-id header

Status: RESOLVED FIXED

(Thunderbird :: Search, defect)

Description

[Andrew Sutherland [:asuth] (he/him)]

Because of the way gmail messages get exposed via IMAP, interesting messages will tend to be exposed to us multiple times.  For example, a message might be starred, have multiple labels applied, be in the Inbox, etc.  Because gloda does not yet specialize for gmail (and is unlikely to for TB 3.0), this means that gloda sees the message multiple times.

This leaves it up to the faceted search UI to try and help avoid worst-case scenarios.

The simplest solution to this problem is to avoid displaying a message multiple times in the result list.  I say coalesce in the subject, but just avoiding displaying messages whose header-id's we have already displayed is probably easiest.

We probably do not want to try and hide messages prior to the filtering step because someone trying to facet based on the folder might get confused.  Attempting to actually coalesce for that stage is non-trivial since the folder attribute would need to be promoted from a singular attribute to a non-singular attribute.

davida is most familiar with the message result display list at this time.

Comment 1

[David Ascher (:davida)]

It'd be annoying for search to fail badly for gmail users, so taking.

Comment 2

[David Ascher (:davida)]

Attachment: patch v1

Andrew, any feedback on this kind of approach?  It seems to work in my non-gmail ("copy message..."-based) scenario, but I haven't pushed it very hard.

Comment 3

[Andrew Sutherland [:asuth] (he/him)]

Comment on attachment 401962 [details] [diff] [review]
patch v1

This is one of the possibilities I was thinking of.  I would do "if (item.headerMessageID in msgIdsSeen)" instead of "if (msgIdsSeen[item.headerMessageID])" to avoid the strict warning, though.

The other place you could implement the simple solution is to process only the messages being displayed.

The advantage of your approach is that the message counts make more sense (although when we hit the LIMIT case, we may no longer have a big obvious round result number like 400.)  The advantage of the alternative is that the folder faceting is more reliable; a message duplicated in gmail's Inbox, Label1, and Label2 will show up in all locations, instead of randomly only showing up in one of them.

The more complex alternative is to do the consolidation only in the result list pass and to actually build up a list of the duplicated messages and present the fact that they are in multiple locations in the UI.

Comment 4

[David Ascher (:davida)]

Attachment: patch v2

Updated patch, thanks.

I considered mucking w/ the display only, but I think then every # shown in the facet views would be wrong, which would be utterly confusing, no?

The complex alternative doesn't seem worth the effort, given the huge increase in complexity I fear it would bring.  It does seem like we need a better solution to deal with duplicates, regardless of where they come from (had lunch today w/ someone who somehow ended up with 21,000 dupes in his IMAP server, no idea how, and no good way out -- gmail wasn't involved, btw).

Comment 5

[Mark Banner (:standard8)]

Checked in: http://hg.mozilla.org/comm-central/rev/2b0d3d5c173d