[faceted search] Deleted for good/expunged messages are shown in search results

RESOLVED FIXED in Thunderbird 3.0b4

Status

Thunderbird
Search
--
major
RESOLVED FIXED
9 years ago
3 years ago

People

(Reporter: Thomas D. (currently busy elsewhere), Assigned: asuth)

Tracking

({privacy})

Trunk
Thunderbird 3.0b4
privacy
Bug Flags:
blocking-thunderbird3 +

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

Messages that have been deleted for good (expunged) with <shift>+<del> continue to show up in search results, even after closing and reopening thunderbird.
They shouldn't, as soon as they are shift-deleted (while showing mails from trash is ok). Iow, when user shift-deletes a message, all relating data should be immediately removed from gloda, and never be re-indexed as long as the deleted-for-good-flag is set.

It's bad enough that shift-deleted messages physically remain in the folder file until you compress, but showing them in search results is definitely a considerable privacy issue --> major.
(Reporter)

Comment 1

9 years ago
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.3a1pre) Gecko/20090911 Shredder/3.1a1pre

Worse, even compacting does not make such messages go away, still shown in search results. That's a perfect failure in terms of privacy. Nominating blocking tb3.
Flags: blocking-thunderbird3?
(Reporter)

Comment 2

9 years ago
Apart from compromising privacy, of course this will also render gloda-based searches completely unusable very quickly, as search results will continue to show false positives from the vast and (as per comment #1) ever-increasing choice of messages that have ever(!) been expunged. This should block, no doubt.
Andrew, is this a duplicate of bug 465618?
(Assignee)

Comment 4

9 years ago
No.  Fixing bug 465618 would make this problem less obvious, but the problem is GlodaMsgSearcher is building a query that can return messages that are marked deleted.  This happens because it is using hand-rolled SQL and has insufficient test coverage.
Assignee: nobody → bugmail
Status: NEW → ASSIGNED
(In reply to comment #0)
> They shouldn't, as soon as they are shift-deleted (while showing mails from
> trash is ok).

IMO, those should be grayed out, put at the end of the search results or something like that ...
(Reporter)

Comment 6

9 years ago
Which are "those"? I assume you mean mails in trash? We definitely can't show expunged mails that have been shift-deleted.
Status: ASSIGNED → NEW
Flags: in-testsuite?
Flags: blocking-thunderbird3? → blocking-thunderbird3+
Target Milestone: --- → Thunderbird 3.0b4
(Assignee)

Comment 7

9 years ago
Created attachment 400558 [details] [diff] [review]
filter out deleted messages

This works, but it turns out our deletion processing might be more broken than I was thinking.  Using a local account, I deleted a message from my sent folder so it ended up in trash.  Then I deleted it from trash.  But it still showed up when deleted from trash.  Then I emptied the trash, and the search no longer found it.
Attachment #400558 - Flags: review?(bienvenu)

Comment 8

9 years ago
Comment on attachment 400558 [details] [diff] [review]
filter out deleted messages

this fixes my simple test of searching for messages I've just deleted from the imap trash.
Attachment #400558 - Flags: review?(bienvenu) → review+
Attachment #400558 - Flags: approval-thunderbird3+
(Reporter)

Comment 9

9 years ago
(In reply to comment #7)
> Created an attachment (id=400558) [details]
> filter out deleted messages

Andrew, thanks for adressing this.
Which kind of deleted messages are you filtering out, trash-mails AND expunged or expunged-only?
I think user should have some way of including trash into the search. Either just include them, and/or gray them out, and/or put them at the end of search results (as per comment #5), and/or provide a checkbox to include them. After all, trash-mails still exist, and maybe the one you are looking for accidentally ended up in the bin.

> This works, but it turns out our deletion processing might be more broken than
> I was thinking. Using a local account, I deleted a message from my sent folder
> so it ended up in trash.  Then I deleted it from trash.  But it still showed
> up when deleted from trash.  Then I emptied the trash, and the search no
> longer found it.

This bug is not about the mails in trash, it's only about mails that have been expunged using *<shift>*-delete (in ANY folder, not just trash).
Do shift-deleted messages from other folders still show up after your patch (they shouldn't)?
(Assignee)

Comment 10

9 years ago
pushed:
http://hg.mozilla.org/comm-central/rev/3cb3449994f6
Status: NEW → RESOLVED
Last Resolved: 9 years ago
Resolution: --- → FIXED
(Assignee)

Comment 11

9 years ago
(In reply to comment #9)
> Which kind of deleted messages are you filtering out, trash-mails AND expunged
> or expunged-only?

The filtering being performed is for correctness.  If a message is expunged, gloda should no longer know about it but for performance reasons we can't instantly perform all the purges (nor would we actually want to).  The filtering is of messages that are in limbo and technically should no longer exist.  Having said that, there are still bugs in deletion handling that are being worked on right now.  No guarantees until the unit tests land.

Gloda still knows about messages in the trash.
Duplicate of this bug: 516650

Updated

9 years ago
Component: General → Search
QA Contact: general → search
Flags: in-testsuite?
You need to log in before you can comment on or make changes to this bug.