1245532 - Quick filter by body doesn't find message if message stored in IMAP folder

Status: NEW

(Thunderbird :: Filters, defect)

Description

[Robin Green]

I am connecting to gmail via IMAP. When I search for "Reopened" in body using quick filter, I don't get any results, even though there is a mail that contains that word. The mail is multipart/alternative and the text appears in both parts, including the text/plain part which comes first.

Comment 1

[Jorg K (CEST = GMT+2)]

Does this work correctly on a local folder? Can you please attach the message that is not found (drag to the desktop as a .eml file, attach the file).

Comment 2

[Robin Green]

Yes, it works correctly with a local folder.

Sorry, I can't attach the email because it is a work email.

Comment 3

[Jorg K (CEST = GMT+2)]

Attachment: Sample e-mail showing the problem

(In reply to Robin Green from comment #2)
> Yes, it works correctly with a local folder.
> Sorry, I can't attach the email because it is a work email.
OK, well, I have an e-mail which reproduces the problem. Look for "250". If you store the message in an IMAP folder, it doesn't get found, but in a local folder it does.
Robin: Could you please try whether this message reproduces the problem for you.

Comment 6

[trashcan4junk]

I think until these issues are resolved, there needs to be a obvious warning or disclaimer that GLODA does not work on IMAP reliably.

I submitted a bug but it was marked as not useful or providing extra information.  I am attaching some screenshots illustrating what sounds like a different problem, but related to search.  Let me know if a separate bug report is needed but my previous was marked a duplicate of this.

Not sure I can upload screenshots however.

Comment 7

[trashcan4junk]

Attachment: Quickfilter of Archives folder with Subject and Body, 48 results from 2016

Comment 8

[trashcan4junk]

Attachment: Dragon Search (ctrl+K) yeilds only 10 results, some from another account entirely, only the inbox is of one account is searched, not the archives folder for some reason

Comment 9

[trashcan4junk]

A comment about both of the screenshots I uploaded.  This was after I removed the global-messages-db.sqlite file and reindexed with a per message setting (per mbox before yielded same issues with search).  Finally, global-messages-db.sqlite exists on an NFS shared home directory which may be the root of the problem for me in particular (file locking issue, NFS related).

Thanks everyone.

Comment 10

[trashcan4junk]

As such, I will copy my profile to local, create a symlink, and try these steps again to see if it's NFS related or not.

Comment 11

[trashcan4junk]

After moving and rebuilding the index, I can confirm that quickfilter with body does not work reliably for me either.

I will keep an eye on it to see if I can identify cases where search does not yield a result where we'd expect one, and provide the .eml example or whatever is helpful.  My method of comparing with Quickfilter and GLODA won't be effective if the body cannot be filtered reliably.

Thanks

Comment 17

[Mannshoch]

It seems that this bug got forsake. Is there a way to get it on schedule back for fixing?

Comment 18

[Jorg K (CEST = GMT+2)]

Sadly body search (including "Search Message", Filters and Quick Filter) suffers from design flaws which are not easy to fix other than by (major) redesign. Recently I've fixed the most blatant faults in bug 1259534 and bug 1427124 for searching local folders. You could try whether that improves the situation for IMAP folders synchronised for offline use in TB 58 beta 3 or TB 52.6 coming out in the next few days.

Further reading: Bug 1259534 comment #14.

Comment 19

[Mannshoch]

Thanks for your answer.
I'm not up-to-date with the development status of TB. Firefox now has finished their big transfer to the new Architecture and If TB now do also redesign I hope these Filter system get the needed redesign to.

Comment 21

[RobL]

This exact same issue happens for me: Every time I move an email to a folder for an IMAP account, that email will NEVER be found by the "Filter these messages" box (aka, the quick filter).

It is very easy to reproduce. Choose an email in some folder A that contains some memorable text FOO. Now drag that email from folder A to folder B. Now select folder B. Now type FOO into the "Filter these messages" box, make sure "Body" is selected, and hit enter. Your FOO email will not be found.

This bug is literally 100% reproducible: It happens EVERY time I drag an email to a folder, regardless of the version of Thunderbird, which OS I'm using, or who is hosting the email account. I have reproduced this on Windows 7 with various older versions of Thunderbird with version numbers in the 90s. I have reproduced it on Lubuntu with a version in the 90s and the latest version. I am currently using Linux Mint and Thunderbird version 91, and it happens here too. I have reproduced it with a gmail hosted IMAP account, as well as a rackspace hosted IMAP account.

This should definitely be prioritized higher for several reasons:

1. the problem widget is right on the main window, and thus will be heavily utilized, and people are going to rely on it to work every single time
2. people who don't know about the bug will quick filter using this widget and will receive a false negative and assume it is correct and base important decisions off of that false negative
3. people who DO know about this bug are going to be incredibly annoyed every time it happens
4. I suspect the fix is going to be relatively simple. I would not be surprised if the function that does the indexing is not being called whatsoever when an email is dragged and dropped into a different folder, so it may be as simple as adding one line of code to call that function.
5. Competing email client Evolution does not have this issue, and it's annoying enough to make at least one person (me) switch.

This bug has been reported separately a number of times, dating back at least 11 years. I have been able to find these so far:
https://bugzilla.mozilla.org/show_bug.cgi?id=569009
https://bugzilla.mozilla.org/show_bug.cgi?id=1777443
https://bugzilla.mozilla.org/show_bug.cgi?id=1245532

It does not appear that anyone has yet attempted to verify or reproduce this bug.