Global search/indexing is limited to first 20k of message. Does not find text in large message.
Categories
(Thunderbird :: Search, defect)
Tracking
(Not tracked)
People
(Reporter: kowalsky.jens, Unassigned)
References
(Blocks 1 open bug)
Details
User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36 OPR/64.0.3417.92
Steps to reproduce:
I am using TB 68.2.2 (64bit) on Win7. I also see the same issues on Win10/Ubuntu18.04.3 with 64 and 32 bit Versions.
I’ve set up three/four mail accounts (IMAP – Google and others with each couple of thousand mails) and want to find an Email containing a certain string.
Therefore, I use global search (ctrl+k) and quick search (ctrl+shift+k) but I do not get correct results.
Actual results:
In detail I want to find the string “codingkids” or “cgi-Datei=” but NO result shows up! If I search for “Hallo” it shows some results, but I guess not all. If I log in into gmail web, I can find the string and mail easily and with the title and date of the mail I can now find it in TB.
So I tried to reindex (https://support.mozilla.org/en-US/kb/rebuilding-global-database) but no success.
Expected results:
Correct and complete search results.
What is wrong with the search/indexing function in TB, I’ve faced such problems a long time with several TB versions and I’ve seen a lot people who have the same issues. Now I’m annoyed enough, to open a bug report to find the root cause, since this is a absolute MUST WORK function!
|
||
Comment 1•6 years ago
|
||
I am able to find both strings in the bugmail I received for this bug report. But, if I reply to this bugmail and send it to myself, the strings are not found. This points out one of two perhaps unexpected search behaviors: 1. quoted text is not indexed, 2. only the first 20k is indexed (IIRC).
Do either of those conditions exist for the message your are not finding?
|
Reporter | |
Comment 2•6 years ago
|
||
Hey, thanks for the fast reply!
It’s an Email I send, so if I understood you correct, only my original mail should be found, but not the reply with quoted text.
Thanks to your 20k hint I found:
bug 543737 gloda needs to avoid indexing ridiculously large message bodies (especially from offline store corruption) (indexing stops at 20k per message part)(not fixed in v3.0.x)
But after reading I only understand that large message are a problem, not what the 20k exactly means :).
Its a Mail with a 100kB attached cgi file. The Email body (source code including header up to the attachment) contains less than 2,5k letters, the whole eml file is 137kB. I guess this should not be too much? Can this limit for indexing be modified by a user somewhere in the config? If you like, I could send yout the eml file via email?
kr and nice weekend
jens
|
|
Reporter | |
Comment 3•6 years ago
|
||
Nobody has a solution? Everybody is happy with the actual, not working, search function?
|
|
||
Comment 4•6 years ago
|
||
As can be seen in the code there is no user accessible run time preference. You must alter const MAX_SANE_BODY_PART_SIZE = 20 * 1024; and compile the source.
|
|
Reporter | |
Comment 5•6 years ago
|
||
Hello, unfortunately that does not help.
As I’ve explained in my last post, the size of the mail without attachment is too small to hit that criteria. But thanks for showing the correct place in the source which clearly shows the 20k is 20x1024byte (20kB or correct 20KiB). Maybe I should have mentioned that my installs are all 1-10 years old and with copied profiles (from old PCs to newer ones since very old Versions of TB) -> That brought me finally to a solution.
However I forwarded the mail again to me and send it in 5 time to me (copied the text and attachment, right click and create new mail from the old one) .. always same result : the old email is not found but the resend ones can be found. So I checked the source of the mails but now problems. Without attachment, the mail is 2,5kB -> SO SIZE DOES NOT MATTER.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!BUT THE EMAIL IS STILL NOT INDEXED!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
I asked one of our admins and the original answer was:
“yes yes that’s normal with thunderbird it does not search correct .. make new profile or use outlook” 😊
So I did some more research, because I know this is not an unknown issue for you and Wayne you should know about that: (somehow after updates the database versions are corrupted ..) eg:
https://bugzilla.mozilla.org/show_bug.cgi?id=541349
The solution I finally found here:
http://forums.mozillazine.org/viewtopic.php?f=39&t=1737905
I deleted the “global-messages-db.sqlite” and all “*.msf” files in the profile folder. Than Restart Thunderbird and it begins to download ALL emails again. After some hours the corresponding folder of my account with the missing emails was synced and search works again!!! Problem: my profile size is not 13GB but 22GB now (I guess I have to delete the whole message folder).
BTW Another (maybe small) issue I found:
During updating all the emails it shows “Mail 100 of 99 indexed” 😊
Are you sure this is only an optical issue and not messing up the indexing if you have wrong list-lengths ??? (I can send screenshot if needed).
My request to you is the following:
Since I've seen so many people complaining about TB search you should give a clear advise how to handle those things:
Under https://support.mozilla.org/de/kb/die-globale-suchdatenbankdatei-neu-erstellen
It should be added to remove *msf files or maybe better the whole message folders (like explained here: http://kb.mozillazine.org/Transferring_data_to_a_new_profile_-_Thunderbird ) if the search does not work correctly after reindexing! Especially after Version updates. (Maybe you should do something like that automatically during updates if you change the database structure or version)
Also the limitations of the search should be mentioned there:
- 20KiB limit (header only )
- No quoting
- ...
Last Request:
- Please do not close threads with peoples problems without solving them and without taking it serious!!!
kr Jens
|
|
Reporter | |
Comment 6•6 years ago
|
||
Whats wrong with you guys? I know this is free software but you are completely ignoring users and problems???
AGAIN:
THIS TOPIC IS NOT RESOLVED AND NOT INVALID
ARE YOU NOT ABLE OR NOT WILLING TO HELP ???
AGAIN ONE OBVIOUS EXAMPLE: ???HOW CAN YOU INDEX 100 of 99 MAILS ??? CAN YOU? WHY DONT YOU COMMENT OR CHANGE YOUR HORRIBLE PROGRAMMING ISSUES?????
You did not took any of my hints how to overcome this issue! None of the websites is giving a hint to solve the issue even after almost 2month!!! Are you really ignoring most of your users and their problems and even the many Add on-programming guys?
YES REALLY you AGAIN made it! Almost all add-ons are incompatible in version 68.4.1!!
GOOD JOB GUYS ….
HONESTLY IM OUT OF THUNDERBIRD .
TO EVERYBODY reading this. Good alternatives are “Sylpheed” (super nice, fast, WORKING GOOD, back to basics) and the many following links to free email clients:
https://sylpheed.sraoss.jp/en/
https://www.claws-mail.org/
http://trojita.flaska.net/
https://pawsa.fedorapeople.org/balsa/
https://wiki.ubuntuusers.de/Geary/
THANKS TEAM THUNDERBIRD FOR DESTROYING A FORMERLY NICE SOFTWARE WITH YOUR ARROGANCE AND INCOMPETENCE!
BTW: normally in a forum or blog there should be a button to re-edit own comments… :) but thats just the opinion of a unneeded stupid user ;)
|
|
||
Comment 7•1 month ago
|
||
In retrospect, this should not have been closed. Sorry, my error.
Description
•