1953038 - gloda search only returns mails dated year 2000 or later

Status: UNCONFIRMED

(Thunderbird :: Search, defect)

Description

[Bruce Allen]

Steps to reproduce:

[1] Type "Barish" into search bar at top of main mail window.
[2] Click on "Show results as list"
[3] Sort list by date
Earliest message found is from the year 2000.

If instead I use the EDIT->FIND->SEARCH MESSAGES dialog I find messages dating back to the mid 1990s.

This pattern is repeated with other search patterns. The search bar does not return results dated earlier than 2000, whereas using the EDIT->FIND->SEARCH MESSAGES correctly returns messages from earlier dates.

Actual results:

Returned list of mails dated year 2000 or later

Expected results:

Returned list of mails dates from earlier times as well.

Comment 1

[Magnus Melin [:mkmelin]]

Find | Search Messages uses the traditional search
The search bar relies on the gloda search index. I guess this one has a bug then...

Comment 2

[Anje]

Does rebuilding the global database fix the issue?
https://support.mozilla.org/en-US/kb/rebuilding-global-database

Comment 3

[Bruce Allen]

No, unfortunately it does not.

In fact I had some serious problems with searching, in part because I have a large email collection (69GB) and in part because one of my IMAP inboxes contains 1700 subfolders. This puts some stress on the search system.

Because I had problems, I did a complete clean reinstall of TB, by creating a new profile, populating my three email accounts, then waiting a couple of days for the 69GB of mail to download and for the 2GB global-messages-db.sqlite to rebuild.

After that process had completed, I did some testing. What I found was interesting: all of my search bar queries would only return results dated 2000 or later. Whereas, if I used the EDIT->FIND->SEARCH MESSAGES dialog, then I also found emails from earlier.

I'm happy to carry out experiments and tests, produce log files, and so on, but need clear instructions.

Comment 4

[Wayne Mery (:wsmwk)]

Bruce, is the problem solved in beta 138? If yes, then the fix came via bug 1916063.

Comment 5

[Bruce Allen]

I will download and install beta 138 to test this. I'll do that in the next 24h and report back here.

A question: how do do this? The simple way is for me to replace my current stock TB installation with the beta 138 version. Is that what you suggest? Or should I try using a 'cp -R' to duplicate my TB profile/mail files, then rename that copy with a different profile and expose that to the beta 138? Or is there another approach that you would suggest?

Comment 6

[Bruce Allen]

PS: looking the beta 138 release notes here:
https://www.thunderbird.net/en-US/thunderbird/138.0beta/releasenotes/
I see no mention of "search problems fixed".
Are you sure that the fixes are in beta 138?

Comment 7

[Bruce Allen]

I will download and install beta 138 to test this. I'll do that in the next 24h and report back here.

A question: how do do this? The simple way is for me to replace my current stock TB installation with the beta 138 version. Is that what you suggest? Or should I try using a 'cp -R' to duplicate my TB profile/mail files, then rename that copy with a different profile and expose that to the beta 138? Or is there another approach that you would suggest?

PS: looking the beta 138 release notes here:
https://www.thunderbird.net/en-US/thunderbird/138.0beta/releasenotes/
I see no mention of "search problems fixed".
Are you sure that the fixes are in beta 138?

Comment 8

[Wayne Mery (:wsmwk)]

(In reply to Bruce Allen from comment #7)

I will download and install beta 138 to test this. I'll do that in the next 24h and report back here.

A question: how do do this? The simple way is for me to replace my current stock TB installation with the beta 138 version. Is that what you suggest? Or should I try using a 'cp -R' to duplicate my TB profile/mail files, then rename that copy with a different profile and expose that to the beta 138? Or is there another approach that you would suggest?

PS: looking the beta 138 release notes here:
https://www.thunderbird.net/en-US/thunderbird/138.0beta/releasenotes/
I see no mention of "search problems fixed".
Are you sure that the fixes are in beta 138?

That fix came in 133. So actually, an easier path is to install 137 https://www.thunderbird.net/en-US/thunderbird/all/?release=release over top of 128.

Comment 9

[Wayne Mery (:wsmwk)]

Hmm, I'm NOT thinking clearly. [and also not typing well] I was just thinking you are on esr, but you are on release.

Best if you just stay with what you have. Then install https://addons.thunderbird.net/en-us/thunderbird/addon/glodaquilla-ng-index-ondisk/ and see if the messages you are not finding are a) indexed, as indicated by the g_id column, b) has message body downloaded, which indicated by the g_offline column.

Comment 10

[Bruce Allen]

Attachment: Result from Message Search, showing emails from 1990s in the Inbox/barish folder

Result from Message Search, showing emails from 1990s in the Inbox/barish folder

Comment 11

[Bruce Allen]

Attachment: Screenshot from quick search bar, not displaying emails from before 2000. Note that contents are coming from the correct folder!

Screenshot from quick search bar, not displaying emails from before 2000. Note that contents are coming from the correct folder!

Comment 12

[Bruce Allen]

OK. I updated to the current release channel version of TB, which is 137.0.1 (64-bit) on Mac OS X.
I installed the glodaquilla-ng-index-ondisk add-on.

I then did a search using the search bar, after turning on the additional columns. That only returned results dated after 2000. See second screeshot I uploaded. Then I did a search using the Message dialog. That returned results from before 2000. See first screenshot.

I was concerned that perhaps the search bar search was not going into subfolders. But you can see clearly that it is looking there, from the location column.

I'll upload one more image showing the full tool pane.

Comment 13

[Bruce Allen]

Attachment: Showing full pane view of Search Bar results. Note missing messges from the 1990s.

Showing full pane view of Search Bar results. Note missing messges from the 1990s.

Comment 14

[Wayne Mery (:wsmwk)]

Anything showing in Error Console (ctrl+shift+J) after startup related to search, indexing or gloda?

Comment 15

[Bruce Allen]

(In reply to Wayne Mery (:wsmwk) from comment #14)

Anything showing in Error Console (ctrl+shift+J) after startup related to search, indexing or gloda?

No, nothing. The only entry in the Error console is a log message which appears when I sort the results by date. I've cut-and-pasted this below. I think it is inconsequential.

WebExtensions: loadinto tab.browser ChromeWindow about:blank implementation.js:1432

Comment 16

[Wayne Mery (:wsmwk)]

Any better now with version 150 and deleting the global index, to let it rebuild?