783991 - Global search, sorted by date: clicking on subject of found message in faceted search results opens a blank tab instead of message tab

Status: REOPENED

(Thunderbird :: Search, defect)

Description

[Leo.Maathuis]

User Agent: Mozilla/5.0 (Windows NT 5.1; rv:14.0) Gecko/20100101 Firefox/14.0.1
Build ID: 20120713134347

Steps to reproduce:

I typed search words in the appropriate window.
A list of mails appeared. Since default presentation is not "sorted on date" (which I nearly almost prefer) I hit "date".
I see messages listed on date and I see some of the content and the date for the message.
I hit the title of the message I would want to read.



Actual results:

In the new window where one would expect to see the content NOTHING shows.


Expected results:

The content of the chosen message should have been displayed.

My workaround is to look in the unsorted Inbox at the date of the searched for message to display it this way

Comment 1

[Luke Bodeen]

I see this bug also in the same way as described.

Comment 2

[Wayne Mery (:wsmwk)]

Do you still see this problem?

Comment 3

[Leo.Maathuis]

Hi,
I don't have the problem any more. I suppose a new release solved the problem, Besides my OS is now W7
Thanks

Comment 4

[Wayne Mery (:wsmwk)]

Leo, Thanks for the update

Comment 5

[WBT]

I just observed this bug and can refile it with more detail.  Here's what happened:
1. I sent a message to X, to which a response was expected.
2. An appropriate amount of time passed.
3. X sent a note indicating nonreceipt of the message in 1.
4. I did a global search for terms I knew I'd included in the subject line.
5. The desired message came right up near the top of the search.  The subject line, sender, date, attachment, quoted text, etc. all looked right.
6. I clicked on the search result (hyperlinked header) expecting to see the message.
7. Instead, no message showed up, just an empty message list and default "Welcome to Thunderbird...why we need donations" pane in the position where the e-mail should have been.  This was very concerning.  I'd spent a lot of time on that message - had Thunderbird lost it? 
8. I went back to the global search results tab and noted that this view does not give the location (folder) where the message was found, so I clicked "Open email as list" to try to get that information.
9. Same result as in 7; empty message list shown.
10. I logged in to a webmail view of the IMAP mailserver, and verified that the message was in the Sent folder (which is a subfolder of INBOX).
11. I then returned to Thunderbird, and clicked to view the Sent folder. 
12. Thunderbird showed the Sent folder with messages up until a date prior to the day the message in question was sent.
13. After a moment, Thunderbird showed the more recently sent messages, including the one in question, in the list.
14. Returning to the global search results tab, when I clicked on the target message again, it showed the message as originally expected (i.e. different result than in 7).  
15. Returning to the global search results tab, when I clicked Open Email as List, I got a message list including the target message (i.e. different result than in 9).
16. Returning to the Sent folder, I used Tools-> Run Filters on Folder to move the message to a folder designated for correspondence with X, which I thought should have been run automatically after sending.  Other messages went to other folders as well, according to filter rules.
17. I repeated the global search for the same terms.
18. The target message disappeared from search results.
19. I opened the folder designated for correspondence with X.
20. At first, the target message was not there. 
21. After a few moments (for communication with the IMAP server), the target message (and some others) appeared in the list, sorted in at their proper position by date (not the most recent). 
22. I repeated the global search.
23. The target message appeared in the search results, but with no quoted text, even though the search terms did appear in the body of the message as well as the subject line.
24. I clicked on the search result for the target message.  
25. A message list pane opened, with one message (the target), and a blank pane below.
26. After a moment (for server communication), the message loaded in the preview pane.
27. I repeated the global search, with the same result as in 23.
28. I went back to the folder view and clicked to view the target message.  It loaded after a moment (for server communication).
29. I repeated the global search, again with the same results as in 23.  I tried several more times but was unable to get quoted text to show up in the search results for the target message, by the time I submitted [reopened] this bug.  

I've had this happen to me several times, and it's always very concerning to lose a message like that.  In some cases, I was never able to find the message because I didn't know which folder it was in to explicitly check there.  However, I haven't previously been able to nail it down as specifically as in this report. 

It appears that what fixed the problem for Leo was not a new release or OS upgrade, but merely that his target message had been downloaded from the server on explicitly viewing the folder it was in.  

It appears that there is an issue here with Thunderbird not indexing location information about where a message can be found when indexing the rest of the message, and not giving the user any error message or information about what's happened when it tries to open a search result that hasn't been downloaded from the server (nor does it just download and display the message).  

I would like to at least see a Location field added (or addable by the user) to the default search results view, to aid in the workaround for this bug.

Comment 6

[WBT]

30. In the Message Filters, I opened the filter which sends correspondence with X to its designated folder.
31. I saw a new option at the top of Filter Rules under "Apply Filter when:" which said "After Sending" which was unchecked.  This was new compared to the old setup in which there was one option for running all filters when a message was sent, and is disappointing to see that I'll have to go through hundreds of filters and manually make that change on each one in order to restore the functionality I had before.  
32. I checked that option for this particular filter.
----
33. On the message pane showing the target message, I right-clicked and chose Edit As New.
34. I prepended "Resending: " to the subject line and prepended some text to the message body (including the word "resending").  I did not change recipients and left the one attachment.
35. I sent the message.  At the end of the sending, the box said it was copying the message to the folder designated for correspondence with X.
36. I looked at that folder which was already open in the main message list, and didn't see the message I'd just sent.  This was concerning. 
37. I did a global search for "resending" and sorted by date.  The message from 35 was not there. Again, this was concerning. 
38. I clicked to view another folder (not "Sent.")
39. I clicked back to view the folder I had just been viewing.
40. After a moment, the new message appeared in the message list. 
41. I repeated the global search.
42. This time, the new message was in the list, but just the header information with no quoted body. 
43. I went back to the message pane and clicked on the new message to show it in the preview pane. 
44. As with step 29, I could not get quoted text to appear with the search result for the new message.  However, repeating the search from step 29 did give quoted text.  
45. I waited a bit over half an hour.
46. Repeating the global search for “resending” showed the new message including quoted text.

I would expect Thunderbird to load a message that was just sent and make it findable by search, and when a result appears in search, it will also appear when viewing search results as a list, as well as be viewable when clicking on a search result directly.  If these expectations won’t be met, it’d be nice to have some warning or explanation about how to still access the message, including having the location appear in the search results list.  

I hope this level of detail helps others reproduce the conditions or at least know how to look for the bug. 

Also, if someone is experiencing something like this, managed to find this bug report, and needs a temporary, unintuitive workaround, try clicking through all of the e-mail folders to load new messages in each, visiting whatever folder was originally open at least twice.

Comment 7

[Wayne Mery (:wsmwk)]

Please post what you see in tools | error console

Comment 8

[WBT]

The following two items appear in the error console at step 6 in comment 5. Repeating that click, the same pair continue to reappear.  Both are icon-tagged as Warnings.

Item 1:
Warning: Content Security Policy: Directive 'frame-src' has been deprecated. Please use directive 'child-src' instead.

Item 2:
Timestamp: 1/26/2017 3:29:00 PM
Warning: Expected media feature name but found '-webkit-min-device-pixel-ratio'.
Source File: https://www.mozilla.org/media/css/thunderbird-start-bundle.131bf653d838.css
Line: 2, Column: 706
Source Code:
[dir="rtl"] main>div:first-of-type{float:left}}@media(min-width:640px) and (max-width:799px){header,section{width:70%}footer{position:absolute;top:120px;width:30%}[dir="ltr"] footer{right:0}[dir="rtl"] footer{left:0}}@media(min-width:480px) and (max-width:639px){footer li{width:50%;min-width:10em}[lang|="en"] footer li{width:33%}[dir="ltr"] footer li{float:left}[dir="rtl"] footer li{float:right}}@media(min-width:480px){header h1{display:table-cell;-moz-padding-start:110px;padding-inline-start:110px;height:100px;vertical-align:middle}header h1:before{width:100px}}@media(max-width:479px){header h1{padding-top:110px;text-align:center}header h1:before{width:100%;background-position:center top}}@media(-webkit-min-device-pixel-ratio:1.5),(min-resolution:1.5dppx),(min-resolution:144dpi){header h1:before{background-image:url("/media/img/thunderbird/logos/release-200.cc5a08767ff7.png");background-size:100px 100px}html[data-channel="earlybird"] header h1:before{background-image:url("/media/img/thunderbird/logos/earlybird-200.d696a95c5943.png")}html[data-channel="daily"] header h1:before{background-image:url("/media/img/thunderbird/logos/daily-200.7e697927422d.png")}}

The following three items were found a bit more than half an hour earlier, showing up as the remainder of the error console.  Items 3 and 4 have the same text but the first is an Error and the second is a Message.

Item 3:
autosyncActivities	ERROR	onDownloadCompleted: TypeError: this._syncInfoPerFolder[folder.URI] is undefined
Source File: resource:///modules/gloda/log4moz.js
Line: 691

Item 4:
autosyncActivities	ERROR	onDownloadCompleted: TypeError: this._syncInfoPerFolder[folder.URI] is undefined

Item 5: 
Error: TypeError: this._syncInfoPerFolder[folder.URI] is undefined
Source File: resource:///modules/activity/autosync.js
Line: 308

Comment 9

[Ryan Lee Sipes]

I'm experiencing this in 2024, on Thunderbird Daily

Comment 10

[Wayne Mery (:wsmwk)]

(In reply to Ryan Lee Sipes from comment #9)

I'm experiencing this in 2024, on Thunderbird Daily

Ryan, Are you saying this only recently started happening for you?

Comment 11

[Ryan Lee Sipes]

As far as I can tell, this just started, still present on Daily.

Comment 12

[Ryan Lee Sipes]

Clicking on other messages in the list, eventually one did open. But many others did not. Even stranger.

Comment 13

[Wayne Mery (:wsmwk)]

Next time you see this, check the error console.

I am not able to reproduce using today's build.

Comment 14

[Alessandro Castellani [:aleca]]

I was able to reproduce this and got this error in the console:

Couldn't restore state: Component returned failure code: 0x80070057 (NS_ERROR_ILLEGAL_VALUE) [nsIMsgDBView.findIndexOfMsgHdr] NS_ERROR_ILLEGAL_VALUE: Component returned failure code: 0x80070057 (NS_ERROR_ILLEGAL_VALUE) [nsIMsgDBView.findIndexOfMsgHdr]

I was able to confirm that the issue is Gloda showing messages I previously deleted in the search result.
Could be maybe related to bug 1898780.

Ryan, would you be able to confirm if the messages that you can't open are previously deleted messages?
Ben, can you investigate this and see if Gloda is maintaining some weird cache and returning results that don't exist?

Comment 15

[Ben Campbell]

(In reply to Alessandro Castellani [:aleca] from comment #14)

I was able to confirm that the issue is Gloda showing messages I previously deleted in the search result.
Could be maybe related to bug 1898780.

Snap! I didn't read that last line carefully enough and literally just wrote: "This sounds suspiciously similar to Bug 1898780" :-)

Ben, can you investigate this and see if Gloda is maintaining some weird cache and returning results that don't exist?

There was a bug in gloda that meant messages were not being removed from the index after compaction.
Bug 1898780 saw a patch land to fix it, so I think it's probably OK now. If not, NI me again and I'll investigate!

I'm not quite sure what's changed in the folder compaction which might have caused this bug to start being a problem - as far as I can tell, the bug has been there for a decade or so, so it really should never have worked correctly!
My guess would be the old code had some brittle reliance on the order/timing of NotifyCompactionComplete() notifications... that kind of thing used to be really haphazzard and scattered all over the place.

Comment 16

[Ryan Lee Sipes]

Following up on this, so I still see this bug, but it looks VERY MUCH like the stuff that won't load these days are deleted messages. I know we were aware that might be a problem, so confirming this. On Daily, up-to-date 130a1 (2024-07-09), Windows 11

Comment 17

[Ryan Lee Sipes]

NI'ing Ben Campbell and Alessandro for visibility.

Comment 18

[Alessandro Castellani [:aleca]]

Nice, thanks for confirming it.
Ben and I already talked about this yesterday and we were exploring options.
This will 100% be solved by the global database since it will remove the need of a cache altogether, but nonetheless Ben will explore if there's an easy win to quickly fix this and properly clear cached messages and prevent them to show up in search results.

Comment 19

[Ben Campbell]

Have we got any (recent) steps to reliably reproduce this?

I've been trying to get it to happen with a fresh profile being fed from a local IMAP server with lots of fake messages.
But I haven't seen any oddness with deleting messages from my Inbox- as soon as they are deleted, they disappear from subsequent search results.
Doing a compaction immediately after deletion doesn't seem to help trigger the bug either.

The only oddness I see is that not all the messages I expect appear in the search results, but I think that's just gloda being really slow to index stuff in the background (or doing it in too small batches). I was doing 10000 or so messages, on a debug build.