626469 - Archived messages not in global search index when we're not UIDPLUS informed. Need to mark target folder dirty so index sweep is triggered

Status: NEW

(Thunderbird :: Search, defect)

Description

[Francois Trahan]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/8.0.552.237 Safari/534.10
Build Identifier: 3.1.7

I'm using IMAP and I keep my messages on the server and on the local computer for both my inbox and my archives.

When I archive a message, it is not available through global search until I have actually opened that folder (at which point Thunderbird seems to synchronize the content of that folder with my local one). 

It will also work if I go offline and choose to synchronize messages.

Reproducible: Always

Steps to Reproduce:
1. Receive a message
2. Go to inbox (and stay there afterward)
3. Archive the message (i use the keyboard shortcut "a"... if it makes any difference)
4. Search for it in global search (CTRL-K) -> Not going to work
5. Visit the archive folder where it went
6. Search for it again -> Going to work

Actual Results:  
Did not find anything at point 4

Expected Results:  
Should have found it at point 4

It actually makes global search useless unless you actually know in which folder to look...

Workaround : Go offline and have Thunderbird sync all your folders.

What would be usefull : Have a "sync everything" function that doesn't require going offline. Make this "full sync" an option prior to executing global search.

Observed since Thunderbird v3.0 on linux and windows

Comment 1

[Henti Smith]

Same problem here. I only use archived mail when I need to search for things, which makes this bug extremely annoying. Can folders not by syned when thunderbird starts up to fix this problem ?

Comment 2

[Jonathan Protzenko [:protz]]

Andrew, I thought we were supposed to open folders in that situation? But that does seem to correlate with my own experience of the thing...

Comment 3

[Andrew Sutherland [:asuth] (he/him)]

protz, this is just the IMAP move case.  If we're not UIDPLUS informed, then we should mark the target folder dirty and an indexing sweep should be triggered on the target folder.  Additionally, the autosync mechanism should also open the folder if we don't get in there.

In any event, the global search failure mode should be along the lines of "it shows up in the global search results, but when I click on it to show it in the thread pane, it isn't there".  Specifically, I would expect it to get indexed in the Inbox, at which point it should be in the system even if the location information is wrong.

Comment 4

[Henti Smith]

Hi Andrew, Jonathan.

What does this mean for the end user ?

Comment 5

[Wayne Mery (:wsmwk)]

valid bug per andrew 

(I don't know if this is a duplicate or not)

Comment 6

[Wayne Mery (:wsmwk)]

Francios, Henti, (or even protz) can you still reproduce this?
I can't, using currently nightly with my gmail account.

(I didn't find a likely duplicate bug)

Comment 7

[[e-mail]]

Here's the problem as I experience it:

<<AT FIRST>>
- Message is on my IMAP server.
- Thunderbird search can find it, display it, let me click through to it.

<<BUT THEN>>
- I have Thunderbird archive it to a local folder
- Now Thuderbird search may show it on the list but cannot display its content, and I can no longer click through to the message from the search results.

Non-default settings containing the string "archive":
mail.identity.id1.archive_folder = mailbox://nobody@Local%20Folders/local%20archive/Monthly
mail.identity.id1.archive_granularity = 2
mail.identity.id1.archive_keep_folder_structure = true
mail.identity.id1.archives_folder_picker_mode = 1
mail.server.default.archive_granularity = 2
mail.server.default.archive_keep_folder_structure = false

Comment 8

[Henti Smith]

The example Chris has given above is exactly what happens. 

I would really like a way to update the index if this cannot be solved. Deleting the index file is not an option as it really doens't work. 

Henti

Comment 9

[pfc]

I am here after almost 3 years after the last comment was posted and over 4 years after the problem was discovered. Could someone >please< fix that utter idiotic behaviour? If it weren't for the workaround described here, I'd have to dig through a stash that contains e-mails from few years. Oh for crying out loud... why does Thunderbird work like Lotus Notes? The most basic functionality like search also doesn't work there and LN is famous for that. However I can't comprehend how in the world an open-source e-mail client has a broken >search< feature! Since 2011!

Comment 10

[Wayne Mery (:wsmwk)]

(In reply to pufcio from comment #9)
> The most
> basic functionality like search also doesn't work there and LN is famous for
> that. However I can't comprehend how in the world an open-source e-mail
> client has a broken >search< feature! Since 2011!

... because search works well for most users, there is a shortage of developers, and this example is not a common failure case (as demonstrated by no comments in 3 years)  But we'll see what can be done about it.

Comment 11

[Wayne Mery (:wsmwk)]

is gmail really still brain dead in this area? (bug 467841)

Comment 12

[pfc]

(In reply to Wayne Mery (:wsmwk) from comment #10)
 > ... because search works well for most users, there is a shortage of
> developers, and this example is not a common failure case (as demonstrated
> by no comments in 3 years)  But we'll see what can be done about it.

I'm not trying to be malicious here, but will you say that car's brakes "are ok" when you press them 100 times and they don't cause the braking at the 100th press? :) It just can't be explained as "99% of users use POP3, so the search does work correct for most of people" (and even if 99% used POP3 instead of IMAP, it's still no excuse for such behaviour).

I can't see any difference (besides GUI) between TB "one-digit-version" and 38.0a2, so from my POV they're tweaking & changing something under the hood while aforementioned "brakes" are still faulty.

Comment 13

[Wayne Mery (:wsmwk)]

If you have a reproducible testcase that someone else will be able to reproduce on a different computer, then please describe it in excruciating step by step detail, including conditions under which it does NOT reproduce.  It will be helpful to test your steps under an new, clean profile https://support.mozillamessaging.com/en-US/kb/using-multiple-profiles

Comment 14

[pfc]

(assuming one has a clean profile, IMAP account to connect to & some messages in the inbox)
1. start TB
2. use the account creator, connect to the IMAP account
3. wait for TB to download the messages for the inbox
4. create a folder in the IMAP directory structure named "archive" (NOT local, remote!)
5. move some messages from inbox to the "archive" directory
6. try to find something from those messages (topic or whatever)
you won't find a thing, anything placed outside the basic inbox/sent/whatever seems to be invisible for TB search

solution: go to offline mode

after going offline, TB seems to index everything placed anywhere; you can go back online and search will still work as intended, so a "lazy fix" would be to force the indexing TB makes when going offline every time TB downloads new mail (and periodically when it's left idle).

ps. deleting global-messages-db.sqlite and letting TB rebuild the index doesn't fix anything, you have to use the "go to offline" hack, to make it work

Comment 15

[Kent James (:rkent)]

"If we're not UIDPLUS informed, then we should mark the target folder dirty and an indexing sweep should be triggered on the target folder" I don't understand that, isn't the solution just to trigger an update on the destination folder? Does "Additionally, the autosync mechanism should also open the folder if we don't get in there." occur?

Marking the target dirty would cause resync of that entire folder, which seems to be extreme for every time an archive message is created. Or do I not understand the situation?

Comment 16

[Atul Jangra [:atuljangra]]

I have not worked on this area of mozilla-land for a long time. Bug 467841 is still a problem imho