433175 - freeze when specific custom tags are added in a specific order. (A tag, "abc", and tag of substring of the tag, "bc", then loop in saved search of "Tag contains bc")

Status: VERIFIED FIXED

(MailNews Core :: Database, defect)

Description

[Florian]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9pre) Gecko/2008050904 Minefield/3.0pre
Build Identifier: version 2.0.0.14 (20080502)

I moved a message into another folder and:
added tag "egit / jgit" (tag number 3) to it.
added tag "mailinglist" (tag number 7) to it.
tried to add tag "git" (tag number 5) to it, but the UI froze instead.

I forced the application to quit and tried again to add the tag "git" to the message.

Again with the same freeze as result. I tried it a third time with the same result.

As I don't know if it is important:
I have a saved search for messages with the "egit / jgit" tag and a saved search for messages with the "git" tag.

Reproducible: Always

Steps to Reproduce:
1.
2.
3.

Comment 1

[Florian]

I found a workaround:

I removed the "egit / jgit" tag, added the "git" tag and added the "egit / jgit" tag again.

Comment 2

[Florian]

I can reproduce the bug for another message. It seems that the order in which I add the tags makes the difference.

Comment 3

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #0)
> tag "egit / jgit"
> a saved search for messages with the "egit / jgit" tag

What folder name did you assign for the saved search folder?
Does the folder name include "/"?
If yes, Bug 286523 and/or Bug 358208 etc. can occur.
(see dependency tree for meta Bug 124287). 

Comment 4

[Florian]

I can reproduce the bug at a common folder without a "/".

Comment 5

[WADA:World Anti-bad-Duping Agency]

What is defined for tags in prefs.js & search folder? What is put in mail hedader?
 - prefs.js (probably followings)
   user_pref("mailnews.tags.egit___jgit.tag", "egit / jgit");
   user_pref("mailnews.tags.gitgit.tag",      "git");
   user_pref("mailnews.tags.mailinglist.tag", "mailinglist");
 - virtualFolders.dat
 - X-Mozilla-Keys: header of mail source  

Comment 6

[Florian]

-prefs.js (replaced some strings with ***):
user_pref("mailnews.tags.commit.tag", "commit");
user_pref("mailnews.tags.devel.tag", "devel");
user_pref("mailnews.tags.egit.tag", "egit / jgit");
user_pref("mailnews.tags.***.tag", "***");
user_pref("mailnews.tags.git.tag", "git");
user_pref("mailnews.tags.***.tag", "***");
user_pref("mailnews.tags.mailinglist.color", "#009900");
user_pref("mailnews.tags.mailinglist.tag", "mailinglist");
user_pref("mailnews.tags.***.color", "***");
user_pref("mailnews.tags.***.tag", "***");
user_pref("mailnews.tags.***.tag", "***");
user_pref("mailnews.tags.version", 2);
user_pref("mailnews.tags.***.tag", "***");
user_pref("mailnews.tags.***.tag", "***");

related lines from virtualFolders.dat:
------------------------
version=1
uri=mailbox://nobody@Local%20Folders/mailinglists/git
scope=mailbox://nobody@Local%20Folders/mailinglists
terms=OR (tag,contains,git)
searchOnline=false
...
uri=mailbox://nobody@Local%20Folders/mailinglists/egit%2093b37edd
scope=mailbox://nobody@Local%20Folders/mailinglists
terms=OR (tag,contains,egit)
searchOnline=false
...
----------------

Comment 7

[Florian]

Attachment: Screenshot of email header, before adding the "git" tag.

Note that X-Mozilla-Keys has a wrong value. The content of the Tags field is correct.

Comment 8

[Florian]

Attachment: Screenshot of email header, before adding the "git" tag.

Just notice that the header field X-Mozilla-Keys doesn't get updated if I chance the tags of the email.
I made a new screenshot. This time I clicked on another email and then on the email for the bug reproduction in order to get up to date header values.

Comment 9

[WADA:World Anti-bad-Duping Agency]

> user_pref("mailnews.tags.egit.tag", "egit / jgit");

Tag is displayed as "egit / jgit" at UI, but is put as "egit"(xxx of tags.xxx.tag) in X-Mozilla-Keys: header. Label of tag("egit / jgit" in this case) can be altered via UI, but "egit"(xxx of tags.xxx.tag) is not altered on tag label change.

> uri=mailbox://nobody@Local%20Folders/mailinglists/egit%2093b37edd

What name is displayed for this search folder at folder list pane?
How did you create this search folder?
Created as subfolder named "egit%2093b37edd" (or "egit 93b37edd") under "mailinglists" folder? (I can't think so.)
After restart of Tb, what name is displayed for this search folder at folder list pane?

Can you reproduce your problem after delete/re-define of this search folder?
 1. Keep back up of file named "egit%2093b37edd.msf" or "egit 93b37edd.msf"  
 2. Delete this search folder via Tb's UI
 3. Terminate Tb, and delete files related to this search folder
    under directory for "mailinglist"(directory named mailinglist.sbd).
 4. Restart Tb, and define the search folder again(name as "egit")

> Just notice that the header field X-Mozilla-Keys doesn't get updated if I
chance the tags of the email.

Sounds to be similar phenomenon to Bug 398164 or a variation of phenomena of Bug 398164.

Comment 10

[Florian]

The name of the search folder was "egit / jgit". I guess I named it "egit", "jgit" or "egit / jgit" at creation time.

I did the steps you told me. There where three files in the mailinglists.sbd folder related to egit: "egit 93b37edd.msf", "egit 93b37edd" and "egit". I wasn't able to create a new egit search folder, because Thunderbird told me that there is already a folder with that name. Strange is that there was no "egit" folder visible in the UI.

I restarted Thunderbird again and noticed that a "egit" folder appeared in the UI. I clicked on the folder, but the folder content view didn't update to that folder. The folder content view simply stayed at the folder displayed before.

I closed Thunderbird, removed the files "egit" and "egit.msf" (didn't notice that file before) and started Thunderbird again.

I tried again to create the egit search folder, on the first click on "Ok" it did nothing and on the second click it displayed me the message that the folder already exists.

I restarted Thunderbird, removed the (now appeared) egit folder with the UI, closed Thunderbird and removed the egit.msf file.

I tried then to create the search folder "egit" and it worked.
I tried to reproduce the bug in the "git" folder and the bug was still there.

Comment 11

[WADA:World Anti-bad-Duping Agency]

I think your case is a variation of phenomena of Bug 286523.
(problem when '/' in search folder name. no relation to tag)
As seen in Bug 286523, Bug 358208 etc. (see dependency tree for meta Bug 124287), phenomenon seems to depend on OS, file system, return code for file access API and so on, and phenomenon seems to vary by Tb's version or build ID.

FYI.
When Tb 2.0.0.14(en-US) on MS Win-XP SP2(locale=Japanese), it was impossible to create search folder of "egit / jgit", due to following error. 
> Error: [Exception... "Component returned failure code: 0x80550013 [nsIMsgFolder.addSubfolder]"
>  nsresult: "0x80550013 (<unknown>)"  location: "JS frame ::
> chrome://messenger/content/commandglue.js :: CreateVirtualFolder :: line 1062" data: no]
> Source File: chrome://messenger/content/commandglue.js Line: 1062

Comment 12

[WADA:World Anti-bad-Duping Agency]

FYI.
File set of "egit 93b37edd" and "egit 93b37edd.msf" was created when search folder  of "egit" was renamed to "egit / jgit". (Tb 2.0.0.14, MS Win-XP).
And following was written in virulFolders.dat.
> uri=mailbox://b@b.b.b/Test-7/egit%2093b37edd
I think you did rename of search folder.

Comment 13

[WADA:World Anti-bad-Duping Agency]

See also Bug 416888 for funny phenomenon when "/" is involved in local mail folder name. Bug 416888 is not for folder name of "search folder", but it can affect "search folder" case, because it relates to "mail folder name & file name for a mail folder".

Comment 14

[Florian]

I wonder why the freeze bug does still occur.

Comment 15

[WADA:World Anti-bad-Duping Agency]

> I wonder why the freeze bug does still occur.
Are you talking about following case?
> I tried to reproduce the bug in the "git" folder and the bug was still there.

Freeze(hang due to CPU 100%. probably loop) was re-produced with Tb 2.0.0.14.
When Tb trunk, "add tag" did nothing, and no freeze.
> 2 tags of egit, git ("git" is substring of "egit")
>  user_pref("mailnews.tags.egit.tag", "egit");
>  user_pref("mailnews.tags.git.tag",  "git");
> virtualFolders.dat
>  uri=mailbox://b@b.b.b/Test-7/git
>  scope=mailbox://b@b.b.b/Test-7
>  terms=OR (tag,contains,git)
>  searchOnline=false
(Test scenario)
 1. Copy two mails to search target folder(local mail folder)
 2. mail-1 : Add tag of "git", then add tag of "egit" => No problem
 3. mail-2 : Add tag of "egit" => OK,
             then add tag of "git"
             => CPU 100% and hang (Tb 2.0.0.14)
             => Nothing happens("git" is not added) (Tb trunk 2008/5/08 build)

Workaround: Don't use tag name(xxx of tags.xxx.tag) of substring of other tag.
            (use "Xgit" or "gitX" instead of "git" in your case)

Comment 16

[Florian]

(In reply to comment #15)
> Are you talking about following case?
yes

Comment 17

[WADA:World Anti-bad-Duping Agency]

Adding "substring" in bug summary, for ease of search.

Comment 18

[WADA:World Anti-bad-Duping Agency]

FYI. I've opened Bug 439548 for trunk only issue in Comment #5(unable to add tag).

Comment 19

[WADA:World Anti-bad-Duping Agency]

I've opened Bug 439835 for opposite sequence case, (1) "X-Mozilla-Keys: abc bc" already exists, (2) then define saved search folder of "Tag contains bc", to provide problem re-creation procedure with Tb trunk.

Comment 20

[WADA:World Anti-bad-Duping Agency]

Bug 439548 has been fixed(and Bug 439835 has been resolved by it too). 
So we can re-produce Bug 433175(this bug) with Tb Trunk.
> Name=Thunderbird,Version=3.0a2pre,BuildID=2008062003 on Win-XP SP2

Comment 21

[Karsten Düsterloh]

Attachment: fix nsMsgSearchTerm::MatchKeyword

JFTR: SeaMonkey trunk is also affected; I didn't check any branches.

The problem is this line in the code section for matching the contains/doesn't contain rules, which can't *ever* have produced correct results:
<http://bonsai.mozilla.org/cvsblame.cgi?file=/mozilla/mailnews/base/search/src/nsMsgSearchTerm.cpp&rev=1.161&mark=1402#1383>

This patch fixes this similar to bug 439548, plus some readability cleanup.

Comment 22

[Karsten Düsterloh]

Attachment: more efficient nsMsgSearchTerm::MatchKeyword fix

Comment 23

[David :Bienvenu]

Comment on attachment 326128 [details] [diff] [review]
more efficient nsMsgSearchTerm::MatchKeyword fix

thx, Karsten.

Comment 24

[Karsten Düsterloh]

Landed on CVS trunk aka 1.9.0.
I don't think it's necessary to port to 1.8 branch, but I'm open to arguments.

Comment 25

[WADA:World Anti-bad-Duping Agency]

VERIFIED with Name=Thunderbird,Version=3.0a2pre,BuildID=2008062503 on Win-XP SP2.