140613 - Single thread split into two or more threads

Status: VERIFIED FIXED

(MailNews Core :: Database, defect)

Description

[Ian Pottinger]

From Bugzilla Helper:
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:0.9.9+)
Gecko/20020426
BuildID:    2002042608

In not sure if this is a true bug or if there is some logic here that I'm
overlooking.  Anyway, sorting by subject will always group together a set of
messages sharing the same subject. However, sorting by thread will sometimes
split that same set of messages into two or more distinct threads.

Reproducible: Sometimes
Steps to Reproduce:
1. Select a mail folder that contains a thread of messages.
2. Sort by subject
3. Sort by thread

Actual Results:  2. Sort by subject groups the threaded messages together as
expected.
3. Sort by thread might splits a thread into two or more distinct threads.

Expected Results:  All messages of the same thread should be grouped together in
a single thread.

Comment 1

[Ian Pottinger]

Attachment: Image of thread splitting

Comment 2

[Ian Pottinger]

I was able to find a set of steps that will reproduce this bug.  It is seems
that  if mozilla is exited and restarted after marking all the messages in a
thread then any new messages will be placed in a new thread instead of being
appended to the original.

From Bugzilla Helper:
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.0.0+)
Gecko/20020523
BuildID:    2002052308

Reproducible: Always
Steps to Reproduce:
1. email the a message with subject "AAA" 3 times to an account.
2. email the a message with subject "BBB" 3 times to the same account.
3. verify that, when recieved, the treaded messages are properly group as:

   Q  V [ ] AAA
         |
         +- [ ] AAA
         |
         +- [ ] AAA

   Q  V [ ] BBB
         |
         +- [ ] BBB
         |
         +- [ ] BBB

4. >>> MARK THE MESSAGES AS READ <<< *REQUIRED TO REPRODUCE BUG*
5. >>> EXIT AND RESTART MOZILLA  <<< *REQUIRED TO REPRODUCE BUG*
6. email the a message with subject "AAA" to the above account.
7. email the a message with subject "BBB" to the above account.

Actual Results:
when recieved, the treaded messages are improperly group as:

   Q  V [ ] AAA
         |
         +- [ ] AAA
         |
         +- [ ] AAA

   Q  V [ ] BBB
         |
         +- [ ] BBB
         |
         +- [ ] BBB

        [ ] AAA

        [ ] BBB


Expected Results:
when recieved, the treaded messages are properly group as:

   Q  V [ ] AAA
         |
         +- [ ] AAA
         |
         +- [ ] AAA
         |
         +- [ ] AAA

   Q  V [ ] BBB
         |
         +- [ ] BBB
         |
         +- [ ] BBB
         |
         +- [ ] BBB

Comment 3

[Aleksey Nogin]

See also Bug 64476, "<Offtopic>" in comment 86 and a few responses to it
afterwards. In short, I see this on BuildId 2002051816 on RH Linux 7.2 and I
believe that this is a somewhat recent regression

Comment 4

[David :Bienvenu]

not a recent regression at all. This is a dup of a fairly old bug - I'll go find
it. The problem has been that we don't have any reproducible steps - I'll see if
the offtopic comments help.

Comment 5

[David :Bienvenu]

bug 105013 is the dup - if these steps work for me in reproducing the problem,
I'll mark that one as a dup of this one. I doubt that marking the thread read
actually causes this problem, but I'll try it out.

Comment 6

[David :Bienvenu]

these steps were not reproducible for me. I do have message filters firing; I'll
try turning those off. I think my comments in bug 105013 are relevant. It may be
that if you delete these threads once and retry the test, it happens.

Comment 7

[Aleksey Nogin]

I was able to reproduce using (essentially) the steps in comment #2 using both
NNTP and POP. I am running BuildID 2002052314 (compiled on RHLinux 7.2 optimized
with -O2, non-debug, with bienvenu's NNTP db sync patch from Bug 64476) running
on RHLinux 7.3 (all updates). 

The only modification for steps I used is in step 5 to quit Mozilla, perform
steps 6 and 7 and only then restart Mozilla. When using NNTP, in steps for I
tried both marking the whole thread as read (using the "R" accelerator) and
simply going through messages by pressing the "Next" button repeatedly - didn't
make any difference.

But here is one very interesting observation - when I succesfully reproduced the
bug, I used another Mozilla (exact same binaries as above) on another machine
(RHLinux 7.2 + all updates) to send all 8 test messages. However when I tried to
reproduce the bug by sending messages (both to IMAP and POP accounts) using the
/usr/bin/mail on localhost, the threading worked correctly!!!

In other words: send messages using *another Mozilla* when trying to reproduce!
Weird...

Comment 8

[Ian Pottinger]

Sorry for the error in my test case.  Indeed [e-mail] is correct in
that marking the messages read is *not* required to reproduce this bug.

I tried again today after reading his comments and was able to reproduce this
bug with and without marking the messages read.  I tried to be as meticulous as
possible, sending messages between pop and imap accounts and marking and not
marking messages read.  Consistently, every time I restarted mozilla (with
QuickStart disabled) the messages failed to append themselves to the original
threads - save one instance when the BBB message did but the AAA message did not.

If you have any suggestion on how I could better test this bug or how I could
better describe how to reproduce it, please email me.  I dislike this bug
immensely.  For some reason I can handle Big Bugs, but it seems to be the little
things like this that really get under my skin.

Comment 9

[David :Bienvenu]

It's possible that this bug is more easily reproducible with release builds than
debug builds (which could point to accessing an unitialized variable somewhere,
or a problem with some boolean logic). I remember when I tried to reproduce it
before, I saw it a couple times with a release build, but couldn't get it to
happen in a debug build.

Here's something you can try:

1. Set up a mail filter that will filter your mail to a brand new, initially
empty folder.
2. Repeat your steps and see if the problem still occurs.

My suspicion is that the bug depends on the state of the db (the .msf file), so
starting with a new folder will remove that variable
Also, when you guys are sending your test messages, are you using a template
message (in a template folder), or composing messages from scratch each time? It
shouldn't matter (I've been using a template) but you never know.

Comment 10

[Aleksey Nogin]

I composed from scratch every time.

Comment 11

[Aleksey Nogin]

> which could point to accessing an unitialized variable somewhere

Well, at least all of the potentially relevant "might be used uninitialized"
warnings were fixed (bug 126460), so if something is uninitialized, it must be
one of the things gcc does not check for...

Comment 12

[David :Bienvenu]

I was able to recreate this by sending individual messages (not using template
msg, not that it should matter) to a new pop account. What seems to be going on
is that the FindRow call to find threads with the same subject is failing
because the mork rowspacemap is empty. Why that would be is a mystery to me.
What's supposed to happen is that the first time we try to do a findrow, the
index (or row space map) is automatically created.

Comment 13

[Ian Pottinger]

FYI: all results the same reguardless of whether using newly composed vs
template messages, or whether filtered to a new folder vs not.

Comment 14

[Ian Pottinger]

While your looking at this bug, take a peek at bug 140517 which deals with
incorrect date sorting.  I think it might share the same root cause with this. 
IIRC, it affects threads as well as subject sorted messages but I'll have to
comfirm that.  I will do some testing to see if a modified version of the above
test case reproduces that bug.

Comment 15

[David :Bienvenu]

removing keyword - I don't think this is a regression.

Comment 16

[David :Bienvenu]

I don't think bug 140517 is related, and isn't neccesarily even a bug (I can't
tell w/o more info).

More detail about what's going on here: the problem arises when you get two
messages with the same subject at one time and then close the app. After that
point, the db is messed up such that the table of threads that we use to map
thread subjects to threads is empty, so subsequent messages that we need to
thread by subject don't get threaded. So I believe the steps to reproduce this
can probably be reduced to simply:

1. Send two messages with subject AAA.
2. Get new mail.
3. Shut down
4. Restart, send one message with subject AAA, and get new mail.

I don't know why the table is getting emptied yet.

Comment 17

[David :Bienvenu]

OK, I think I finally understand what's going on here - it's not a mork bug, but
a bug in the msg db code - we create rows with the thread table scope for each
new thread, but never actually created a table for the rows, or add the rows to
the table. Sometimes Mork figures out that the rows aren't actually part of any
table and clears them out. So the fix is simply to create a table and add the
thread rows to it. 

Comment 18

[David :Bienvenu]

Attachment: proposed fix, as described above

code to add the table of all threads, so that searching by subject will work.

Comment 19

[David :Bienvenu]

Attachment: corresponding change, since threads really get deleted now

I need to add this null check, because now threads seem to get actually
removed.

Comment 20

[Cavin Song]

Comment on attachment 85308 [details] [diff] [review]
proposed fix, as described above

r=cavin.

Comment 21

[Cavin Song]

Comment on attachment 85309 [details] [diff] [review]
corresponding change, since threads really get deleted now

r=cavin.

Comment 22

[(not reading, please use [e-mail] instead)]

Comment on attachment 85309 [details] [diff] [review]
corresponding change, since threads really get deleted now

sr=sspitzer

Comment 23

[(not reading, please use [e-mail] instead)]

Comment on attachment 85308 [details] [diff] [review]
proposed fix, as described above

sr=sspitzer

Comment 24

[David :Bienvenu]

Attachment: proposed fix

attachment 85308 [details] [diff] [review] caused some problems, because I was setting the wrong table
key for the all threads table. I was also cutting the meta row, which was
incorrect.

Comment 25

[(not reading, please use [e-mail] instead)]

Comment on attachment 85454 [details] [diff] [review]
proposed fix

sr=sspitzer

Comment 26

[David :Bienvenu]

fix checked in.

Comment 27

[David :Bienvenu]

*** Bug 105013 has been marked as a duplicate of this bug. ***

Comment 28

[laurel]

Not able to reproduce, OK using jan6 commercial trunk: win98, mac OS 10.2, linux
rh8.0