271036 - View -> Threads -> Read threading displays incorrectly

Status: NEW

(Thunderbird :: Mail Window Front End, defect)

Description

[Jordan Abel]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.5) Gecko/20041111 Firefox/1.0 (Debian package 1.0-2)
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.5) Gecko/20041111 Firefox/1.0 (Debian package 1.0-2)

When selecting "View -> Threads -> Read" the messages have indentation assigned
to them but are shown in jumbled order - Threading should assign each message to
be under the nearest _visible_ ancestor in the tree

Reproducible: Always
Steps to Reproduce:
1. Open a newsgroup
2. View -> Threads -> Read
3. Witness chaos

Actual Results:  
Some messages were shown too far to the left/right, with tree lines not
connecting to anything visible - others were jumbled at the bottom of the thread

Expected Results:  
Nicely-threaded messages

Comment 1

[Mike Cowperthwaite]

> When selecting "View -> Threads -> Read"

Presumably you mean   View | Threads | UNread

> Threading should assign each message to be under the nearest _visible_
> ancestor in the tree

In my testing (TB 0.9,Win2K), it appeared that this does happen.  Can you 
generate a folder containing a mix of read and unread messages that exhibits 
this behavior?  If so, please attach the folder file to this bug.

Comment 2

[Jordan Abel]

Yes, I did mean unread - I forgot to mention - this behavior mainly occurs for
me on usenet - I haven't observed it on email [mainly because my email
conversations aren't nearly as complex as usenet threads] - It's also a bit
temperamental, and doesn't occur consistently

Comment 3

[Nikolaus Rath]

I can confirm this bug as well. Unfortunately I've just hit C, so the exact
read/unread combination is lost. I'll attach an mbox as soon as it happens again.

Comment 4

[Nikolaus Rath]

(In reply to comment #1)
> In my testing (TB 0.9,Win2K), it appeared that this does happen.  Can you 
> generate a folder containing a mix of read and unread messages that exhibits 
> this behavior?  If so, please attach the folder file to this bug.

I've got a folder which triggers the bug. Which file do you need exactly?

Comment 5

[David :Bienvenu]

the folder itself, which is in your user profile directory.

Comment 6

[Nikolaus Rath]

Attachment: Screenshot of bug

Comment 7

[Nikolaus Rath]

Attachment: MBox of the displayed thread

Comment 8

[Nikolaus Rath]

Attachment: Folder file from .mozilla-thunderbird

Comment 9

[Nikolaus Rath]

Attachment: Folder file from .mozilla-thunderbird

Comment 10

[Nikolaus Rath]

I have attached some files demonstrating the bug. I don't know when the bug
occured exactly. When I entered the group, the complete thread was unread. I
than read une message after another using "n". Having read all messages in the
thread, I noticed that the threading was wrong, but I suspect that it was wrong
from the beginning.

Comment 11

[David :Bienvenu]

I tried saving those two mboxes (the third attachment is bogus) and opening
them, and the threading displayed correctly. The screen shot looks like a known
problem that happens when you delete messages from a thread, but it's hard to
delete messages from a newsgroup,other than using cancel. So, you're saying that
if you save one of those attachments as a mail folder, start thunderbird, and
open the folder, that the thread displays as it does in the screenshot? I'm
running a trunk build, but I don't know that there have been any fixes in this
area...

Comment 12

[RenegadeX]

I believe this is caused by the same issue in Bug 293421
https://bugzilla.mozilla.org/show_bug.cgi?id=293421 and is thus a duplicate.

I can recreate (on WindowsXP system) the broken thread-lines as seen in the image posted in Comment #6 by setting:
'View->Sort by-> Date-> Threaded' + 'View->Messages->All' + 'View->Threads->All'
.. and then switching to 'Messages->Unread' (also sorted by Date & Threaded).

A workaround until it is fixed - go to 'View->Messages' and set it to 'All'; set all your Sort and View settings the way you like them, then switch back to the 'Messages->Unread'. It may still be broken until you select 'View->Threads->All', which for some reason is missing its bullet.

Comment 13

[Mike Cowperthwaite]

(In reply to comment #12)
> I believe this is caused by the same issue in Bug 293421
> and is thus a duplicate.

I think you need to read comment 0 of this bug more carefully -- this bug
isn't about the ordering of threads (which is what you discuss at that bug)
but about the visual alignment of messages within a thread.

Comment 14

[RenegadeX]

Attachment: Screencap of broken thread lines/broken sort as seen in attachment in Comment#6

This was seen in 'Messages->UnRead' view.
See Comment 12 for steps to recreate.
(I used TB 1.5RC2 with 0 extensions, also affects SeaMonkey 1.0b)

Screencap with correct function taken approx 3 seconds later in 'Read' view with expected results to follow..

Comment 15

[RenegadeX]

Attachment: Expected Sort by Date, Threaded view.. aligned properly

(attached taken 3 seconds after previously attached screencap..
this is how the view when sorted by Date and in Threaded mode should look.
The only difference between the 2 was switching to 'Messages->All' mode to take this screenshot.

Re: Comment 13:
Mike, I can read.
Comment 0 says:
"messages have indentation assigned to them but are shown in jumbled order - Threading should assign each message to be under the nearest _visible_ ancestor in the tree".

Which I have shown in attachment 207034 [details]

And Comment 0 also says:
"..with tree lines not connecting to anything visible"

Which I have also shown in attachment 207034 [details].

Notice how Comment #1, by none other than yourself - points out that the OP meant that this happens in the 'UNread' view?

In my discussion on Bug 293241, I proved that a group that is set to Messages=>Unread but has different Sort/Threading settings to 'Messages->All' will automatically 'forget' the sort settings for the Unread view each time the newsgroup is entered. See Comment 0 above ... "open a newsgroup..... witness chaos".

So Mike, your turn - please expand on how these 2 issues are different again?

Comment 16

[Mike Cowperthwaite]

(In reply to comment #15)
> Notice how Comment #1, by none other than yourself - points out that the OP
> meant that this happens in the 'UNread' view?

No: I point out that this happens with  View | Threads | Unread  -- which is 
not a "view" in the same sense as selecting 'Unread' from the Message View dropdown or using the  View | Messages | Unread  menu.  It's possible that the end result in the backend of the code is the same.  If you dig into the code deep enough to prove the case, maybe you could provide a patch at the same time.


> So Mike, your turn - please expand on how these 2 issues are different again?

It's two different symptoms, simple as that.  That bug is about "forgetting 
the view settings for a newgroup or folder"; this bug is about "incorrectly ordering messages within a thread."  The fact that you happen to see both symptoms at the same time does not mean they're the same bug; in this case, I think they're obviously different.

You know, I spent a lot of time checking what you described at that other bug, and I agree you found something worth fixing -- in fact, something that hasn't been the subject of a bug so far.  You should open a bug for that, and not clutter up *other* bugs that are about *other* symptoms.  Or, you could 
continue to get defensive and refuse constructive criticism and end up having all your future comments ignored entirely.

Comment 17

[Mike Cowperthwaite]

As far as this bug's original report goes:  As with comment 11, I also could 
not reproduce the problem from Nikolaus Rath's attachment.  David said "The screen shot looks like a known problem that happens when you delete messages from a thread" -- I think that's bug 154403.

This bug may be a dupe of bug 208324.

Comment 18

[David E. Ross]

Thunderbird 1.5

I'm seeing this while viewing the newsgroup mozilla.support.thunderbird at news.mozilla.org.  

Comment 19

[David E. Ross]

The problem is most visible when threads branch.  That is, a thread of the form
  xxxx
     xxxx
        xxxx
        xxxx
           xxxx
is less likely to show the problem than a thread of the form
  xxxx
     xxxx
        xxxx
     xxxx
        xxxx
where the second and fourth messages are replies to the original message.  This seems especially problematical if the third message (reply to the second) is later than the fourth message.  

It seems the threads are displayed with the messages chronologically rather than in their "reply to" relationships even though the threaded display attempts to indicate those relationships.  Instead, the indicated relationships are often false, with a reply attached to the wrong preceding message.  Trying to make sense of this mess is made even more difficult by the lack of the capability requested in bug #229463.  

Comment 20

[David E. Ross]

I'm seeing mangled threads when ALL messages are still visible (when messages that have been read are then marked as unread).  

Comment 21

[Ronald Killmer]

(In reply to comment #20)
> I'm seeing mangled threads when ALL messages are still visible (when messages
> that have been read are then marked as unread).  
> 

This threading issue may be related to the message reference ID's perhaps not being used to align the threads.  NC4.x used the message references that were displayed in the header of the viewing message to generate the threading.

Comment 22

[David E. Ross]

When I am interested in a newsgroup thread, I often mark messages within that thread as "unread" after I read them.  Thus, I now can view several threads where all messages are visible.  This problem exists in those threads.  I am therefore changing the Summary to remove "when not all messages are visible".  

On closer examination, it appears that the messages are sorting by date, not by threading.  Then, an arbitrary threading tree is created that cannot be correct if the thread has branches.  

Comment 23

[RenegadeX]

I'm still waiting for someone to post the complete, and accurate, steps to reproduce this Bug. The original post' steps are too simplified, and as we see from Comment#1 and #2, not accurate. 

As I said a while back, I can recreate the symptoms described by everyone here at will. And I can also 'fix' it at will. Mike indicates that 2 similar symptoms do not equate to 'same issue'. But as nobody has posted their complete 'View' menu settings here, how can we be sure? Without a doubt Mike knows a heck of a lot more about Mozilla coding that I do, so I'm not doubting him, but I recently  took it upon myself to learn how to create a patch for a Bug & submit it, and am now keen to get cracking at some more issues.

If you can recreate this Bug, please tell me what settings you are using for each of the following:
1) View->Sort By [_____], [ascending/descending], [Threaded/Unthreaded/Grouped by Sort]
2) View-> Messages [All/Unread]
3) View-> Threads [All/Unread/Threads With Unread]

Comment 24

[David E. Ross]

More investigation -- 

See newsgroup mozilla.support.thunderbird at the news.mozilla.org server.  See threads with subject "Phantom Newsgroup Message" (original post 2/12/06) and subject "Newsgroups - Crossposting" (original post 2/15/06).  

Threading works okay with the folling settings for View (menu bar): 
  Sort by > (Order Received, Ascending, Threaded)
  Threads > Threads with Unread
  Messages > All (forced by Threads > Threads with Unread)

However, many users prefer to have Messages > Unread.  This causes a resorting of the messages, corrupting the display of the threads.  

Setting Threads > Unread does not correct the display and also switches Messages back to All.  If you then set Messages > Unread, Thunderbird might clear all Threads settings -- or might not (being somewhat unpredictable).  This illustrates that changing one View preference may automatically change another preference in a manner that the user does not want and might not even notice until later.  

Part of the problem might be caused by the implementation of excessively complicated View preference options.  

Comment 25

[Mike Cowperthwaite]

Attachment: Screenshot: thread with All messages

Comment 26

[Mike Cowperthwaite]

Attachment: Screenshot: thread with only Unread

OK, I think I've reproduced the problem; the threading sequence here shows a reordering, with messages on one subthread appearing as children of messages in a different branch:

The "Matt Nordhoff" message is the unread message closest to the top in the 
full tree structure, and it is selected as the "root" in the filtered (unread-only) view.  It has no unread children of its own.

Other subthreads that are "earlier" in the thread don't have unread messages until deeper into the tree.  Each of the other subthreads appear as siblings, and the one actual group of unread messages in a thread are shown threaded.

Jordan Abel, David Ross: Is this the same problem you're seeing?  It's not (quite) the same as the other screenshots posted earlier.

Comment 27

[David E. Ross]

Attachment: Screenshot: View unread but all message indeed unread

Yes, your two attachments (comment #25 and comment #26) are what I see for View:All and View:Unread.  

I have added an attachment showing the same messages (plus at least one new one) as in "Screenshot: thread with All messages".  Here View:Unread but all messages are indeed unread.  Compare the ordering of the messages with the ordering in "Screenshot: thread with All messages".  In my attachment, all the messages are actually sorted in time sequence, not in thread sequence.

Comment 28

[Mike Cowperthwaite]

(In reply to comment #27)
> In my attachment, all the messages are actually sorted in time sequence,
> not in thread sequence.  

When I generated my screenshots, I was using Sort By | Order Received, as you stipulated in comment 24.  Were you still using that order, or had you switched to Sort by Date?  Also, it's not clear which set of messages you'd marked Read in that thread; it's possible that my having marked some older messages Unread affected that ordering.

I continue to believe this is the same behavior as bug 208324; cc'ing that 
bug's reporter.

That bug, and this, report the problem originally under News -- not surprisingly, as that's where threading is most consistently used.  But, has 
it been reproduced by anyone in a local folder?  In in an IMAP folder?

Comment 29

[David E. Ross]

I have View > Sort by > Order Received.  

With newsgroup messages, I'm not sure about the difference between Date and Order Received.  Actually, the whole View menu is quite confusing.  It's overly complicated compared with the options that were available with Netscape 4.7, which was my newsreader until Thunderbird 1.5 became available.  And there is no real Help information about the View menu.  

Comment 30

[Mike Cowperthwaite]

(In reply to comment #29)
> With newsgroup messages, I'm not sure about the difference between Date and
> Order Received.  

Date is sorted by the Date header; Order Received is sorted by the order in which the server presents the postings, which is likely the order they were received by the server, and so liable to propogation delay.

Also, sorting threads by Date puts them in order based on the newest message in the thread; sorting threads by Order Received puts them in order based on the Order Received of the available root node of the threads.  Which, I guess, is even more variable if you're filtering the available messages via Unread.

> Actually, the whole View menu is quite confusing.  It's overly complicated 

I agree; I opened bug 237164 about this, altho I've subsequently WontFix'd it 
in favor of some other bugs which probably won't get implemented either.  Of particular interest to this bug is, I think, bug 321739.

Comment 31

[David E. Ross]

Regarding [Sort by > Date] versus [Sort by > Order Received], see bug #264941.  

Comment 32

[Wayne Mery (:wsmwk)]

Comment on attachment 191043 [details]
Folder file from .mozilla-thunderbird

display of threads still fails per reporter attachment 191042 [details] MBox of the displayed thread
version 3.0a1pre (2007120503)

dupe to bug 208324 or dependent?  (not clear to me why this hasn't been done already)

Comment 33

[Joshua Cranmer [:jcranmer]]

Looking at these comments, I seem to see two distinct issues:

1. The threading lines are not correct (known problem, of which bug 208324 is probably the best bug to dupe to).

2. View threads as unread seems to imply a thread hierarchy which is not correct.

If this is so, then the first part should be shifted to the more correct bug, and the latter part either duped or split off into a new bug, or this bug's summary be changed. Making a new bug seems more correct to me...

Thoughts?

Comment 34

[Ronald Killmer]

I agree this is a mixed bag of fruit. A fix of #208324 may improve or fix the second issue.

Comment 35

[Wayne Mery (:wsmwk)]

so we need fix for bug 208324 to see what's left.
let's take this off UNCO even though problem is not well understood.