Closed Bug 123858 Opened 23 years ago Closed 22 years ago

Mail UI shows duplicate messages in a newsgroup thread when reading the first message

Categories

(SeaMonkey :: MailNews: Message Display, defect)

Product:
SeaMonkey
Component:
MailNews: Message Display
Type:
defect
Priority:
Not set
Severity:
major

Tracking

(Not tracked)

Status:
VERIFIED FIXED

People

(Reporter: mc2374, Assigned: Bienvenu)

References

(
URL
)

Details

(Whiteboard: [adt2 rtm])

Attachments

(7 files)

The UI after opening the threads - before reading messages
57.58 KB, image/jpeg
Details
UI status after reading a message in an unread thread
63.16 KB, image/jpeg
Details
UI status after reading - fake thread open showing duplicates
64.69 KB, image/jpeg
Details
Screenshots for test scenario
188.91 KB, image/jpeg
Details
workaround hiding the deeper problem, but perhaps roughly indicating it's direction
604 bytes, text/plain
Details
proposed fix
1.26 KB, patch
sspitzer
: superreview+
jud
: approval+
Details | Diff | Splinter Review
UI after expanding threads
126.98 KB, image/jpeg
Details 
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.8) Gecko/20020204
BuildID:    2002020406

After upgrading to Mozilla 0.9.8, I started to see that strange thing; my mail
component is using the threaded view (threads with unread view option) for the
newsgroups: every time I select for reading the FIRST message in one open and
unread thread in the nwsgrp window (ie. there is a minus sign on the left of the
thread tree), Mozilla UI duplicates all the messages BELOW the one being read:
ie the message being read becomes the father of another subthread and that
subthread contains the same messages that were present in original one.


Reproducible: Always
Steps to Reproduce:
1.open a newsgroup with threaded view and download new messages
2.open all threads (keyboard: "*")
3.choose the first message of an unread thread


Actual Results:  The selectd message is marked as read (the subject is in normal
font, not bold): but 
- the subject is underlined;
- the minus sign on the left becomes a plus sign;
- by opening the "new" thread tree, you can notice that the child messages are
the identical to those already displayed in the "original" thread tree.

Expected Results:  Only mark the message as read.

Plain Mozilla 0.9.8 (2002020406) running on Win98SE.
I see that only with the first msg of a thread tree: ie, selecting a child of
that msg does not trigger the bug; closing and reopening Mozilla restores a
proper appearance.
It'is somehow similar to bug 122828, but only shows up with the first message of
a thread and does not seem to involve opening and closing the threads.
Mozilla 0.9.7 was working OK for me, so it could be a regression.
I will attach 3 jpgs showing the thing at the start (threads open), after the
first message read and after opening the fake thread.
QA Contact: esther → laurel
*** Bug 131024 has been marked as a duplicate of this bug. ***
*** Bug 122828 has been marked as a duplicate of this bug. ***
*** Bug 130033 has been marked as a duplicate of this bug. ***
*** Bug 131680 has been marked as a duplicate of this bug. ***
Marking All/All since I'm on Linux. This is happening to me with mozilla 0.9.9.
I have experienced this problem with 0.9.6, asking myself how such a bad bug
could slip into a milestone. Then when 0.9.7 came out, I was very surprised that
the problem was still there. I looked for the bug in bugzilla, and was surprised
again that I couldn't find it. I then asked on IRC whether anyone else using
mozilla for news had this problem, and noone had. The problem then disappeared
when I blew away (part of?) my profile, so I thought it was just a fluke, some
sort of corruption. But now that this happens again with 0.9.9, I think it is a
more serious problem.


The problem occurs consistently while reading n.p.m.mathml, but strangely enough
I'm unable to reproduce it on other newsgroups. I am using  "threads with
unread" mode, too. The theme doesn't matter; I'm using modern.

My steps to reproduce (do not contradict the ones originally reported):

1. Open a completely new, unread thread by clicking on the twisty before the
message, without clicking on the message. The twisty changes to indicate an
expanded thread. That's ok so far.

2. The first message line of the thread is now bold to indicate that it is
unread. Click on it. The message is loaded and displayed, and the line is not
bold any more. Unexpected additional effect of this step: The twisty changes
back to indicate an unexpanded thread. This is the bug.

3. Now click on the twisty again to expand the thread. The thread is displayed
twice, with the exception of its first message which is displayed only once.

(The following is off the top of my head only:) If it was looking like

    A
    +-B
    | +-C
    +-D

then it is now looking like
  
    A
    +-B
    | +-C
    +-D
    +-B
    | +-C
    +-D

4. Click on the twisty again to the unexpand the thread. The twisty changes its
appearance to indicate an unexpanded thread, but only the second copy of the
expansion disappears. The state reached by this step is identical with the state
after step 2.


For me, this is the only problem I have with mozilla as a newsreader, and it is
serious enough to make me want to avoid using mozilla for that task. In my
opinion, this is nsCatFood and needs to be fixed for 1.0. To contact me, please
mail me directly (not via cc:list) since I have bugmail turned off.
OS: Windows 98 → All
Hardware: PC → All
A similar thing happens when I am reading email in my IMAP email account.  I
have not been able to pinpoint what it is that causes this to happen, but the
result is the same.
*** Bug 136825 has been marked as a duplicate of this bug. ***
Have now verified that this bug occurs always on Linux 2002041307.

And as comment #9 said, this behavior - amazingly enough - does not occur on
other newsgroups. See, for example, news.vmware.com  newsgroup vmware.express.
The only thing I see different about the vmware.express NG is that messages are
displayed almost instantly when selected, and it takes "forever" to display a
message on general; is this possibly a race condition?

While this is not a major problem for me, shouldn't it be given the dignity of
being marked something other than "unconfirmed"?


Confirming bug per comment 12.
Status: UNCONFIRMED → NEW
Ever confirmed: true
I can easily reproduce this bug in newsgroups. It _always_ acts this way.

I will attach an image with series of screenshots. They show the following
steps, wiewing the n.p.m.general newsgroup.

You need a newsgroup marked for reading offline. View must be set to 'Threads
with unread' (with simple threaded view it works properly). Go to this newsgroup
and let Mozilla automatically load new headers. If the first message is inside a
thread, Mozilla opens that thread in the view. (If it's a message that opens new
thread - you're unlucky - try again later).
Everything's OK so far, except that first messages of the threads which contain
new posts are no longer available for offline - maybe I should file another bug
for that.

Now download this group for offline viewing through the group's Properties. When
any message on that (automatically open) thread gets downloaded then that thread
appears collapsed - the arrow at the thread's head changes to point right (in
Modern). But the messages in that thread are still visible as a tree.
Now when you open that thread by clicking on the arrow the tree gets duplicated
- this bug shows it's ugly face...

I even got the thread's tree to be displayed 3 times, as the screenshots show,
but I don't know if I could reproduce that. I think it appeared after clicking
on the unread message inside the thread and then clicking on the thread's head
(the first message, which was also new).

The screenshots in attached image show:
1. Thread pane after clicking on a newsgroup (new headers were loaded).
2. After downloading the messages for offline viewing.
3. After clicking on arrow to expand the thread.
4. After a click on the second message in the thread.
5. After a click on the first message in the thread. This beahaviour I cannot
reproduce.

I am not sure if the group must be generally selected for offline, or is it
enough just to download messages right after clicking on newsgroup. But it
definitely must be in the "Threads with unread + Threaded" view - never seen
that with "All + Threaded" (or any other combination). I've also seen that bug
on a newsgroup not selected for offline, but I can't reproduce that so easily as
the above scenario.

This was present at least since 0.9.8 and is present in 1.0 RC1.

I encountered this bug with mail as well as news folders, in 0.9.9/1.0RC1 (did
not see this in 0.9.8, IIRC), on windows. Meanwhile, it's a permanent problem,
it's easily reproducible with any thread in any folder of any account (very
annoying!):

If the thread is expanded, the root message is marked unread, and the root
message is selected, then the thread changes to collapsed, without removing the
child entries from the view.
I stripped this down to a local mail folder containing only two files, in a hope
to have a test case I can submit, but surprisingly, the same folder (including
the msf) in a clean profile does not show the behaviour :(
Frank Schönheit in comment #16 wrote that he can reproduce the bug in any
account in any folder with any message. Contrary to that, I had this behavior
not with all messages, i.e. once I saw it with 2 threads in a newsgroup folder
and the third thread in the same newsgroup didn't show any bug.

It was with the latest branch build 2002050106.
This bug seems to be gone as of a day or two ago.
The bug was present when I first downloaded RC1's for Win98 and Linux for
mozilla.org's newsgroups only - not present on VMware newsgroup, Mail, etc.
Implication is that bug is related to mozilla.org's server setup(s) and not
actually a mozilla bug (?)
Anyone else see this?
I'm using 1.0 RC1 on both Linux and Win98 running on Win4Lin/Linux.
Phil
Phil,
sorry, can't confirm. I tried the latest nightly (1.0.0+ Gecko/20020502)for
Windows, and nothing changed - still the same behaviour, independent on the
account type (see this with company-internal and external newsgroups, POP3
accounts, local folders, and IMAP accounts).
At least I take back what I said about "any folder": I indeed found a IMAP
folder (a thread therein, respectively) where I do _not_ encounter the bug ....
Frank,
Sigh! You're right. Today, the bug was gone; tonight it's back.
It showed on about half of new threads in Linux.
Right now (11 p.m. MST), the bug is gone again from from both Linux and Win98 -
but I'm sure it will return!
I suppose it might be server load-dependent (gone early in day, present under
evening high loads, gone later at night), but it's strange that I've never seen
it on any other newsgroup!
Now it's back on Linux :(
Phil
I think the invariant in question here is the following:

   A twisty (or the plus/minus sign in classic) on the left indicates an
   expanded thread if and only if the view shows an expanded thread, 
   i.e. iff all the other messages in the thread are displayed as well.

At some point, this invariant is violated, and from that on, you can "expand" an
already expanded thread and thus get all messages in the thread displayed twice.

According to both the original report and my comment 9, this sometimes happens
if you have selected View->Messages->Threads With Unread" and you have an
*expanded* thread where *all* messages are *unread*, and you click on the
*first* message of the thread.
Then the twisty and the thread view get out of sync, if some certain trigger
condition holds (which is still unknown): The twisty changes state from expanded
to collapsed, but the thread view doesn't follow.

Can someone please point us to the code that is executed in the above case? Once
we know the code that is run, we could try to find out why the twisty is allowed
to change its state in the described situation, under some circumstances.

Or is there a "JS debugger for dummies" tutorial somewhere that could help us?
finally, found some time to do dig through the code a little bit ....

To me, it seems that the relevant flag, set on a message header, is
MSG_FLAG_ELIDED. This flag indicates a collapsed thread, and leads to a +
instead of a - beeing drawn (talking about the classic chrome here).
Additionally, it's relevant for thread roots only (more precise, for messages
which have another flag, indicating children beeing present, set), it's ignored
for the rest.

The most probable place (in my opinion) which could cause the bug is
nsMsgDBView::OnKeyChange. This is implicitly triggered by marking a message as
read (some callback from the underlying database where this is done), and it
tampers with the message flags. As I understand it, it caches the flags which
are already present in the message headers, for faster, index-based, access to
them. Now in OnKeyChange, it keeps this cache synchronous with the messages the
DB notifies, and this code is, hmm, intolerant against errors in the database.
It takes MSG_VIEW_FLAGS | MSG_FLAG_ELIDED from it's own old cached flags, and
_all_ new flags from the database. So if for whatever reason the database is
corrupt and contains the MSG_FLAG_ELIDED, too (it shouldn't - as I understand
it, the flag should not be persistent), this is transfered to the cache
implicitly just by marking the message read. And this means the thread is
recognized as collapsed, though it's children are still present in the view.

This is not only pure theory :). I tried masking the new flags got from the
database with the ones which really are persistent (means &= ~(MSG_VIEW_FLAGS |
MSG_FLAG_ELIDED)), and here on my profile it seems to fix the problem
(admittedly not very extensive tests, yet, but looks good so far). This is a
workaround at most, as it just hides the problem, and does not care for the
deeper reason (I assume the database gets corrupted somewhere).

What I do not understand is that the nsMsgDBView::OnKeyChange should be
triggered with the instigator set to the instance beeing the listener itself, so
it is expected to do _nothing_. However, it seems this is not the case. So there
may be another problem, too, which is also hidden with the workaround 
(Unfortunately, I still did not manage to have the problematic profile _and_ a
debugger for Mozilla on the same machine ....).

I'd be interested in ideas of anybody which knows the code better, and does not
just guess like I do ....
CC bienvenu for review.  This problem is hitting me a lot.
*** Bug 122762 has been marked as a duplicate of this bug. ***
I'd like to find out a reproducible case for this and better understand what's
going on. If this is biting you all the time, hakan, perhaps you have some
insight into what situation causes it? I'll see if the offline setting helps me
reproduce this.
Code-wise, it looks like Frank is onto something here, but it was a while since
I was into this code so I'd defer to you or some other backend guy. :-)

I've tried to find a solid testcase for this bug a lot, but I've never been able
to.  I can reproduce it (sometimes) when browsing newsgroups, or when reading
threaded email.  Sometimes when I click on a message in a thread to read it, the
twisty collapses and the message displays. Note that in this state, the thread's
children are still expanded, but the twisty is collapsed, so what we have is a
mismatch.  I'm not sure what causes it, but looking into why the twisty suddenly
collapses sometimes is probably interesting.
yes, Frank is definitely on the right track - I just want to find the root cause.
Re: comment 26: I'd like to find out a reproducible case for this

Please try my scenario described in comment 14 here. It's 100% reproducible for
me on 1.0rc1 (and it was in 0.9.8).
Just remember basics: View-> threads with unread, newsgroup already has some
messages in it, only the new one gets inside a thread and opens that thread
after clicking on newsgroup and downloading new headers.
I know that downloading for offline is not the best way to test this bug, but at
least it's 100% reproducible.
Adam, your scenario does cause the problem for me. I'll look into it. Taking.
Assignee: sspitzer → bienvenu
If it's of any help, I have a simpler testcase, 100% reliable.

  1. Go in any newsgroup.
  2. Sort the group by "Threads with Unread"
  3. Expand one thread (or take one already expanded). 
  4. If the first message of the thread is already marked as "read", change it
back to "unread".
  5. Select the message (if it was already selected, you need to unselect it
first (by Ctrl+clicking or selecting another message)).

Then you can see the twisty changing from "expanded" to "collapsed" but the
child messages are still visible. If you expand the thread, all children are
duplicated.
That's how comment #14 got 3 duplicates. You can actually have as many
duplicates as you want. 
And it's not necessary to have "offline use"
I've found the code that's setting the elided bit on the msg hdr
(nsMsgThreadsWithUnreadDBView::AddMsgToThreadNotInView) and I'll attach a patch
soon.
Attached patch proposed fix — — Splinter Review 
the old code used to communicate the fact that the thread should be collapsed
and has children by setting the view flags in the msg hdr flags, and letting
the view code extract it out. This resulted in the view flags ending up in the
db, which is wrong. The new code just diddles the view flags directly after
adding the hdr.
Navin, can I get a review? thx.
Status: NEW → ASSIGNED
Comment on attachment 83744 [details] [diff] [review]
proposed fix

r=naving
Attachment #83744 - Flags: review+
Attachment #83744 - Flags: review+ → superreview+
*** Bug 140007 has been marked as a duplicate of this bug. ***
I've checked in the fix on the trunk. This should stop this bug from happening,
going forward (i.e., for new threads). It doesn't fix it for old threads - I may
check in code to clear this flag from the db when retrieving msg hdr flags, but
I hate to slow down that code for this bug.

The situation that causes this bug is being in the Threads with Unread view, and
getting a new unread message in a thread that didn't previously have any unread
messages, and thus, wasn't in the view. The parent of the thread was added to
the view, and the elided/collapsed bit set on the parent msg. All the problems
happen after this, whenever various flags change on that header, and the thread
is expanded in the view, but with the elided bit set on the hdr.
marking fixed - old threads that got the elided flag set will have this problem
going forward when they get new messages, but since old threads tend to die,
this won't be as much of a problem going forward. You can delete your .msf file
if it really bothers you.
Status: ASSIGNED → RESOLVED
Closed: 22 years ago
Resolution: --- → FIXED
Bienvenu, big thanks for fixing this!
I still see this behavior with 1.0rc3 on Red Hat 7.3.  
this fix is checked into the trunk, not the 1.0 branch, so it won't be in 1.0
but it's in the current daily builds - also, you may need to delete your .msf
file to truly get this problem to go away.
As noted in comment 40, this bug is still present in Mozilla 1.0 RC3 (Linux).

I'm nominating it for mozilla1.0.1 because

* this is the one and only problem I have with Mozilla as a Newsreader
* it is very confusing to users when it occurs
* there does not seem to be an obvious workaround to get rid of the double stuff
* for me, it is reason enough to prefer a good nightly build over the mozilla
1.0 release builds when it comes to reading news. 

I have verified that the bug does not occur in rv:1.0.0+ Gecko/20020530 (Linux).
Keywords: mozilla1.0.1
I support Andreas' nomination. For me, this bug is annoyuing enough to do an own
(fixed) build of 1.0 as soon as it's out. This can't be an intention of the
_release_, can it?
Assuming that a lot of people will work with 1.0 for a long time (as with 1.0 we
will get more users which avoided current mozilla because it was a "developer
version"), the problem will get worse once all their msf files get silently and
steadily corrupted. We can't tell them to delete and rebuild them regulary.
Frank,

Why not rebuild all msf files at the very first start of a new build
automatically? Obviously Moz knows if it's the very first startup (shows mozilla
"welcome" page) or not (shows user defined home page). Self-healing is always
good practice. Of course we will still annoy 1.0 users. For a _release_ I agree
with you that it should have no such obviuos bugs in the GUI.
nominating
Keywords: adt1.0.1
Keywords: nsbeta1+
Whiteboard: [adt2 rtm]
please checkin to the 1.0.1 branch ASAP. once there, remove the mozilla1.0.1+
keyword and add the fixed1.0.1 keyword.
Keywords: mozilla1.0.1 → mozilla1.0.1+
Attachment #83744 - Flags: approval+
fix checked into 1.01 branch
Keywords: mozilla1.0.1+ → fixed1.0.1
This looks good to me using june04 commercial branch build: win98, linux rh6.2
and Mac OS 9.2.  Marking verified on branch - verified1.0.1
Keywords: fixed1.0.1 → verified1.0.1
marking adt1.0.1+ since we would have approved this, but you need adt approval
in addition to drivers approval for checking in.
Keywords: adt1.0.1 → adt1.0.1+
*** Bug 152584 has been marked as a duplicate of this bug. ***
When/how was this resolved? I still suffer this problem in 2002053012.
If you're build was a 5/30 BRANCH build, it would not have the fix. If it was a
trunk build, it should have had the fix. You might want to try a more current build.
you may also need to delete your .msf file, if it's an old thread that this is
happening on.
*** Bug 141471 has been marked as a duplicate of this bug. ***
*** Bug 134189 has been marked as a duplicate of this bug. ***
The fix made it into 1.2b, and since this version, I have a problem which I sue
the fix in attchement 83744 for:
Basically, it comes down to a situation where in
nsMsgThreadsWithUnreadDBView::AddMsgToThreadNotInView, the parent header is the
same as the header just added.

In this case, the newly added header (which is the only header in it's thread,
and thus it's own thread-root header) automatically gets both the ELIDED and
HASCHILDREN flags, which is wrong.
As a result, newly arriving messages in my Inbox (in ThreadsWithUnread) view
seem to be expandable (which is wrong).

IOW, I think the whole block which leads to the OrExtraFlag call in
AddMsgToThreadNotInView must not be executed when "parentHdr == threadHdr".

At least this fixes my above-mentioned problem, though I don't know if there are
side effects to the original bug here.
> AddMsgToThreadNotInView must not be executed when "parentHdr == threadHdr"

s/threadHdr/msgHdr/
yes, I have a fix for that, but I have to wait for the tree to open for 1.3 to
check it in.
OK using jan06 commercial trunk: win98, mac OS 10.2, linux rh8.0.
Original problem okay.
I believe the problem in comment #56 is covered by another bug. Will find it,
update comments.
Status: RESOLVED → VERIFIED
I was thinking of bug 158217... if there's another bug or need for one, David,
please log or point me to it. 
After using Moz 2003030608 for a few days this suddenly started today.
Product: Browser → Seamonkey
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: