520582 - NNTP threaded collapsed message list shows unread messages (underscore)

Status: VERIFIED FIXED

(Thunderbird :: Folder and Message Lists, defect)

Description

[Nikolay Shopik]

Attachment: screnshoot with problem

This problem appears in something like 20ish September build of 3.0 TB.
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.4pre) Gecko/20091004 Shredder/3.0pre

Comment 1

[Ludovic Hirlimann [:Usul]]

Nikolay can you try to find the regression window ? so we can try to pinpoint the bug that caused this ?

Comment 2

[Nikolay Shopik]

Easy workaround is to expand all threads and collapse.
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.3a1pre) Gecko/20091108 Shredder/3.1a1pre

Comment 3

[Nikolay Shopik]

Easy steeps to reproduce.
Subscribe to news list, mark all messages as read. Go to properties press "Rebuild Index" button. and even my workaround not work anymore.

Comment 4

[Nikolay Shopik]

2008-10-30 - last correctly working
2008-10-31 - failed

http://bonsai.mozilla.org/cvsquery.cgi?treeid=default&module=all&branch=HEAD&branchtype=match&dir=&file=&filetype=match&who=&whotype=match&sortby=Date&hours=2&date=explicit&mindate=2008-10-30+00%3A00%3A00&maxdate=2008-10-31+23%3A59%3A00&cvsroot=%2Fcvsroot

You can easily spot it, when you doing rebuild index on newsgroup new dialog prompt starts appearing like its first time downloading newsgroup.

Comment 5

[Nikolay Shopik]

Not OS specific

Comment 6

[Nikolay Shopik]

bonsai link is wrong, because in that time we already start using HG, but I don't know how to show changes in hg for specific dates

Comment 7

[Nikolay Shopik]

David, looks like fix in bug 379806 cause this regression.
http://hg.mozilla.org/comm-central/pushloghtml?startdate=2008-10-30&
enddate=2008-10-31

Comment 8

[David :Bienvenu]

rebuild index on newsgroup seems to be causing a double download of the newsgroup headers, which may be causing all sorts of interesting issues. But I've never seen this problem in normal usage...

Comment 9

[Nikolay Shopik]

Yeah but sometimes it trying rebuild index w/o my prompt just download news group dialog appears, which cause these sort issues.

Comment 10

[David :Bienvenu]

I'm reasonably confident it's the double headers that are the root of the issues here. I agree that the view code changes are most likely the cause of the user visible problem, but I think there's a more fundamental issue. I see double of most newsgroup messages now, because they were downloaded twice.

Comment 11

[Nikolay Shopik]

(In reply to comment #10)
> I see
> double of most newsgroup messages now, because they were downloaded twice.

Yeah that's what I'm exactly see too sometimes, double messages, when I mark as read one other instantly marked as read too.

Comment 12

[Nikolay Shopik]

I know NNTP is low priority as for now, but regression is most nasty type of bugs. So I wonder(genuine) is this require a lot changes to fix?

Comment 13

[Nikolay Shopik]

regression window comment#4

Comment 14

[David E. Ross]

Mozilla/5.0 (Windows NT 5.1; rv:6.0.1) Gecko/20110830 Thunderbird/6.0.1
	Default 6.0.1 theme
	View > Layout > Wide View
	View > Threads > Threads with Unread
        Newsgroups initially open collapsed

I think I am seeing this even when running in Safe Mode.  In the Header pane, some threads appear underlined.  Sometimes, the number of underlined threads exceeds the indicated count of unread messages.  I expand the underlined threads, and a number of them contain no unread messages.  In a few cases, expanding and then collapsing the afflicted threads does NOT clear the underling.  

I am seeing the above when viewing newsgroups on the following servers:
reader.albasani.net
news.eternal-september.org
=>  news.mozilla.org  <=

Comment 15

[David E. Ross]

Selecting the Repair Folder button on the Folder Properties popup dialogue while in Safe Mode made things worse instead of better.

Comment 16

[Nikolay Shopik]

As workaround open all threads with keyboard shortcut "*" and close "\" that usually fix it.

Comment 17

[David E. Ross]

Attachment: Screenshot of TBird with excess indicators of unread messages.

The attachment shows on the left that there are 3 unread messages and on the right that there are 14 threads with unread messages.  When I expanded all threads, there really were only three unread messages (one in each of the last threads, including the non-thread single message at the very end).  

Re comment #16:  I have tried this over and over again.  In most cases, it does not work.  I just tried it three times on the newsgroup in the attachment (rec.gardens); none of the underlines were removed.  I tried it again, leaving the threads all expanded when I terminated TBird.  I then relaunched TBird.  When I then collapsed the threads, the underlines were still there.

Comment 18

[David E. Ross]

I am seeing this with more and more newsgroups.  This impacts my ability to review new messages in existing threads because I now have to expand many threads merely to find the few that truly contain new messages.  

I cannot clear the underline that indicates unread messages in a thread.  I have tried marking the thread as read; that did not help.  I have tried marking the entire newsgroup as read; that did not help.  Using k, I have marked an afflicted thread to be ignored; when I later remove the k, the problem reappears.  

The definition of Minor Severity for a bug report states: "minor loss of function, or other problem where easy workaround is present".  There is NO workaround that consistently mitigates this problem.  Thus, I believe the Severity of this bug report should be changed back from Minor to at least Normal.

Comment 19

[David E. Ross]

Mozilla/5.0 (Windows NT 5.1; rv:10.0.2) Gecko/20120216 Thunderbird/10.0.2
      Default 10.0.2 theme
      See comment #14 for the rest of my configuration

While viewing a newsgroup, I again saw this problem.  Using the k on my keyboard, I marked the affected thread -- on the original message -- as "Ignore" since I was really no longer interested in it.  (Of course, the thread did not disappear from the header pane; that only happens if I select a different newsgroup, close the account, or terminate Thunderbird.)  After I collapsed the thread, the underline remained on the original message.  

Prior experience has been that, if I view ignored threads and clear the "ignore" flag by again using the k on my keyboard, the underline is still there even though there are no new messages in the thread.  This happens even if I terminate Thunderbird and shut down my PC overnight and then reboot and launch Thunderbird.  

The .rc file for the account containing an affected newsgroup does not show any gaps in the numbering of messages in that newsgroup.  

Since there is no workaround (as required for a "Minor" bug), I am changing the severity to Normal.

Comment 20

[Jim Porter (:squib)]

I've noticed this occasionally, and I think I've found an interesting way of diagnosing this. Unfortunately, I haven't seen the issue lately, so I can't test it myself.

If this happens to you, install the add-on Mail Summaries* and look at the "Recent Messages" section for the Folder Summary for that group. Look to see if there are any unread messages in that list (i.e. bold entries). If there are, figure out what thread they're in, check the thread structure, and then click the unread message in the summary. It should show up in the thread now. My guess is that the unread message in question got hidden (somehow), and opening it via Mail Summaries fixes that. If someone could figure out what exactly is happening here, that would get us a lot closer to fixing the bug.

* You might need the in-development version at https://bitbucket.org/squib/mail-summaries

Comment 21

[David E. Ross]

Mozilla/5.0 (Windows NT 5.1; rv:17.0) Gecko/20130328 Thunderbird/17.0.5
	Default 17.0.5 theme
	View > Layout > Wide View
	View > Threads > Threads with Unread

I am seeing this problem right now in the rec.gardens newsgroup on the reader.albasani.net NNTP server.  I tried a little experiment:  

1.  After opening the rec.gardens newsgroup with all threads collapsed, I selected [View > Threads > Unread].  The afflicted disappeared.  

2.  I then selected [View > Threads > All].  The afflicted thread reappeared among other threads.  The afflicted was underlined.  None of the others were underlined.  

3.  Expanding the afflicted thread, there were no new messages.  I then collapsed that thread.  

4.  I selected [View > Threads > Threads with Unread].  Only the afflicted thread remained in the thread pane, still underlined.  

5.  I checked file reader.albasani.net.rc.  There were no gaps in the message numbers for rec.gardens.  

6.  I checked the rec.gardens newsgroup on the news.eternal-september.org NNTP server.  The thread with the problem on reader.albasani.net had the same 9 messages on news.eternal-september.org.  Furthermore, after marking that particular thread as already read on news.eternal-september.org and collapsing the thread, it remained underlined.  Thus, this problem is not peculiar to only one NNTP server.

Comment 22

[David E. Ross]

This bug might be a mere artifact of bug #799040.

Comment 23

[David E. Ross]

Then again, this might NOT be symptomatic of bug #799040.  I see this problem with threads where there is NO ignored subthread.

Comment 24

[David E. Ross]

Windows 7 (x64)
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0
	Default 24.6.0 theme
	View > Layout > Wide View
	View > Threads > Threads with Unread

This problem is now quite persistent in the mozilla.support.seamonkey newsgroup on the news.mozilla.org server.  My news.mozilla.org.rc file for that server shows no gaps in the sequence numbers of messages.  Enabling the viewing of ignored threads shows no ignored subthreads for mozilla.support.seamonkey, only complete threads being ignored.

Comment 25

[David E. Ross]

Windows 7 Ultimate SP1
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0
NoGlass 0.11 theme
	View > Layout > Wide View
	View > Threads > Threads with Unread

This is very persistent in a newsgroup, starting today.  For affected threads, I tried [Mark > Thread as Read].  I tried Repair Folder on the newsgroup, which removed all stars from starred messages.  I checked the .rc folder for the account, and found no gaps for the affected newsgroup.  I still see many underlines in the Thread pane where there are no unread messages for those threads.  However, the newsgroup line in the Folder pane, does not indicate any unread messages, either by count or bold text.  

I get the same result launching Thunderbird in Safe Mode.  

The affected newsgroup is alt.windows7.general.  I read it through an account at reader.albasani.net.  I do not see the problem if I use an account at news.eternal-september.org.  I cannot tell if the problem is within Thunderbird or reader.albasani.net.

Comment 27

[Wayne Mery (:wsmwk)]

Alfred does this reproduce for you?

Comment 28

[Alfred Peters [:infofrommozilla]]

(In reply to Wayne Mery (:wsmwk) from comment #27)

Alfred does this reproduce for you?

Yes, I can still see that here in TB132.0a1.

I have also seen the double header download (Comment #8) in the past. But not at the moment. So that is probably not the cause.

Comment 29

[Hartmut Welpmann [:welpy-cw]]

(In reply to Alfred Peters [:infofrommozilla] from comment #28)

(In reply to Wayne Mery (:wsmwk) from comment #27)

Alfred does this reproduce for you?

Yes, I can still see that here in TB132.0a1.

If repairing the folder helps, this could be a remnant of the behavior fixed in bug 448876. If you encounter this with newly received threads as well, then rather not.

Comment 30

[Alfred Peters [:infofrommozilla]]

(In reply to Hartmut Welpmann [:welpy-cw] from comment #29)

If repairing the folder helps,

In fact, “Repair Folder” (see comment #8 / comment #15) is causing the problem for me.

STR:

1. Unsubscribe group.
2. Close Thunderbird.
3. Make sure that all group files in the profile have been deleted.
4. Start Thunderbird.
5. Subscribe to the group.
6. "Group" -> Context menu -> Properties -> /Synchronization\
   => [x] Select this folder for offline use
7. "Group" -> Context menu -> Properties -> /Synchronization\
   => [Download Now]
8. Select "Group"
9. Menu -> Message -> Mark -> All Read
10. Restart Thunderbird.

Check: all articles are still read

11. "Group" -> Context menu -> Properties -> /General Information\
    => [Repair Folder]

Check: closed threads have underscore now.

12. Select an article and Menu -> Go -> Next -> Unread Message

Thunderbird opens all threads but doesn't find any unread articles.

Comment 31

[Hartmut Welpmann [:welpy-cw]]

(In reply to Alfred Peters [:infofrommozilla] from comment #30)

Check: all articles are still read

11. "Group" -> Context menu -> Properties -> /General Information\
    => [Repair Folder]

Check: closed threads have underscore now.

I can reproduce this. There certainly is something off …

12. Select an article and Menu -> Go -> Next -> Unread Message

Thunderbird opens all threads but doesn't find any unread articles.

Does this persist afterwards (or after typing * and \)?

Comment 32

[Alfred Peters [:infofrommozilla]]

(In reply to Hartmut Welpmann [:welpy-cw] from comment #31)

(In reply to Alfred Peters [:infofrommozilla] from comment #30)

11. "Group" -> Context menu -> Properties -> /General Information\
    => [Repair Folder]

Check: closed threads have underscore now.

These also survive a restart of Thunderbird.

I noticed something else. Despite the underlines, the Menu->Message->Mark->All Read menu item is disabled.

Thunderbird opens all threads but doesn't find any unread articles.

Does this persist afterwards (or after typing * and \)?

No, once a thread has been opened (no matter how) it loses this status and thus also the underline after it has been closed again.

Comment 33

[Hartmut Welpmann [:welpy-cw]]

Attachment: Bug 520582 - Adjust unread status of NNTP headers when downloading. r=infofrommozilla,mkmelin!

This patch sets the unread status of downloaded headers according to the newsrc file as soon as they are added to the message database. Previously this was only done when the view was initialized, but nsMsgDBView::AdjustReadFlag wasn't called on elided messages in collapsed threads. Especially after repairing the folder or getting the next n messages, this could result in an incorrect and fragile view.

Comment 34

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/f4ff3b339a71
Adjust unread status of NNTP headers when downloading. r=mkmelin

Comment 35

[Magnus Melin [:mkmelin]]

comm/mailnews/news/test/unit/test_filter.js is failing

Comment 36

[Magnus Melin [:mkmelin]]

Backout: https://hg.mozilla.org/comm-central/rev/2367a8f96fe2b45343f60e857c2d8b34f510fa04

Comment 37

[Hartmut Welpmann [:welpy-cw]]

The resetFilter() function used in that test did not reset the newsrc file as well between iterations. This works now, Try looking much greener.

Comment 38

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/105af642383a
Adjust unread status of NNTP headers when downloading. r=mkmelin

Comment 40

[Vlad Lucaci, Thunderbird QA (:vlucaci)]

Confirming this issue as verified fixed with 133.0b3(20241107210012) using Windows 11, macOS 14, and Ubuntu 22.

Comment 41

[Hartmut Welpmann [:welpy-cw]]

Comment on attachment 9430958 [details]
Bug 520582 - Adjust unread status of NNTP headers when downloading. r=infofrommozilla,mkmelin!

[Approval Request Comment]
User impact if declined: Threads in a newsgroup may appear unread even when containing only read messages. Repairing the folder could make the matter worse, also getting the next n message may not work as expected. Ultimately the thread pane may hang/break in these cases.
Testing completed (on c-c, etc.): c-c and beta
Risk to taking this patch (and alternatives if risky): low

Comment 42

[Hartmut Welpmann [:welpy-cw]]

Release Note Request (optional, but appreciated)
[Suggested wording]: Correctly maintain the unread/read status of newsgroup articles.

Comment 43

[Corey Bryant]

Comment on attachment 9430958 [details]
Bug 520582 - Adjust unread status of NNTP headers when downloading. r=infofrommozilla,mkmelin!

[Triage Comment]
Approved for esr128

Comment 44

[Daniel Darnell [:dandarnell]]

Thunderbird 128.4.3esr:
https://hg.mozilla.org/releases/comm-esr128/rev/f849d90feb72

Comment 45

[Vlad Lucaci, Thunderbird QA (:vlucaci)]

Confirming this issue as verified fixed with 128.4.3(20241108210256) on macOS 14, Windows 11 and Ubuntu 24.