448876 - Killed (ignored) threads show non-zero count of unread mail / news article

Status: RESOLVED FIXED

(MailNews Core :: Backend, defect)

Description

[Nelson Bolyard (seldom reads bugmail)]

Attachment: screen shot of message list pane with two examples

Running SM trunk nightly build from August 1.
Displaying messages in a news group, all threads, viewing killed threads too.
Killed threads show non-zero unread message counts

Comment 1

[Nelson Bolyard (seldom reads bugmail)]

Oh, rebuilding the summary file doesn't help

Comment 2

[Joshua Cranmer [:jcranmer]]

*adds to list*

Rebuilding the summary file doesn't recalculate the counts. Honestly, this is a value that could most likely be computed easily at runtime if the msfs were SQLite (SELECT COUNT(*) FROM messages WHERE threadID=?1 AND read=FALSE). It certainly seems that the cached values are problematic in that they are not set in hard-to-reach corner cases.

Comment 3

[Nelson Bolyard (seldom reads bugmail)]

Note that the erroneous counts seem to be counting the messages that were
added to the thread after it was already killed.  It only includes messages
that were added while running the August 1 build, not messages that were
added while running the earlier build I had been running (from early July.

Comment 4

[Tony Mechelynck [:tonymec]]

By "rebuilding the summary file" in comment #1, you mean the .msf don't you? Have you tried the workaround described at bug 294754 comment #3 (and mentioned in bug 79130, to which bug 294754 has been duped)?

Comment 5

[Nelson Bolyard (seldom reads bugmail)]

(In reply to comment #4)
> By "rebuilding the summary file" in comment #1, you mean the .msf don't you?

I mean that I go into the properties dialog for the news group and bonk the 
button labeled "Rebuild Summary File", which does whatever it does. 

> Have you tried the workaround described at bug 294754 comment #3 (and 
> mentioned in bug 79130, to which bug 294754 has been duped)?

I don't wish to mark the entire group read.

Comment 6

[Stewart Gordon]

ISTM this could be the root cause of bug 79130.  But we need to investigate further.

The problem seems to be that Mozilla saves a per-thread tally of unread messages separately from the per-message read/unread flag.  Why?  It's probably sensible to keep such a tally in memory while Mozilla is running, but not to have this tally persist from session to session.

Comment 7

[Joshua Cranmer [:jcranmer]]

(In reply to comment #6)
> ISTM this could be the root cause of bug 79130.  But we need to investigate
> further.

More likely bug 79130 impacts this one. The root problem of 95% of news count problems can be traced to the fact that a) servers lie about how many messages they have and b) if message 6514 and 6516 exist, it's not guaranteed that 6515 exists, thanks to canceling and superseding. Actually, cases can theoretically become even more edgy (6515 exists and then is canceled later), but we'll leave that aside for now.

Between XOVER+XHDR, XOVER+HEAD, and vanilla HEAD, the edge case of canceled messages is hard to get correct in all the cases, especially when one takes into account that we get new message counts by a GROUP, so it's almost always an overestimate to begin with. There's a lot here I need to play with to get all the cases right, but unfortunately my workload is too far stretched as it is.

> The problem seems to be that Mozilla saves a per-thread tally of unread
> messages separately from the per-message read/unread flag.  Why? 

Caching. All thread information could be generated from runtime information (and pretty quickly, too), but we cache the data. This decision was made even before NS 4 (IIRC), so it's some old stuff.

> It's probably
> sensible to keep such a tally in memory while Mozilla is running, but not to
> have this tally persist from session to session.

Something I've played with in my head is to make Rebuild Index a lot more destructive to the cache and making Mark XYZ as Read also more destructive to read count caches when they get things just plain wrong. Unfortunately, I don't think this would be on the easier side of things.

Comment 8

[Mark Banner (:standard8)]

Given that message counts in Thunderbird are not displayed by default for most people, they aren't going to notice this. I also think it is not a significant enough bug that we would block a beta on it.

We still want a fix, not sure we'd would block TB 3 on it, so leaving that as a ?

Comment 9

[David Ascher (:davida)]

I don't think we'd block.

Comment 10

[Dan Mosedale (:dmosedale, :dmose)]

Given our re-jiggering of how we're using blocking and wanted <https://wiki.mozilla.org/Thunderbird:Release_Driving>, unless we have some reason to believe this specific bug covers more than NNTP, I'm setting wanted-.  In general, NNTP is lower priority these days, and the driving process won't be devoting a lot of energy to it.

Comment 11

[Daniel]

I was looking at Bug 288120 as a way of reporting this Killed Threads situation, but now Joshua Crammer has pointed me to this thread, so, maybe, someone needs to set 288120 as dependant on this bug being fixed first

Comment 12

[Joshua Cranmer [:jcranmer]]

(In reply to comment #11)
> I was looking at Bug 288120 as a way of reporting this Killed Threads
> situation, but now Joshua Crammer has pointed me to this thread, so, maybe,
> someone needs to set 288120 as dependant on this bug being fixed first

Bug 288120 is something completely different: an RFE to exclude the ignored messages when we download for offline. The code that would be involved in such a process is entirely different from the code that is involved in this bug. In particular, the problem here is that some of the DB unread count caching is borked, and the unread counts would not come into play wrt downloading messages for offline.

Comment 13

[Daniel]

Joshua, I'm only a dumb user here, but it would seem reasonable to me that I only want to download messages that are marked to indicate that I haven't read them.

If they are marked as read (either because I've read them or they are marked as read because I want to ignore (killfile) the thread, it does not matter), I don't want to download them.

Effectively, Ignore Thread (K) should act as though it reads "Mark every message that *ever* attaches to this thread as read". Then it will not be seen or downloaded (as, I'm guessing, downloading ONLY downloads messages marked as unread).

Comment 15

[Mats Palmgren (inactive)]

It also affects the total unread message count in mail folders.
Very annoying since the unread messages are hidden, so when I type
'n' I get the "do you want to advance to ... <other folder>" popup.

(In reply to Daniel from comment #13)
> Effectively, Ignore Thread (K) should act as though it reads "Mark every
> message that *ever* attaches to this thread as read".

+1

Comment 17

[Mats Palmgren (inactive)]

I'd really like to see this bug fixed.  I hereby offer $100 (USD) to anyone who fixes
this bug when a working patch lands for Thunderbird.

Comment 19

[Piotr Dobrogost]

After having read https://blog.mozilla.org/thunderbird/2019/01/thunderbird-in-2019/ I thought I would bump this (sorry!) with the hope that maybe, somehow after all these 11 years it gets fixed this year…

Comment 21

[PierU]

I can't understand how/why this 13-years old bug has still not been assigned a priority (let alone a high priority).

This bug makes the "Kill (sub)thread" feature almost useless for those who watch the newsgroups in threaded view, as it gets impossible to quickly spot which threads contain unread messages :(

Comment 22

[Hartmut Welpmann [:welpy-cw]]

Attachment: Bug 448876 - Adjust unread child count of thread when marking a killed header read. r=mkmelin

Inspired by https://github.com/Betterbird/thunderbird-patches/blob/main/91/bugs/799040-fix-unread-in-ignored-thread.patch

Comment 25

[Hartmut Welpmann [:welpy-cw]]

Attachment: Bug 448876 - Fix unread status of ignored threads. r=mkmelin

Previously, messages received in an ignored thread/subthread were marked as read before being
committed to the database. As a result, the thread's child unread count wasn't updated correctly.
Also, in IMAP folders, the messages themselves would reappear as unread because the necessary sync
wasn't performed. This should fix these issues.

Comment 26

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/f65e517ea040
Fix unread status of ignored threads. r=mkmelin

Comment 27

[Hartmut Welpmann [:welpy-cw]]

Comment on attachment 9371711 [details]
Bug 448876 - Fix unread status of ignored threads. r=mkmelin

[Approval Request Comment]
User impact if declined: Thread with ignored subthreads keeps showing up as unread. Unread count of IMAP folder is changed by ignored messages.
Testing completed (on c-c, etc.): c-c and beta
Risk to taking this patch (and alternatives if risky): low

Comment 28

[Wayne Mery (:wsmwk)]

Comment on attachment 9371711 [details]
Bug 448876 - Fix unread status of ignored threads. r=mkmelin

[Triage Comment]
Approved for esr115

Comment 29

[Rob Lemley [:rjl]]

Thunderbird 115.9.0:
https://hg.mozilla.org/releases/comm-esr115/rev/d5387e5a97e6