newsgroups periodically lose unread count for messages whose headers haven't been downloaded

RESOLVED FIXED

Status

MailNews Core
Backend
RESOLVED FIXED
12 years ago
9 years ago

People

(Reporter: Ryan Hill, Assigned: Bienvenu)

Tracking

({fixed1.8.1.1})

Trunk
fixed1.8.1.1
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(3 attachments, 1 obsolete attachment)

(Reporter)

Description

12 years ago
User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8b2) Gecko/20050623 Firefox/1.0+
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8b2) Gecko/20050623 Firefox/1.0+

last produced with TB 1.0+ 20050623.  first noticed over a month before this.

after retrieving message headers from the news server, often after a short time
(15 minutes or so) the unread messages count for each news group other than the
one selected will vanish.  choosing a group will cause the previous count to
reappear or update if any new messages are available.  this seems to happen
sporadically and so far i don't have a reliable way to reproduce it.

i don't have TB set to retrieve new news messages periodically, but i do for
mail and RSS.  it's a possibility that this timer might be involved as it's the
only thing i know set to check every 10-15min.  i have news set up to never
delete messages and i don't work offline so the bodies don't get retrieved until
i click to view them.

i'll try turning off the check for new mail timers and download message bodies
beforehand to see if makes a difference.

Reproducible: Sometimes

Steps to Reproduce:
(Reporter)

Comment 1

12 years ago
nope, neither did.  but i think i noticed a bit of a pattern.

- have x number of newsgroups, each with unread messages.
- check the server for new messages.
- there are new messages retrieved for some but not all of the groups.
- a short time later the groups that did happen to have new messages available
will keep their unread count while groups that didn't get reset to 0.

Comment 2

12 years ago
Had it happen to me on 2 different machines with slightly different
configuration but same news server (news.supernews.com).

I think that it is something the news message expiring system. I have it set to:
- Delete messages more than 45 days.
- Only message body less than 30 days old.

Comment 3

12 years ago
This bug also affects Windows. It's not consistent but it happens enough that
I've been watching this bug. I only recently realized that it's marked for Linux.

Updated

12 years ago
Severity: minor → normal
Status: UNCONFIRMED → NEW
Component: General → MailNews: Backend
Ever confirmed: true
OS: Linux → All
Product: Thunderbird → Core
Version: unspecified → Trunk

Comment 4

12 years ago
*** Bug 301057 has been marked as a duplicate of this bug. ***

Comment 5

12 years ago
(In reply to comment #0)

> after retrieving message headers from the news server, often after a short time
> (15 minutes or so) the unread messages count for each news group other than the
> one selected will vanish.

Hey, I'm not alone :-) See https://bugzilla.mozilla.org/show_bug.cgi?id=301057

Meanwhile I can say that exactly every 5minutes some newsgroups are affected,
until all unread message counts vanished.

> choosing a group will cause the previous count to
> reappear or update if any new messages are available.

Or you simply fold in and out the server in the folderpane.

> this seems to happen
> sporadically and so far i don't have a reliable way to reproduce it.

Here every time/5min and absolutly reproduceable.

> i don't have TB set to retrieve new news messages periodically, but i do for
> mail and RSS.  it's a possibility that this timer might be involved as it's the
> only thing i know set to check every 10-15min.

I searched 'about:config' for events that starts every 5minutes. I changed all
found items to an other time, but the Bug still works in his 5min timeperiode.

I also do not find any errors in smtp/nntp-logs.

Comment 6

12 years ago
Seen in Seamonkey on Win32 and OS/2.

Comment 7

12 years ago
(In reply to comment #6)
> Seen in Seamonkey on Win32 and OS/2.

Thanks for reporting.
Is that bug new for you? Did you change the version/build? Buildnumber?

Comment 8

12 years ago
(In reply to comment #7)
> Is that bug new for you? Did you change the version/build? Buildnumber?

No, it's been around for quite a while (1.0a release and earlier, haven't tried later builds).  I just now bothered to look for it being listed in Bugzilla.  :)
(Reporter)

Comment 9

12 years ago
i think this occurs for newsgroups that have new messages available but the headers of those msgs haven't been downloaded.  once you actually click on the newsgroup the headers are grabbed and it seems to work (for that group).

Comment 10

12 years ago
(In reply to comment #9)
> i think this occurs for newsgroups that have new messages available but the
> headers of those msgs haven't been downloaded.

Yes.

> once you actually click on the
> newsgroup the headers are grabbed and it seems to work (for that group).

The newsgroup in the focus is never affected and postings that once was read and after that marked as unread again are also not affected.

For me the most interesting thing is, the 5minutes-intervall the bug do his work. Three persons verifying the bug now. Should we set blocking to '?' now?
(Reporter)

Comment 11

12 years ago
nah, without at least an idea of what the cause is i doubt this qualifies, especially since it's so close to release time.  it's annoying but it's been around since at least 0.8.  a little longer won't hurt. ;)
(Reporter)

Comment 12

12 years ago
updated summary to be more relevant.

it might also be worth mentioning that if there are unread messages that _do_ have their headers downloaded (eg. during a previous session) the unread count will revert to that number.  
Summary: unread newsgroup counts disappear after a short time → newsgroups periodically lose unread count for messages whose headers haven't been downloaded
(Reporter)

Comment 13

12 years ago
okay, this is getting even worse on trunk.  all you have to do now is hover the cursor over a newsgroup title for a couple seconds and watch the unread message count disappear.  can anyone confirm if this happens on the branch?

Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9a1) Gecko/20051223 Thunderbird/1.6a1 ID:2005122306

req blocking.
Flags: blocking1.9a1?

Comment 14

12 years ago
(In reply to comment #13)

> okay, this is getting even worse on trunk.  all you have to do now is hover the
> cursor over a newsgroup title for a couple seconds and watch the unread message
> count disappear.  can anyone confirm if this happens on the branch?

You mean, I only have to put the mouse over a newsgroupname in the folderpane and after some seconds the newsgroup lost her unread message count?

No, I cant confirm that, but the unread message count always disappears periodical after exact five minutes. Every five minutes three or four NGs are affected until all NGs are thru.

> req blocking.

Yes ACK.

Comment 15

12 years ago
(In reply to comment #13)

> Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9a1) Gecko/20051223
> Thunderbird/1.6a1 ID:2005122306

Okay, now I tested with Thunderbirds actual nightly-build, and now I can confirm that. I made a little .avi from my screen to show what happend here.

The DivX-avi as a zip-file(only 49KB) is here: http://lahls.de/public/counts_gone.zip

Comment 16

12 years ago
Seen in Thunderbird on Win XP SP2

Comment 17

12 years ago
(In reply to comment #0)
> after retrieving message headers from the news server, often after a short time
> (15 minutes or so) the unread messages count for each news group other than the
> one selected will vanish.

Meanwhile the bug had made it into Branch. Some other people in de.comm.software.mozilla.mailnews confirmed the bug for SM1.0

Someone should look for it now.

Updated

11 years ago
Hardware: PC → All
*** Bug 335843 has been marked as a duplicate of this bug. ***

Comment 19

11 years ago
Given the frequency of complaints about this in <news://news.mozilla.org:119/mozilla.support.thunderbird>, could the priority of this be raised?  

Comment 20

11 years ago
*** Bug 355896 has been marked as a duplicate of this bug. ***

Comment 21

11 years ago
I saw this yesterday in an extreme form.  I subscribe to 18 newsgroups on one server.  I opened the server.  Before Thunderbird finished working its way down the list, refreshing the unread counts on each newsgroup, the counts that had already been refreshed started going blank with the newsgroup names changing from bold to normal.  
(Assignee)

Comment 22

11 years ago
Created attachment 248023 [details] [diff] [review]
work in progress

The problem is basically that hovering over a message causes us to open the db, which causes us to only count the headers we've downloaded. I have a patch that basically borrows a concept from the imap code: pending counts, i.e., counts for headers we know about but haven't downloaded yet.

This doesn't totally fix the problem, but I think it improves things quite a bit - if anyone wants to try to run with this patch, please give it a shot.
Assignee: mscott → bienvenu
Status: NEW → ASSIGNED

Comment 23

11 years ago
(In reply to comment #22)
> The problem is basically that hovering over a message

For me its not necessary to hovering the groups. The unread message counts begins to disappears if I wait exactly 5 minutes. Then the first 4 or 5 groups lost their counts. After another 5 minutes the next groups are following, and so on until all groups lost their counts. Reproducible: always since more than a year :-(

> basically borrows a concept from the imap code: pending counts, i.e., counts
> for headers we know about but haven't downloaded yet.
> 
> This doesn't totally fix the problem, but I think it improves things quite a
> bit - if anyone wants to try to run with this patch, please give it a shot.

Sorry, I can not test the patch cause I use Tinderbox, but I like to test a compiled win32-build with this patch included.

Comment 24

11 years ago
Tried the patch, it certainly fixes bug 318792! 
(Can't say about this bug though, since i think I haven't run in to it.)
(Assignee)

Comment 25

11 years ago
Comment on attachment 248023 [details] [diff] [review]
work in progress

the patch definitely seems like an improvement
Attachment #248023 - Flags: superreview?(mscott)

Updated

11 years ago
Attachment #248023 - Flags: superreview?(mscott) → superreview+
(Assignee)

Comment 26

11 years ago
this fix is landed - I think there are still issues, but I think the main issue is fixed. 
Status: ASSIGNED → RESOLVED
Last Resolved: 11 years ago
Keywords: fixed1.8.1.1
Resolution: --- → FIXED

Comment 27

11 years ago
(In reply to comment #26)
> this fix is landed - I think there are still issues, but I think the main
> issue is fixed.

Is this reasonable?  The original bug was filed before the New Message Folder Popup was even implemented, and (as already noted) bug 318792 was open already for that popup-related problem.

Also, I just ran across bug 325299, which seems to be a dupe of this original bug.
(Assignee)

Comment 28

11 years ago
this fix has nothing to do with new mail alerts, and the original bug was just a side effect of new mail alerts - the problem arose whenever the db for the newsgroup was opened - we would throw away all the counts we got from the server, and just use the counts of headers we'd downloaded. Hovering over a newsgroup was causing us to open the db, and exposing the bug. As did having retention settings run, which also causes the db to get opened.

Comment 29

11 years ago
(In reply to comment #26)

> this fix is landed - I think there are still issues, but I think the main issue
> is fixed. 

I would like to test the patch, but their are no new 'WINNT 5.2 sea-win32-tbox Clbr VM-release'-builds since nearly three days in Tinderbox. Isn't it possible to fix that VM?
(Assignee)

Comment 30

11 years ago
see bug 363292 for the tinderbox issues.

Comment 31

11 years ago
(In reply to comment #30)
> see bug 363292 for the tinderbox issues.

Thanks David. Meanwhile it looks like they set up another VM which build a nightly.
(Assignee)

Comment 32

11 years ago
*** Bug 325299 has been marked as a duplicate of this bug. ***

Comment 33

11 years ago
(In reply to comment #26)
> this fix is landed - I think there are still issues, but I think the main issue
> is fixed. 

Newsgroups don't lose unread counts anymore. But I think, their is a regression.
I opened a new bug[1], because I can not exactly say, that this patch cause the bug.

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=363966
(Assignee)

Comment 34

11 years ago
Created attachment 249328 [details] [diff] [review]
fix counts issue

If there are missing headers on the server (e.g., a message is cancelled), we were always showing that message as unread. So instead of clearing the pending counts when we start opening the folder, clear them when we've finished downloading headers for the folder. This fixes the problem for me...
Attachment #249328 - Flags: superreview?(mscott)
(Assignee)

Comment 35

11 years ago
Created attachment 249369 [details] [diff] [review]
adjust unread counts for message headers we don't have

this goes through the unread messages and checks if we have the headers for all of them - if we don't, it marks them read. 

This works for me - the one thing I'm unhappy about is that it does this every time you click on a newsgroup, and if you keep thousands of unread messages, we'll  check them all every time. Ideally, we should keep track of what we've already checked and not check them again. I'll see if I can do that.

This should fix a really long time frustrating issue for news readers...
(Assignee)

Comment 36

11 years ago
Created attachment 249378 [details] [diff] [review]
adjust unread counts for message headers we don't have, but just once for any given header

this keeps track of the range of headers we've checked previously. 

If an article expires after we've downloaded the header, I don't think we care, since the user can mark it read anyway.
Attachment #249369 - Attachment is obsolete: true
Attachment #249378 - Flags: superreview?(mscott)

Updated

11 years ago
Attachment #249378 - Flags: superreview?(mscott) → superreview+

Comment 37

11 years ago
Comment on attachment 249328 [details] [diff] [review]
fix counts issue

I'm assuming we want both this patch and the other one  I just reviewed for this bug...
Attachment #249328 - Flags: superreview?(mscott) → superreview+
(Assignee)

Comment 38

11 years ago
I've landed those two fixes on the trunk and branch as well.

Comment 39

11 years ago
(In reply to comment #36)
> Created an attachment (id=249378) [edit]
> adjust unread counts for message headers we don't have, but just once for any
> given header

Thanks David. The patches works perfect for me.

Updated

11 years ago
Blocks: 363966, 364063

Comment 40

11 years ago
In what release of Thunderbird was this fixed?  I seem to still see this problem in Thunderbird version 1.5.0.10 (20070221) with WindowsXP Sp2.    

Comment 41

11 years ago
The fix is in the Gecko 1.8.1 branch , not the 1.8.0 branch.  That means Thunderbird 2.x and Seamonkey 1.1.x.

Updated

10 years ago
Flags: blocking1.9a1?
Product: Core → MailNews Core
You need to log in before you can comment on or make changes to this bug.