Closed Bug 298737 Opened 19 years ago Closed 18 years ago

newsgroups periodically lose unread count for messages whose headers haven't been downloaded

Categories

(MailNews Core :: Backend, defect)

Product:
MailNews Core
Component:
Backend
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: dirtyepic, Assigned: Bienvenu)

References

Details

(Keywords: fixed1.8.1.1)

Attachments

(3 files, 1 obsolete file)

work in progress
2.25 KB, patch
mscott
: superreview+
Details | Diff | Splinter Review
fix counts issue
3.88 KB, patch
mscott
: superreview+
Details | Diff | Splinter Review
adjust unread counts for message headers we don't have, but just once for any given header
3.40 KB, patch
mscott
: superreview+
Details | Diff | Splinter Review 
User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8b2) Gecko/20050623 Firefox/1.0+
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8b2) Gecko/20050623 Firefox/1.0+

last produced with TB 1.0+ 20050623.  first noticed over a month before this.

after retrieving message headers from the news server, often after a short time
(15 minutes or so) the unread messages count for each news group other than the
one selected will vanish.  choosing a group will cause the previous count to
reappear or update if any new messages are available.  this seems to happen
sporadically and so far i don't have a reliable way to reproduce it.

i don't have TB set to retrieve new news messages periodically, but i do for
mail and RSS.  it's a possibility that this timer might be involved as it's the
only thing i know set to check every 10-15min.  i have news set up to never
delete messages and i don't work offline so the bodies don't get retrieved until
i click to view them.

i'll try turning off the check for new mail timers and download message bodies
beforehand to see if makes a difference.

Reproducible: Sometimes

Steps to Reproduce:
nope, neither did.  but i think i noticed a bit of a pattern.

- have x number of newsgroups, each with unread messages.
- check the server for new messages.
- there are new messages retrieved for some but not all of the groups.
- a short time later the groups that did happen to have new messages available
will keep their unread count while groups that didn't get reset to 0.
Had it happen to me on 2 different machines with slightly different
configuration but same news server (news.supernews.com).

I think that it is something the news message expiring system. I have it set to:
- Delete messages more than 45 days.
- Only message body less than 30 days old.
This bug also affects Windows. It's not consistent but it happens enough that
I've been watching this bug. I only recently realized that it's marked for Linux.
Severity: minor → normal
Status: UNCONFIRMED → NEW
Component: General → MailNews: Backend
Ever confirmed: true
OS: Linux → All
Product: Thunderbird → Core
Version: unspecified → Trunk
*** Bug 301057 has been marked as a duplicate of this bug. ***
(In reply to comment #0)

> after retrieving message headers from the news server, often after a short time
> (15 minutes or so) the unread messages count for each news group other than the
> one selected will vanish.

Hey, I'm not alone :-) See https://bugzilla.mozilla.org/show_bug.cgi?id=301057

Meanwhile I can say that exactly every 5minutes some newsgroups are affected,
until all unread message counts vanished.

> choosing a group will cause the previous count to
> reappear or update if any new messages are available.

Or you simply fold in and out the server in the folderpane.

> this seems to happen
> sporadically and so far i don't have a reliable way to reproduce it.

Here every time/5min and absolutly reproduceable.

> i don't have TB set to retrieve new news messages periodically, but i do for
> mail and RSS.  it's a possibility that this timer might be involved as it's the
> only thing i know set to check every 10-15min.

I searched 'about:config' for events that starts every 5minutes. I changed all
found items to an other time, but the Bug still works in his 5min timeperiode.

I also do not find any errors in smtp/nntp-logs.
Seen in Seamonkey on Win32 and OS/2.

(In reply to comment #6)
> Seen in Seamonkey on Win32 and OS/2.

Thanks for reporting.
Is that bug new for you? Did you change the version/build? Buildnumber?
(In reply to comment #7)
> Is that bug new for you? Did you change the version/build? Buildnumber?

No, it's been around for quite a while (1.0a release and earlier, haven't tried later builds).  I just now bothered to look for it being listed in Bugzilla.  :)

i think this occurs for newsgroups that have new messages available but the headers of those msgs haven't been downloaded.  once you actually click on the newsgroup the headers are grabbed and it seems to work (for that group).
(In reply to comment #9)
> i think this occurs for newsgroups that have new messages available but the
> headers of those msgs haven't been downloaded.

Yes.

> once you actually click on the
> newsgroup the headers are grabbed and it seems to work (for that group).

The newsgroup in the focus is never affected and postings that once was read and after that marked as unread again are also not affected.

For me the most interesting thing is, the 5minutes-intervall the bug do his work. Three persons verifying the bug now. Should we set blocking to '?' now?
nah, without at least an idea of what the cause is i doubt this qualifies, especially since it's so close to release time.  it's annoying but it's been around since at least 0.8.  a little longer won't hurt. ;)
updated summary to be more relevant.

it might also be worth mentioning that if there are unread messages that _do_ have their headers downloaded (eg. during a previous session) the unread count will revert to that number.
Summary: unread newsgroup counts disappear after a short time → newsgroups periodically lose unread count for messages whose headers haven't been downloaded
okay, this is getting even worse on trunk.  all you have to do now is hover the cursor over a newsgroup title for a couple seconds and watch the unread message count disappear.  can anyone confirm if this happens on the branch?

Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9a1) Gecko/20051223 Thunderbird/1.6a1 ID:2005122306

req blocking.
Flags: blocking1.9a1?
(In reply to comment #13)

> okay, this is getting even worse on trunk.  all you have to do now is hover the
> cursor over a newsgroup title for a couple seconds and watch the unread message
> count disappear.  can anyone confirm if this happens on the branch?

You mean, I only have to put the mouse over a newsgroupname in the folderpane and after some seconds the newsgroup lost her unread message count?

No, I cant confirm that, but the unread message count always disappears periodical after exact five minutes. Every five minutes three or four NGs are affected until all NGs are thru.

> req blocking.

Yes ACK.
(In reply to comment #13)

> Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9a1) Gecko/20051223
> Thunderbird/1.6a1 ID:2005122306

Okay, now I tested with Thunderbirds actual nightly-build, and now I can confirm that. I made a little .avi from my screen to show what happend here.

The DivX-avi as a zip-file(only 49KB) is here: http://lahls.de/public/counts_gone.zip
Seen in Thunderbird on Win XP SP2
(In reply to comment #0)
> after retrieving message headers from the news server, often after a short time
> (15 minutes or so) the unread messages count for each news group other than the
> one selected will vanish.

Meanwhile the bug had made it into Branch. Some other people in de.comm.software.mozilla.mailnews confirmed the bug for SM1.0

Someone should look for it now.
Hardware: PC → All
*** Bug 335843 has been marked as a duplicate of this bug. ***
Given the frequency of complaints about this in <news://news.mozilla.org:119/mozilla.support.thunderbird>, could the priority of this be raised?  
*** Bug 355896 has been marked as a duplicate of this bug. ***
I saw this yesterday in an extreme form.  I subscribe to 18 newsgroups on one server.  I opened the server.  Before Thunderbird finished working its way down the list, refreshing the unread counts on each newsgroup, the counts that had already been refreshed started going blank with the newsgroup names changing from bold to normal.
Attached patch work in progress — — Splinter Review 
The problem is basically that hovering over a message causes us to open the db, which causes us to only count the headers we've downloaded. I have a patch that basically borrows a concept from the imap code: pending counts, i.e., counts for headers we know about but haven't downloaded yet.

This doesn't totally fix the problem, but I think it improves things quite a bit - if anyone wants to try to run with this patch, please give it a shot.
Assignee: mscott → bienvenu
Status: NEW → ASSIGNED
(In reply to comment #22)
> The problem is basically that hovering over a message

For me its not necessary to hovering the groups. The unread message counts begins to disappears if I wait exactly 5 minutes. Then the first 4 or 5 groups lost their counts. After another 5 minutes the next groups are following, and so on until all groups lost their counts. Reproducible: always since more than a year :-(

> basically borrows a concept from the imap code: pending counts, i.e., counts
> for headers we know about but haven't downloaded yet.
> 
> This doesn't totally fix the problem, but I think it improves things quite a
> bit - if anyone wants to try to run with this patch, please give it a shot.

Sorry, I can not test the patch cause I use Tinderbox, but I like to test a compiled win32-build with this patch included.
Tried the patch, it certainly fixes bug 318792! 
(Can't say about this bug though, since i think I haven't run in to it.)
Comment on attachment 248023 [details] [diff] [review]
work in progress

the patch definitely seems like an improvement
Attachment #248023 - Flags: superreview?(mscott)
Attachment #248023 - Flags: superreview?(mscott) → superreview+
this fix is landed - I think there are still issues, but I think the main issue is fixed.
Status: ASSIGNED → RESOLVED
Closed: 18 years ago
Keywords: fixed1.8.1.1
Resolution: --- → FIXED
(In reply to comment #26)
> this fix is landed - I think there are still issues, but I think the main
> issue is fixed.

Is this reasonable?  The original bug was filed before the New Message Folder Popup was even implemented, and (as already noted) bug 318792 was open already for that popup-related problem.

Also, I just ran across bug 325299, which seems to be a dupe of this original bug.
this fix has nothing to do with new mail alerts, and the original bug was just a side effect of new mail alerts - the problem arose whenever the db for the newsgroup was opened - we would throw away all the counts we got from the server, and just use the counts of headers we'd downloaded. Hovering over a newsgroup was causing us to open the db, and exposing the bug. As did having retention settings run, which also causes the db to get opened.
(In reply to comment #26)

> this fix is landed - I think there are still issues, but I think the main issue
> is fixed. 

I would like to test the patch, but their are no new 'WINNT 5.2 sea-win32-tbox Clbr VM-release'-builds since nearly three days in Tinderbox. Isn't it possible to fix that VM?
see bug 363292 for the tinderbox issues.
(In reply to comment #30)
> see bug 363292 for the tinderbox issues.

Thanks David. Meanwhile it looks like they set up another VM which build a nightly.
*** Bug 325299 has been marked as a duplicate of this bug. ***
(In reply to comment #26)
> this fix is landed - I think there are still issues, but I think the main issue
> is fixed. 

Newsgroups don't lose unread counts anymore. But I think, their is a regression.
I opened a new bug[1], because I can not exactly say, that this patch cause the bug.

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=363966
Attached patch fix counts issue — — Splinter Review 
If there are missing headers on the server (e.g., a message is cancelled), we were always showing that message as unread. So instead of clearing the pending counts when we start opening the folder, clear them when we've finished downloading headers for the folder. This fixes the problem for me...
Attachment #249328 - Flags: superreview?(mscott)
this goes through the unread messages and checks if we have the headers for all of them - if we don't, it marks them read. 

This works for me - the one thing I'm unhappy about is that it does this every time you click on a newsgroup, and if you keep thousands of unread messages, we'll  check them all every time. Ideally, we should keep track of what we've already checked and not check them again. I'll see if I can do that.

This should fix a really long time frustrating issue for news readers...
this keeps track of the range of headers we've checked previously. 

If an article expires after we've downloaded the header, I don't think we care, since the user can mark it read anyway.
Attachment #249369 - Attachment is obsolete: true
Attachment #249378 - Flags: superreview?(mscott)
Attachment #249378 - Flags: superreview?(mscott) → superreview+
Comment on attachment 249328 [details] [diff] [review]
fix counts issue

I'm assuming we want both this patch and the other one  I just reviewed for this bug...
Attachment #249328 - Flags: superreview?(mscott) → superreview+
I've landed those two fixes on the trunk and branch as well.
(In reply to comment #36)
> Created an attachment (id=249378) [edit]
> adjust unread counts for message headers we don't have, but just once for any
> given header

Thanks David. The patches works perfect for me.
Blocks: 363966, 364063
In what release of Thunderbird was this fixed?  I seem to still see this problem in Thunderbird version 1.5.0.10 (20070221) with WindowsXP Sp2.    
The fix is in the Gecko 1.8.1 branch , not the 1.8.0 branch.  That means Thunderbird 2.x and Seamonkey 1.1.x.
Flags: blocking1.9a1?
Product: Core → MailNews Core
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: