148492 - Only new new news posts and a handful of old displayed

Status: VERIFIED DUPLICATE of bug 145355

(MailNews Core :: Database, defect)

Description

[Daniel Bratell]

From Bugzilla Helper:
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0+)
Gecko/20020317
BuildID:    CVS 2002-06-01 but started months before

I've had this problem for months and always figured that it was such big a
problem that there would be many bugs filed about it and that it would be fixed
soon but that doesn't seem to be the case.

The problem is that in the lists of posts for a news group only new downloads
and a couple of old downloads are shown. When I delete the msf-file and download
all posts again I get to see them all but after restarting once most are gone
and after restarting a couple of times only a handful remain. I've yet to see a
pattern in which posts that are displayed except for the new ones which are
always there.

I do have View->Messages->All selected but I have tried other settings without
any change.

I'm willing to do some debugging on this if someone tells me where to put my
breakpoints. Where are the mails that are going to be displayed selected in the
code?

Reproducible: Always
Steps to Reproduce:
For me, but obviously not for everyone:
1. Delete the .msf file for a newsgroup
2. Download last 50 and mark rest read.
3. Read some posts.
4. Restart Mozilla
5. See the newsgroup and notice that not all of the posts displayed previously
are displayed now.
6. Iterate from 4.

Actual Results:  Only some posts are displayed in the post list.

Expected Results:  Everything downloaded should be displayed (according to the
view->messages setting).

Comment 1

[R.K.Aa.]

Do you have any checked settings in the pref for "Remove message bodies older
than" ? (Asking because i noticed it's "invisible" in 1.0, the pref pane isn't
tall enough to display that option.)

ALso: Can you reproduce if also with a new profile?

Comment 2

[Daniel Bratell]

Oh, I hadn't seen that pref. It was set to keep messages for 30 days and then
only keep unread. I've removed the time limit now but I still have the same problem.

I'm not sure I've seen it in a clean profile. I've manually recreated
localstore.rdf and went through prefs.js without noticing anything wrong so it
may be triggered by some of my settings. I've recreated the news account (in the
same profiler) without changing the behaviour.

I just tried it with the n.p.m.jsdebugger newsgroup and noticed that I didn't
had to restart the program. I deleted the jsdebugger.msf file and answered "yes"
to the question about downaloading ~120 headers. 98 headers were downloaded
(everything read since before). Then I selected another newsgroup and went back
to jsdebugger. Now it was down to 70 messages. Going back and forth decreased
the number by ~10-20% each time. I'm now down to 19 messages. 

Comment 3

[R.K.Aa.]

if you set it to only keep unread... could it be that the ones you don't see by
and by are indeed the read messages?

Comment 4

[Daniel Bratell]

Yes, I don't think I've ever seen it hide an unread message, but I can not be
completely sure since Mozilla messes up the unread count every now and then.

Now, I've think I've narrowed it down somewhat. It seems to occur only when I have 
"Disk Space: When it's time to clean up messages: Keep messages which have
arrived within past XX days" selected I lose messages. I had it on 30 days but
it isn't just those messages that are lost, and it isn't all those messages that
are hidden either. The removal of messages looks kind of random.

By the way, "when it's time to clean up messages", when is that?

Comment 5

[David :Bienvenu]

we age away both news hdrs and news bodies downloaded for offline use. It's true
that the purging of old news hdrs is somewhat random because of a bug (only hdrs
that should be removed get removed, but some hdrs that don't get removed should
get removed). I have a bug on that, and there's a fix in the bug, but I haven't
had a chance to get it reviewed and checked in.

Comment 6

[Daniel Bratell]

But I've seen headers removed that shouldn't be rmeoved too. Most often the
newsgroups only contained a handful of posts, but there had been posts every day
and my aging limit was 30 days.

Comment 7

[David :Bienvenu]

I thought you said to remove read messages as well (i.e., keep only unread messages)

Comment 8

[Daniel Bratell]

Ah, yes. Apparently I've misunderstood the meaning of the options. I thought
that talked about messages 30 days old, not all messages. Then this is a dupe of
your other bug.

Comment 9

[[e-mail] (gone - use [e-mail] instead)]

David, have the bug # handy for the bug you describe?

Comment 10

[David :Bienvenu]

bug 145355 - the patch is incomplete, however, because the other aging routines
are also broken.

Comment 11

[[e-mail] (gone - use [e-mail] instead)]

*** This bug has been marked as a duplicate of 145355 ***

Comment 12

[[e-mail] (gone - use [e-mail] instead)]

verified dup, per bienvenu and neil's comments...and my own cognition