138222 - Newsgroups: broken threads, lost articles

Status: VERIFIED DUPLICATE of bug 131983

(MailNews Core :: Database, defect)

Description

[Rainer Bielefeld]

Mozilla/5.0 (Windows; U; Win98; de-AT; rv:0.9.9+) Gecko/20020416
and all other mozillass beginning with 0.9.7 I have in use have Problems with
newsgroups- threads.
Actually german language Pack is not activated.

Please have a look to de.comm.software.mozilla
thread <Gibt es eine Möglichkeit MIME-Types"umzuleiten"?>

With mozilla the thread looks like att att.2 'treadmoz1.jpg' shows,
and some destroyed posts as shown in att.3 'treadmoz2.jpg'
(sender and date are interesting own problems)
have their own thread. 

With NC7.7 in looks like shown in att.1 'threadNC.jpg'
(here interesting only post of "Timon Christl")

Comment 1

[Rainer Bielefeld]

Attachment: att.1 thread under NC4.7

correct thread view

Comment 2

[Rainer Bielefeld]

Attachment: att.2 Thread with mozilla

some posts are missing

Comment 3

[Rainer Bielefeld]

Attachment: att.3 the missing (broken) posts. with mozilla

missing post(s) of att. 2

I cannot view the message-source of the posts with curious sender

Comment 4

[David :Bienvenu]

looks like a mime header parsing problem...can you view the message source with
4.7 and attach here?

Comment 5

[Rainer Bielefeld]

Attachment: message - source of "durious" posting

Reply to  #4 
Here the source of Timon Christl - posting

Comment 6

[David :Bienvenu]

Cc'ing jgmyers who can probably look at that header and say what's going wrong
in thrice.

Comment 7

[Rainer Bielefeld]

There are several other mutilated postings, if you think it could make things
more clearly please ask for more message-sources

May be a look to netscape.public.mozilla.performance is interesting:
sort by date and look for 01.01.70 01:00

Comment 8

[John G. Myers]

Headers look OK.

Have there been any previous crashes or unclean shutdowns?  In such cases, the
.msf file will get out of sync with reality.  Try quitting Mozilla, removing the
.msf file, and restarting Mozilla.  Please then report whether or not the
missing posts then appear.

Comment 9

[Rainer Bielefeld]

add for #7
on newsserver news.mozilla.org

Comment 10

[Adam Macznik]

I saw this bug with Mozilla 1.0 RC1 with a different newsgroup and tried to
investigate it. That's what I've found:

1. Bug does not appear in Moz 0.9.8 release. Never used nightlies, so I can't
narrow it down.
2. Problem is probably with MIME headers with international characters.

I will attach some screenshots to illustrate.

Now the detailed description of my tests.
1. Used my 098 profile with some newsgroups already loaded. Connected to 
   a newsgroup and downloaded two messages. They both had headers like this:
     Subject: =?iso-8859-2?Q?Re:_Co_si=EA_sta=B3o_z_http://www.ztm.waw.pl_=3F?=
   They appeared in a thread pane with some garbage in the sender column and a
   date of 1970-01-01, like in the reporter's case.
   Other messages in this thread had a Subject encoded like this:
     Subject: Re: Co =?ISO-8859-2?Q?si=EA=20sta=B3o?= z http://www.ztm.waw.pl ?
   and those were displayed properly.

2. Created new profile for 1.0 and dl'd that newsgroup again. This time
   there were more messages in this thread displayed improperly. Screenshot
   in att.1 shows this.
   Properly displayed message had headers not encoded:
     From: "£ukasz Kalbarczyk" <[e-mail]>
     Subject: Re: Odp: Co siê sta³o z http://www.ztm.waw.pl ?
   (view with ISO 8859-2)
   or encoded like (posted 2002-04-19 18:54 on screenshot):
     Subject: OT Re: Odp: Co =?ISO-8859-2?Q?si=EA=20sta=B3o?= z http://www.ztm.waw.p
        ?
   or like (posted 2002-04-19 23:43):
     Subject: Re: Co =?ISO-8859-2?Q?si=EA_sta=B3o_z_http=3A//www=2Eztm=2E?=
        =?ISO-8859-2?Q?waw=2Epl_=3F?=

   Messages displayed with garbage had subjects like:
     Subject: Re: Co =?iso-8859-2?q?si=EA_sta=B3o?= z http://www.ztm.waw.pl ?
   or, like in previous case:
     Subject: =?iso-8859-2?Q?Re:_Co_si=EA_sta=B3o_z_http://www.ztm.waw.pl_=3F?= 

   Note, that the "right" ones with encoding were two-liners - maybe that's
   the difference?
   Also note that the sender and date in message pane are OK, only the
   thread pane displays wrong info - similar to bug 115240.
   Additionally, I could not view source of "garbaged" messages (about:blank
   was displayed).

3. When browsing through "garbaged" thread (as seen in att.1) the body pane
   displayed still the same message (header-body mismatch).
   Then I downloaded this newsgroup for offline. Downloaded msgs in "garbaged"
   thread were now displayed better - bodies were now different in different
   messages (!).

4. I copied the dl'd news file to Local Folders. Looked at this thread and
   there was no garbage in 'Sender' column, but still some messages had
   1970-01-01 displayed in 'Date' column. I selected those 6 messages and
   copied them to separate local folder, which I am attaching now as att.2.
   Note that the all have Subject "all encoded" (i.e. like
     Subject: =?iso-8859-2?Q?Re:_Co_si=EA_sta=B3o_z_http://www.ztm.waw.pl_=3F?=
   ).
   I also added a message displayed in the message pane on screenshot in
   attachment att.1.

5. But then I used a text editor to search for other mesgs "all encoded" in
   that local folder - and there were many. They all display properly both in
   the newsgroup and in the copied local folder. Now I am all confused....
   Also added one of these messages to my att.2 (the one from "thingy 3209").

Whew... Hope this helps.

This bug is very bad for me - I think I won't switch to 1.0 for now just
because of that.

Comment 11

[Adam Macznik]

Attachment: Screenshot of garbage in thread pane (att.1).

It shows the "garbaged thread" with some proper messages.
Also note it's okay in message pane.

Comment 12

[Adam Macznik]

Attachment: Some "garbaged" messages in mbox format (att.2).

... with added two more testcases. Load it to your Local Folders and you
will see 6 messages with date 1970-01-01 in thread pane, but right info in
message pane.

Comment 13

[Boris 'pi' Piwinger]

I can confirm this problem with Mozilla/5.0 (X11; U; Linux i686; en-US;
rv:1.0rc1) Gecko/20020419 -> Status=New, OS=ALL

Further observation: You cannot view the source of the postings which are
displayed outside the threading as garbage, even though you can read them normally.

When I leave the group and come back they are not displayed anymore, hence lost.
-> Severity=critical, Keywords=dataloss, adjusting Summary

Other as the reporter, for me it worked in 0.9.9 and all previous version. Only
the latest nightlies (estimate: two weeks) show this bug.

pi

Comment 14

[Boris 'pi' Piwinger]

Re comment 8: Deleding .msf is a PITA. Anyways, I tried with a relatively small
group. Of course, now all articles were displayed again, still with the problem
of garbage display. Leaving the group and returning gave the same result. The
article in question was gone. So the problem is even worse than I thought
before. Adding mozilla1.0 keyword.

pi

Comment 15

[David :Bienvenu]

Naoki, did something change in the parsing of mime headers that might have
caused this?

Comment 16

[David :Bienvenu]

OK, I stepped through the code, and it's NS_MsgStripRE that's corrupting the
date header when we're parsing the headers. My guess is that all these problems
have to do with the recent changes to NS_MsgStripRE -> Naoki.

Comment 17

[Frank Widmaier]

the postings from 1970 will be "downloaded" regularly in _some_ newsgroups. When
I unsubscribe/subscrribe them, it wont work. The bug appears only in a few
special groups..

Comment 18

[Mario Mueller]

I think, the bug occurs because the posting has a Content-Transfer-Encoding:
8Bit (and not 8bit). In some wrong postings I have seen the following header lines:

  Content-Transfer-Encoding: 8Bit

If the Content-Transfer-Encoding is written in lowercase letters

  Content-Transfer-Encoding: 8bit

the postings are shown correctly. As the Content-Transfer-Encoding is not case
sensitive, it seems to be a bug in Mozilla.

Comment 19

[nhottanscp]

This is a dup of bug 131983.


*** This bug has been marked as a duplicate of 131983 ***

Comment 20

[Boris 'pi' Piwinger]

Right, this is a dupe.

Pleae transfer your votes manually.

pi

Comment 21

[marina]

fixed by bug 131983