39562 - hostinfo.dat "<name>,,x,y,z" needs to be fixed

Status: RESOLVED FIXED

(MailNews Core :: Networking: NNTP, defect)

Description

[(not reading, please use [e-mail] instead)]

I'm writing out newsgroups as "<name>,,x,y,z"

I have to write the code to the right thing here.

Comment 1

[(not reading, please use [e-mail] instead)]

aceepting.

Comment 2

[(not reading, please use [e-mail] instead)]

aceepting.

Comment 3

[scottputterman]

David,

Is this something that has to be fixed for rtm?  If is isn't could you move this 
to the future milestone?

Comment 4

[David :Bienvenu]

yes, I think this should be fixed for RTM. I'm not sure what should be between
those two commas, but I imagine it was there for a reason. I'll look into it in
beta3.

Comment 5

[lchiang]

correct milestone to align w/nsbeta3 nomination

Comment 6

[David :Bienvenu]

accepting

Comment 7

[David :Bienvenu]

reassigning back to Seth, removing nsbeta3 keyword. This is much further from
being done than I thought. I'm not going to have time to do this.

Comment 8

[(not reading, please use [e-mail] instead)]

clearing milestone, m17 and m18 are meaningless now.  these need to be triages
along with the rest.

Comment 9

[Håkan Waara]

I don't understand this bugreport.

Comment 10

[Thomas]

No action in more than 3 years, can't this bug be closed?

Comment 11

[Phil Ringnalda (:philor)]

No, the passage of time doesn't automatically erase broken things.

It's about the URL field: when nsNntpIncomingServer.cpp writes hostinfo.dat, it adds a useless and static ",,1,0,0" onto each line, apparently trying to be compatible with Netscape 4.x. If someone wanted to reverse-engineer what 4.x was doing there, and then wanted to use whatever it was, they could fix it to write actual data, or, we could save 7 bytes per group by just dropping it, since we no longer really have any desire to keep profile files easily migrated back and forth with 4.x, as long as whatever strips it off when it's read (which I can't seem to find) wouldn't mind seeing our newly comma-free version.

Comment 12

[Joshua Cranmer [:jcranmer]]

I could take a gander at what the fields mean:

<newsgroup name>,<description>,<first>,<last>,<num>.

I'm not sure how much of an impact being wrong would have, especially since we have our own hacked output to deal with anyways. I don't foresee the <first>,<last>,<num> being terribly useful in the future (they'd get out of date pretty quickly); the only one I see much use for is the <description> field (see bug 37775).

Comment 13

[Karsten Düsterloh]

de.comm.software.mozilla,,0,494fa09b,5f77
(In reply to comment #12)
> I could take a gander at what the fields mean:
> 
> <newsgroup name>,<description>,<first>,<last>,<num>.

I mostly disagree.
Here's a snippet from my today's ;-) Netscape 4.80 (en, Windows) hostinfo.dat file:

de.comm.software,,0,494fa098,15a2
de.comm.software.mozilla,,0,494fa09b,5f77
de.comm.software.mozilla.browser,,1,494fa09b,5f78
de.comm.software.mozilla.mailnews,,1,494fa09b,5f79
de.comm.software.mozilla.misc,,1,494fa09b,5f7a
de.comm.software.mozilla.nightly-builds,,1,494fa09b,5f7b

The second field is empty for all groups, on different servers. But I, too, suspect that it had been designated for either the ng tagline or maybe a customized name. I doubt it was ever implemented, though - if NS4 doesn't set it, who else?
I'm not that bored to decompile NS4. ;-)

My guess for x,y,z:

x   0=tree node, 1=actual newsgroup
y   494fa09b=1229955227=2008-12-22 14:13:47=>timestamp
    Some experimentation shows that it doesn't change with list updates or
    news download or news reading, so I suspect "group seen for the first time"
    (I newly created the newsserver account for this test.)
z   looks like a consecutive, newsserver-dependent ng number

Comment 14

[Karsten Düsterloh]

> z   looks like a consecutive, newsserver-dependent ng number

To be more precise:
Whenever Netscape sees a new group name, it builds a new entry for every missing hierarchy node, consecutively numbered. In my case, most of the output of a LIST command is unsorted (maybe roughly sorted by group creation), but you can see how z follows the order of this LIST dump.

Comment 15

[Joshua Cranmer [:jcranmer]]

Well, I highly doubt that preserving the x,y,z is important, since we've lived without it for years without everything blowing up in our face. If NS4 doesn't make use of that second position, we can insert whatever we want in there without anyone complaining.

I think I'll probably go ahead and strip the whole ,,1,0,0 stuff and make the input null-resistant if/when support for descriptions is added.

Comment 16

[Karsten Düsterloh]

(In reply to comment #15)
> I think I'll probably go ahead and strip the whole ,,1,0,0 stuff and make the
> input null-resistant if/when support for descriptions is added.

Sounds reasonable.
Since descriptions can contain commas, it wouldn't have worked anyway (without some escape mechanism)...

Comment 17

[Edmund Wong (:ewong)]

Taking this bug for a spin.

Comment 18

[Edmund Wong (:ewong)]

Attachment: Removed ,,x,y,z from the newsgroups lines in hostinfo.dat (v1)

Comment 19

[Karsten Düsterloh]

Comment on attachment 501002 [details] [diff] [review]
Removed ,,x,y,z from the newsgroups lines in hostinfo.dat (v1)

>+++ b/mailnews/news/src/nsNntpIncomingServer.cpp
>@@ -811,9 +811,9 @@ writeGroupToHostInfoFile(nsCString &aEle
>         return PR_FALSE;
>     }
>     PRUint32 bytesWritten;
>-    // XXX todo ",,1,0,0" is a temporary hack, fix it
>+

No empty line needed.

>     stream->Write(aElement.get(), aElement.Length(), &bytesWritten);
>-    stream->Write(",,1,0,0"MSG_LINEBREAK, 7 + MSG_LINEBREAK_LEN, &bytesWritten);
>+    stream->Write(MSG_LINEBREAK, MSG_LINEBREAK_LEN, &bytesWritten);
>     return PR_TRUE;

The change also craves for a comment before this:

1265     char *commaPos = (char *) PL_strchr(line,',');
1266     if (commaPos) *commaPos = 0;

Like "Old hostinfo files (v1) had additional data fields, delimited by commas."
And, of course, you need to increment VALID_VERSION.

Comment 20

[Edmund Wong (:ewong)]

Attachment: Removed ,,x,y,z from the newsgroups lines in hostinfo.dat (v2)

Comment 21

[Karsten Düsterloh]

Comment on attachment 504677 [details] [diff] [review]
Removed ,,x,y,z from the newsgroups lines in hostinfo.dat (v2)

>   if (mHasSeenBeginGroups) {
>+  	// v1 hostinfo files had additional data fields delimited by commas.
>+  	// with v2 hostinfo files, the additional data fields are removed.
>     char *commaPos = (char *) PL_strchr(line,',');

Don't use tabs for indentation, just spaces.
r=me with that.

Comment 22

[Edmund Wong (:ewong)]

Attachment: Removed ,,x,y,z from the newsgroups lines in hostinfo.dat (v3)

Fixed nit from comment #21.

Comment 23

[David :Bienvenu]

Comment on attachment 512074 [details] [diff] [review]
Removed ,,x,y,z from the newsgroups lines in hostinfo.dat (v3)

you don't need sr, but it wouldn't hurt to have jcranmer look at this...

Comment 24

[Joshua Cranmer [:jcranmer]]

Comment on attachment 512074 [details] [diff] [review]
Removed ,,x,y,z from the newsgroups lines in hostinfo.dat (v3)

I see no problems with this.

Comment 25

[Karsten Düsterloh]

Pushed as <http://hg.mozilla.org/comm-central/rev/060665d0fca7>.