555475 - For Newsgroup Accounts, Sort File hostinfo.dat

Status: RESOLVED WORKSFORME

(MailNews Core :: Networking: NNTP, enhancement)

Description

[David E. Ross]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.8) Gecko/20100205 SeaMonkey/2.0.3
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3

File hostinfo.dat contains the list of newsgroups on the news server for the account.  The general news server that I use (Giganews) supports well over 100,000 newsgroups.  When I want to subscribe to a newsgroup, the Subscribe window takes about 15 seconds to open.  This is because the list of newsgroups is not sorted in the stored hostinfo.dat but must be sorted to provide the threaded display in the window.  

Whenever hostinfo.dat is refreshed from any news server, the sorted file (sorted for display) should be what is written into the account.  


Reproducible: Always

Comment 1

[David E. Ross]

Some further information:  

For my account at Giganews, the hostinfo.dat file is 3.26 MB.  If I open it in a text editor (Wordpad), I see that the entries are not at all sorted.  

I can copy the file -- either onto the same hard drive or onto another internal hard drive -- almost instantaneously.  Thus, I conclude that the delay in opening the Subscribe window is primarily caused by the need to sort the contents of the file for the threaded display, not for the actual reading of the file.  This sorting should occur only once each time the file changes, not for each use.  

My account at Giganews is NOT related to my account at news.mozilla.org.  By coincidence, my ISP outsources its newsgroup service to Giganews.

Comment 2

[Magnus Melin [:mkmelin]]

Hmm, is this WFM as the code seems to imply? Or a bug in the code?
http://hg.mozilla.org/comm-central/annotate/6c605636afe1/mailnews/base/src/nsSubscribableServer.cpp#l505

Comment 3

[David E. Ross]

I no longer use Giganews as a general NNTP service because my ISP discontinued outsourcing that service and no longer provides any NNTP service (internal or external).  The ISP -- primarily serving businesses and not consumers -- claimed I was their only subscriber using NNTP service and they would not pay Giginews's outsourcing fee for only one subscriber.  

I now use reader.albasani.net as my primary NNTP service with news.eternal-september.org as a backup.  Neither of these host as many as 30,000 newsgroups.  (But they host the ones I want.)  With significantly fewer newsgroups (less than 1/3) in hostinfo.dat and having expanded the memory on my PC (which allows for faster sorting), I no longer see the delay in opening the Subscribe window.  

HOWEVER, the contents of hostinfo.dat are still not stored in my Thunderbird profile as sorted.  This means it must be sorted every time the Subscribe window is opened.  Once it is sorted, it should be saved as sorted.  

This is still a bug.

Comment 4

[Suyash Agarwal (:sshagarwal)]

I went through the sorting part that you mentioned in comment 2, and it seemed fine to me, but I couldn't find the place where this content is being written to the hostinfo.dat, and the array mGroupsOnServer.
http://mxr.mozilla.org/comm-central/source/mailnews/news/src/nsNntpIncomingServer.cpp#1667

Comment 5

[Magnus Melin [:mkmelin]]

I guess the bug is somewhere else then. I suppose hostinfo.dat is written to here: http://mxr.mozilla.org/comm-central/source/mailnews/news/src/nsNntpIncomingServer.cpp#792

Note http://mxr.mozilla.org/comm-central/source/mailnews/news/src/nsNntpIncomingServer.cpp#835

Comment 6

[Makhlug Jafarov]

Hello,

Need info to work on this issue. I am a beginner on open source projects. If you think is not for someone new to open source, please, advise what to start with! Appreciate a lot!

Kind regards,
Makhlug Jafarov

Comment 7

[David E. Ross]

As of Thunderbird 52.9.1, it appears that the hostinfo.dat file is indeed sorted. Thus, this bug report should be CLOSED.

Yes, I know I am using a very old version of Thunderbird. However, I have 16 .xpi extensions that I use which are no longer working in later versions.