214525 - Syncing/downloading newsgroups for offline reading is extremely slow.

Status: RESOLVED FIXED

(SeaMonkey :: MailNews: Backend, defect)

Description

[Mark Stier]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4) Gecko/20030624
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4) Gecko/20030624

It seems like the Mozilla newsreader gets the articles one by one. That's not
good for a 60.000-articles newsgroup. Others client take a few minutes to
download the whole group, and not hours like Mozilla. Would be great if that
could be fixed since that forces me to use another newsgroup client.

Reproducible: Always

Steps to Reproduce:
1.
2.
3.



Expected Results:  
Use the protocol efficiently.

Comment 1

[David :Bienvenu]

could you try a 1.5 build and let me know if it's faster? Do you hear the disk
chugging when you download newsgroups? thx.

Comment 2

[David :Bienvenu]

I think 1.5 is faster, but it's still slow. There's a similar bug for IMAP - bug
214245 - my best guess is that it's slow because we're opening and closing the
offline store for every message, and we're commiting the db for every message.
There's a patch in the other bug that I hope speeds up IMAP, by reusing the
store. I'll try to come up with a similar patch for news.

Comment 3

[David :Bienvenu]

Attachment: don't keep closing and re-opening output stream

this patch makes it so we don't keep closing and re-opening the output stream,
which should speed things up a little. It also has a bit of cleanup.

BTW, there's no NNTP command that I know of that will download more than one
message at a time (unlike IMAP).

Comment 4

[David :Bienvenu]

last fix was checked in, r/sr=mscott. You'll have to download tomorrow's 1.5b
daily build to see if it helps.

Comment 5

[David :Bienvenu]

we're still committing the db for every message, and updating the status line
for every message. Both of those will slow us down. I'll add a patch to fix
those problems.

Comment 6

[David :Bienvenu]

Attachment: don't commit every msg, and don't update progress/status every msg

This makes it so we don't commit the db for every msg downloaded, and we only
update status msg every 3/4 of a second, and progress percent when it has
changed.

Comment 7

[David :Bienvenu]

Comment on attachment 129254 [details] [diff] [review]
don't keep closing and re-opening output stream

this was checked in.

Comment 8

[(not reading, please use [e-mail] instead)]

Comment on attachment 129443 [details] [diff] [review]
don't commit every msg, and don't update progress/status every msg

r/sr/a=sspitzer for 1.5 beta

Comment 9

[David :Bienvenu]

fix checked in; please let me know if it's faster. I don't have a news feed fast
enough to tell...