301222 - News Reader Failing to Download Multiple News Message Files and Attachments.

Status: RESOLVED DUPLICATE of bug 66150

(MailNews Core :: Networking: NNTP, defect)

Description

[Nuncus]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.8) Gecko/20050511 Firefox/1.0.4
Build Identifier: Thunderbird (news reader) version 1.0.2 (20050317)

The newsreader seems to lose track of itself and will fail when asked to
download multiple message files and file attachments from the Usenet
news group from which the downloads are attempting to be made.

There seems to be a correlation in part to the "Too Many Open Connections"
bug that has been elsewhere reported for the news reader although the
characteristics of the suspected bug reported herein do not necessarily
track or parallel the "Too Many Conections" bug in all respects.

SEE bug report 288977 and comments and illustrative JPEG 
screenshot attachments thereto.

Also see the discussion below.


Reproducible: Always

Steps to Reproduce:
1.  Launch the T-B news reader and subscribe to  a group on Usenet.  Pick a
group such as a nospam group with many files/attachments displayed in order.

2.  Attempt to download message files sequentially by double-clicking message
headers to initiate download of the message files from the Usenet news group.

3.  OR, mark a number of files in sequential, or non-sequential, order and
attempt to download all the message files for the message headers using the
attachment extractor extension.

SEE full work-up below.

Actual Results:  
1.  Message files fail to download; the news server "too many open connections"
error message will usually appear, and the download will be terminated by
the news server.

2.  There is a logical pattern to the failure/success occurrance:

    On this user's ISP news server, the ratio/pattern was as follows:

          fail

          success
          success
          success
          success

          fail

          success
          success
          success
          success

          #n	fail

          #n+1	succeed
          #n+2	succeed
          #n+3	succeed
          #n+4	succeed

The news reader is apparantly attempting to open up a new news server
connection instead of queing the downloads through the allowable number
of open connections permitted by the news server; and

The news reader is apparently failing to keep abreast of its progress
in effecting the downloads and "where it is" in the download process.

SEE below and see screen shot JPEG file attachment.


Expected Results:  
The news reader should have downloaded each and every message file associated
with the message headers with respect to which the User attempted to initiate
message file downloads.

The actual testing information is too long to enclose here; a comment attachment
will be filed, together with an enclosed JPEG screenshot that illustrates
graphically the bug reported herein.

Comment 1

[Nuncus]

Attachment: For Bug RPT. #301222 - (TB NWS RDR) Illustrates Intermittent Failure to download Usenet Message Files.

This is a screen shot that illustrates the intermittent failure
of the news reader to successfully download message files from
the Usenet.

To accompany Bug RPT. #301222

See apparently related bug report (comments and screen shots): 288977

Comment 2

[Nuncus]

COMMENTS, HISTORY, TEST RESULTS, AND DETAILED DESCRIPTION
OFTHE APPARANT BUG AND MALFUNCTION:

PRODUCT IS THUNDERBIRD/CORE -- USENET NEWS READER.

-------------------------------------
REASON FOR FILING A "NEW" BUG REPORT:
-------------------------------------

	The reporter was unable to locate a bug report
covering this issue desite searching through the 
list of previously reported bugs for the news reader
or by using the "search" function.

Bug report 288977, hoever, definitely appears to be a
related bug or malfunction to the bug or malfunction 
discussed in this bug report.


------------
DESCRIPTION:
------------

TITLE DESCRIPTION:	News Reader Failing to Download Multiple
			News Message Files and Attachments.

BRIEF DESCRIPTION OF THE BUG:  The newsreader seems to
lose track of itself and will fail when asked to download
multiple message files and file attachments from the
Usenet news group from which the downloads are attempting
to be made.

	There seems to be a correlation in part to the
"Too Many Open Connections" bug that has been elsewhere
reported although the characteristics of the suspected
bug reported herein do not necessarily track or parallel
the "Too Many Conections" bug in all respects.

SEE bug report 288977 and comments and illustrative JPEG 
screenshot attachments thereto.

	News reader intermittently fails to download files
that have been marked for download when multiple files are
involved.

	Suspect:
	--------

	1.	News reader is not queing downloads through
		the allowable open news server connections
		in the order that it should; and

	2.	News reader is failing to maintain download
		progress and loses track of where it is in
		the download process.

ATTACHMENT (JPEG):
------------------

	I will try to accompany this bug report with a
screenshot JPEG that fairly illustrates the problem raised
in this bug report; and I sincerely hope enough attention
will be given to the core technology of the news reader
such that we who are Open Source enthusiasts will have a robust
and dependable news reader to accompany the emailer and the
browser.

	The name of the file attachment is as follows:

		"Dwnld_in_seq_001.JPG"

The JPEG attempts to illustrate thedownload patterns and
inconsistencies in downloading that are discussed herein.


USER TESTS PERFORMED:
---------------------

Here are the test patterns:

I.	PATTERN "1":  Double-click individual message
	headers, one-by-one, in sequence, five at a
	time -- first one fails entirely to download, next
	four download; repeat sequence:  first file fails to
	download, next four succeed, etc., etc., ad infinitum.

	--------------------------------------------
	Pattern "1" does NOT involve the use of the
	attachment extractor extension.
	--------------------------------------------
-----------------------
O/S = XP, Svc Pk 2.
Version 5.1 (Build 2600.xpsp_sp2_gdr.050301-1519:Service Pack 2)

T-Bird = version 1.0.2 (20050317)

Skin = Default.

Extensions in use = None.

ISP = news.east.earthlink.net; ELNK server appears to
be Tornado v1.0.5.360.
----------------------- 

The header list from which downloads of messages will be
made is as follows (all message files have file attachments):

	001.eml
	oo2.eml
	003.eml
	004.eml
	005.eml
	006.eml
	007.eml
	008.eml
	009.eml
	010.eml
	011.eml
	012.eml
	013.eml
	014.eml
	015.eml
	016.eml

The news reader will not allow me to download more
than four messages at a time, so I will do (have done)
the downloading in quartets as follows:

	001.eml
	oo2.eml
	003.eml
	004.eml

	005.eml
	006.eml
	007.eml
	008.eml

	009.eml
	010.eml
	011.eml
	012.eml

	013.eml
	014.eml
	015.eml
	016.eml

RESULTS:

	001.eml - successful download and file attachment save
	oo2.eml - successful download and file attachment save
	003.eml - successful download and file attachment save
	004.eml - successful download and file attachment save

	005.eml - blank download window and no download
	006.eml - successful download and file attachment save
	007.eml - successful download and file attachment save
	008.eml - successful download and file attachment save

	009.eml - blank download window and no download
	010.eml - successful download and file attachment save
	011.eml - successful download and file attachment save
	012.eml - successful download and file attachment save

	013.eml - blank download window and no download
	014.eml - successful download and file attachment save
	015.eml - successful download and file attachment save
	016.eml - successful download and file attachment save

NOTE:	Files that failed to download will in fact download
	if, in fact, the news reader is exited and restarted;
	the exact same 1-4, 1-4, 1-4 pattern of failure and
	download occurs in the second download attempt pursuant
	to which the failed message file downloads from the
	first download attempt are attempted to be downloaded
	in the second attempt.

	Files that fail to download cannot be downloaded until
	the emailer/news reader app is exited entirely and
	restarted.  It is NOT necessary to reboot the computer,
	only restart the app.

REPRODUCABLE?:	YES.  Every time sequence is run; no deviation.
		Have run test sequence at least 20 times.

II.	PATTERN II.	Mark "some" number of message headers
	in news reader (I used from 50 to over a hundred in lthe
	testing sequences).

-----------------------
O/S = XP, Svc Pk 2.
Version 5.1 (Build 2600.xpsp_sp2_gdr.050301-1519:Service Pack 2)

T-Bird = version 1.0.2 (20050317)

Skin = Default.

Extensions in use = Attachment Extractor 0.4.1.

ISP = news.east.earthlink.net; ELNK server appears to
be Tornado v1.0.5.360.
----------------------- 

	USE the attachment extractor to attempt to download the
	message files and file attachments for the individual
	message headers.

	RESULT:  You get "some" number of message files that
	download from the marked list -- it varies from attempt
	to attempt:  usually from 4 to 15 or so.

	"Sometimes" you will receive the "too many connections"
	from the ISP's news server, which will then terminate
	the download at whatever point the news server has
	detected that Thunderbird has violated the ISP's
	open connections limitation.

	Curiously, although the ISP with respect to which the
	tests herein were carried out has a four (4) open
	connection limitation, the news server sometimes allows
	the attachment extractor download to proceed past the
	normal limitation just a bit before triggering the
	termination of the download.

	Subsequent to the termination of the download with
	attachment extractor extension, and proceeding with
	double-clicking each message header in the news reader
	in sequence one-by-one, the news reader lapses into the
	pattern of behavior described above for the bug when
	sim ply downloading the message files one by one with
	a double-click of the mouse, i.e.:

		file attachment extractor downloads a few files;

		the download terminates unexpectedly;

		news reader lapses into the following
		failure and success pattern:

			#1	fail

			#2	succeed
			#3	succeed
			#4	succeed
			#5	succeed

			#6	fail

			#7	succeed
			#8	succeed
			#9	succeed
			#10	succeed

			...

			#n	fail

			#n+1	succeed
			#n+2	succeed
			#n+3	succeed
			#n+4	succeed

		SEE above.

NOTE:	Files that failed to download will in fact download
	if, in fact, the news reader is exited and restarted;
	the exact same 1-4, 1-4, 1-4 pattern of failure and
	download occurs in the second download attempt pursuant
	to which the failed message file downloads from the
	first download attempt are attempted to be downloaded
	in the second attempt.

	Files that fail to download cannot be downloaded until
	the emailer/news reader app is exited entirely and
	restarted.  It is NOT necessary to reboot the computer,
	only restart the app.

REPRODUCABLE?:	YES; EVERY TIME SEQUENCE IS RUN.

LAYMAN'S OBSERVATIONS:

1.	The news reader appears to be attempting to open
	up an entirely new connection with the ISP news
	server for the downloading procedure as to EACH
	and EVERY file instead of testing the news server
	first for the permissible number of allowed open
	connections and then queing the downloads of the
	message files and file attachments through each
	of the permissible open connections open with the
	news server.

	{Apparantly, Forte Agent somehow runs ques through
	all permissible open connections and keeps track of
	the ques, having several ques behind each open
	connection -- which ques may or may not have been
	marked by the User in sequence.]

	[Microsoft apparently runs one que through one
	open connection, and that is it, folks.  But, then,
	Microsoft has been notorious for having the worst of
	the news readers from among all the commercial news
	readers.]

2.	The news reader appaqrently "loses track of itself"
	when downloaing message files.  This appears to be
	a seperate issue from, though perhaps tangentially
	related issue to, the "too many conncections" issue.

	This is just a guess; but it appears that the news
	reader is either not keeping track of where it is
	in the download task or, if it is keeping track, it
	is failing to institute reads of the progress on a
	fequent enough basis.

	SEE Bug report 288977 for a filing and subsequent
	comments and work-up of the "too many connections"
	problem.

3.	This set of possibly interrelated bugs causes users
	to abandon use of the Thunderbird news reader altogether
	as a viable Usenet news reader.

	Abandonment of the T-B news reader is generally followed
	by abandonment of the emailer -- too "inconvenient" for
	most non-computer types to keep track of two apps and
	retain how to use both when the Outlook Express can do
	a "reasonable" and reliable job of handling both email
	and Usenet news reading.

	Abandonment of the T-B news reader and the emailer is
	generally followed by abandonment of the browser.  I 
	suspect this is because of several, although not necessarily
	rational, practical reasons:

	a.	Users generally think of the browser, emailer,
		and news reader that they use on a day to day
		basis as one unit of interrelated apps.  For
		example, even though the Open source apps are
		not "integrated" with the Windows O/S (which is
		a MAJOR reason why users want initially to switch
		to the Open Source apps -- security reasons), the
		Open Source apps are nevertheless integrated somewhat
		as between themselves.  For example, there is a menu
		command in Firefox that allows the User to choose
		to "send an email" while using the browser.

	b.	Psychologically, if one app does not work in
		a robust way, Users tend to assume none of the
		apps are reliable -- even if the assumption is
		entirely fallacious.  Call it guilt by association.

		Thus, the weakest link in the triad of apps
		can serve to bring the entire triad down.  The
		reverse is rarely true.

		In other words, if the news reader does not
		handle downloads of message files and file
		attachments correctily, such aberrant app behavior
		tends to "poison" the perception of the other
		apps.

	c.	Thus, you get a reverse run of lack of confidence
		building up in the User who is going to assume,
		particularly if the User is using the apps at
		work for a paycheck, that it is simply "safer"
		to NOT use the Open Source apps at all and keep
		one's job.  Concerns for meeting the mortgage
		payment and feeding the kids will win out
		over programming every time; and you can put that
		one in the bank so to speak.

	NOTE:	This last point, "3.", is based ENTIRELY on
		"anecdotal" evidence and is NOT based upon any
		formal survey.  Normal human behavior, however, is
		generally rather consistent in these matters.

4.	You CANNOT assume that every Usenet user is going to
	purchase Forte Agent Pro.  Users who use Firefox will
	want to use the Thunderbird also; and, since most Usenet
	users are NOT "JPEG junkies," you should assume that a
	user of Firefox and Thunderbird emailer will also want to
	use the Thunderbird news reader for such news reading as
	the average user will do -- which is a lot more than any
	of the so-called "polls" show.

	Finally, as indicated above, I will try to accompany this
bug report with a screenshot JPEG that fairly illustrates the
problem raised in this bug report; and I sincerely hope enough
attention will be given to the core technology of the news reader
such that we who are Open Source enthusiasts will have a robust
and dependable news reader to accompany the emailer and the
browser.

	The name of the file attachment is as follows:

		"Dwnld_in_seq_001.JPG"

The JPEG attempts to illustrate thedownload patterns and
inconsistencies in downloading that are discussed herein.

	Rest assured, we are all pulling for you and your
endeavor; and those of us who are not internet app programmers
will hopefully be able to contribute to your efforts by
reporting any apparent bugs and problems to you as your proceed.

Best regards,

L Scott
("Nuncus" in the Support groups and bugzilla)

Comment 3

[Wayne Mery (:wsmwk)]

Nuncus, does this happen for you with ftp://ftp.mozilla.org/pub/thunderbird/nightly/latest-trunk/  or current version of http://getthunderbird.com ?

remove topcrash+ - doesn't meet the definition

[note: nuncus' em address doesn't error out, but hasn't filed new bug in a few years]

Comment 4

[Joshua Cranmer [:jcranmer]]

Talk about detailed.

The problem here, AFAICT, is that we open new connections when downloading each message. I can confirm this on TB 2.0.0.15pre, but it should be present Trunk TB.

Duping to 66150.