33163 - Cannot have four FTP transactions going at the same time (multiple simultaneous downloads)

Status: RESOLVED FIXED

(Core Graveyard :: Networking: FTP, defect, P2)

Description

[:deanis74]

Step to Reproduce:
1. Go the the about FTP url
2. Click on mozilla-win32.zip
3. Select Save File
4. Enter a location, click OK
3. File Transfer ("Saving file" dialog) begins
4. Click on mozilla-win32-installer.exe

Expected Results:
Prompted for a file location right away and download begins.

Actual Results:
Not prompted for a file location until until mozilla-win32.zip finishes 
downloading.  Until then, the status bar text remains as "Beginning FTP 
transaction" and the throbber and progress meter spin away.

Additional Notes:
This isn't related to just file transfers.  Once the file transfer is initiated, 
you cannot change directories.

Platform: WinNT 4 sp5
Build ID: 2000032308

Comment 1

[Gagan]

->jud

Comment 2

[Judson Valeski]

-> law

Comment 3

[Gervase Markham [:gerv]]

Confirming on 20000512. Do we need this for nsbeta2? FTP is fairly broken... You 
can't download more than one file at once...

Gerv

Comment 4

[leger]

[NEED INFO] Is this a beta2 criteria?

Comment 5

[Peter Trudelle]

[nsbeta2+]

Comment 6

[Tom Everingham]

*** Bug 32554 has been marked as a duplicate of this bug. ***

Comment 7

[don]

Bill, can you add some additional information to this bug about why it will suck
if we don't fix this?

Comment 8

[Bill Law]

Because it sucks to request some action and get no response till some random 
point in time later, and, to get bogus feedback on what's happening.  People 
will find other software that doesn't present such obstacles and use that 
instead.  That sucks.

Comment 9

[don]

Move to M18.

Comment 10

[Michael La Guardia]

Removing need info.  We need to fix this.

Comment 11

[Michael La Guardia]

Removing need info.  We need to fix this.

Comment 12

[leger]

Putting on nsbeta2- per Nav Beta2 review today.

Comment 13

[leger]

Marking fixed per gagan.

Comment 14

[Tom Everingham]

verified:
WinNT 2000071308
Mac OS8.6 2000071308
Linux 2000071308

Comment 15

[sairuh (rarely reading bugmail)]

hm, i cannot download more than 1 file at the same time (following dean's 
original steps). this is what i see when using mozilla on win98, 2000.08.31.12 
bits:

after the 1st file starts downloading (mozilla-win32.zip), i doubleclick on 
mozilla-win32-installer.exe. no download dialog appears for this 2nd file 
*until* the 1st has completed downloading.

reopening. unless this is [somehow] expected behavior now, or another bug 
entirely. if so, feel free to close this and i'll open another bug. thx!

Comment 16

[:deanis74]

Yep, it's back.  Using a Sept 1 build.

Comment 17

[sairuh (rarely reading bugmail)]

thx for vrfying, dean. adding regression and nominating for beta3 fix. (also
added more bits to the summary so that my queries hit it. :)

Comment 18

[johng]

nav triage team:
reassigning to Gagan since he fixed this once and it has now regressed.

Comment 19

[Tom Everingham]

This worksforme with todays 2000092005 build on WinNT and Win98.  Tried 3 
simultaneous downloads with no problems.   Marking wfm.  If anyone is still 
having difficulties please reopen.

Comment 20

[Tom Everingham]

verified

Comment 21

[:deanis74]

You really shouldn't be resolving and verifying the same bug.

Anyway, still happens for me using 2000092005 (the exact same build) on WinNT.  
Reopening.

Comment 22

[Tom Everingham]

hmm, this works for me every time I try it (on winNT, win98, and Linux) with the 
exception of an unknown error message that is being addressed in a different 
bug.  Are you possibly seeing a delay before the second file starts downloading?  
Also, is it possible for you download 2 or more .zip files simultaneously?

Comment 23

[:deanis74]

Nope.  The download process for the second file never starts until either the 
first download completes or I cancel it.  Are you grabbing your files out of 
different directories and/or servers, or all from the same directory?  I'm just 
randomly clicking on files in the latest directory.

(Updated URL)

Comment 24

[Tom Everingham]

Same as you, just picking files from the same directory.  I'm selecting large 
ones so I can get multiple transactions going.   OK, I just saw that after 3 
were going I couldn't open a forth (at least right away) without clicking the 
file a bunch of times  ... maybe that is the same thing your seeing.

cc.'ing law 

Comment 25

[:deanis74]

Well, if I keep on double-clicking then eventually I get the Unknown File Type
dialog.  But not until the fourth or fifth double-click.

Comment 26

[Gagan]

I don't think I ever fixed this bug. But I think I did help law fix this once. 
So ->law 

Comment 27

[Gagan]

->

Comment 28

[don]

Well, if you can get at least two or three transactions going then that's good
enough for now.  It sucks that you can't get more than three, but I wouldn't
hold beta for this.  Does anyone want to nominate this for rtm?

Gagan, please don't send this over to Bill without a better explanation of why
this isn't a networking problem.  Have you done any investigation?

Comment 29

[Gagan]

don: this still is law's area. There are bigger problems with multiple download 
which me and law have gone over in past and have worked it out to make it 
possible for multiple FTP downloads to occur simultaneously. I agree that its 
not a stop-ship for beta but this still is law. And please don't assign it back 
till you think law has a problem with fixing this again. 

Comment 30

[:deanis74]

Don, that's not the case.  As an average user (not one that double-clicks on a 
link ten or twenty times), I can only get one FTP transaction going at a time.  
Not three.

Comment 31

[Bill Law]

I don't have a problem with fixing it *again*,  Thing is, I don't recall ever 
fixing it the first time.

I see Jan marked it resolved/fixed on July 13, "per" you.  Tom verified it the 
same day.  So it must have appeared to work then.

I didn't check in anything that would have fixed it.  My recollection is that we 
decided this wasn't [nsbeta2+] because being able to download one file at a time 
was good enough (and this is reflected in Jan's comment on Jul 07.

So if it was truly fixed, it appears as if you'd know how.  If you recall, tell 
me and I'll pursue that lead again.  But as it stands, I have no idea where to 
look.

Here's what I observe using yesterday's build on Win98:

1. I can do 3 downloads from a ftp directory listing at the same time.  E.g., 
try ftp://hobbes.nmsu.edu/pub/os2/system/patches/fixpacks.  There's some big 
files there.
2. If I try a fourth, the progress dialog does not appear until one of the 3 
completes.
3. If I open another browser window, I can do a download from a different ftp 
location (same server, as I recall).  I don't know how many (I didn't have the 
patience to start 3 more and see what happens on the fourth).
4. If I type in a different URL in the first browser window (where the 3 
downloads are happening), it doesn't go there until one of the 3 completes (this 
was an ftp url, different server (ftp.mozilla.org)).  I *can* navigate to an 
http location (by typing in the location bar).

It *seems* as if there's some limit (3) on the number of simultaneous ftp 
channels that can be open for a given window (more likely, for a given load 
group or something).  Given that that the problem appears when simply navigating 
as well, it wouldn't seem that it could be a file downloading thing.  That, plus 
 I can't come up with *any* plausible explanation that would lead me to believe 
that the fix would end up in my code.

An interesting test might be to see if a frameset with 4 frames, each with 
src="ftp:...", works or not.  If it weren't 2:45 am, I'd try that myself.

BTW, I do see other flakiness with ftp.  For example, accessing my internal ftp 
server, I can't download anything (e.g., 
ftp://law.mcom.com/d/dropbox/appcore.zip gets me the Unknown File Type dialog.  
If I choose Save File... then I get the download progress dialog for a second or 
so, then a JavaScript alert from it reporting an NS_ERROR_FAILURE.  I see this 
on the console:

Error loading URL ftp://law/d/dropbox/appcore.zip: 804b0002
Error loading URL ftp://law/d/dropbox/d/dropbox/appcore.zip: 804b0002

It looks like there's something fishy going on talking to my FTP server.

Comment 32

[sairuh (rarely reading bugmail)]

adding to my radar.

Comment 33

[Tom Everingham]

nominating rtm, we should offer a reliable way to do simultaneous downloads (as 
in 4.7)

Comment 34

[don]

This won't stop ship.  Sorry.

Comment 35

[Bill Law]

Nominating.  I need to dig into this again to see if what I observed most
recently still holds (and figure out why).  I'm adjusting the summary to say
"cannot have *four* FTP transactions...".

Comment 36

[Dan M]

  Doesn't the limit of four simultaneous downloads come from the upper limit on 
the number of simultaneous worker threads allowed by the system 
(NS_FILE_TRANSPORT_WORKER_COUNT_MAX is set to 4; defined in 
nsFileTransportService.h)? I believe that all asynchronous I/O is pushed onto a 
list of pending requests, to be processed when a worker thread is available to 
take it on, and we have a hard limit of four such threads.
  We might could bump that number up on systems that can handle it. I think it's 
set to four as a lowest common denominator kind of thing.

Comment 37

[Bill Law]

But the limit isn't 4, system-wide.  It's 4 *ftp* downloads *per window* (or so
I observed a while back).  Actually, 3 (the *fourth* blocks).

Comment 38

[Bryant Castaneda]

NETSCAPE 6 does view ftp links.
Is this functionality going to be in the works in the future.
If there is a solution, please email me please.

Thank you in advance.

Bryant Castaneda
Arizona State University
Computer Science Engineering
[e-mail]

Comment 39

[Paul Chen]

nav triage team:

Not a beta stopper, marking nsbeta1-

Comment 40

[Doug Turner (:dougt)]

I have changed the workings of FTP.  There is now no 4 connection limit.

Comment 41

[Viswanath Ramachandran]

nav pretriage: moving to Future. 

Comment 42

[Asa Dotzler [:asa]]

This should not be marked Future.  It should be resolved as Fixed.  If the new
architecture (which does not have a 4 download limit) has other problems those
should be filed as new bugs.  If anyone objects feel free to reopen.  Markign Fixed

Comment 43

[benc]

qa to me... I'll review this later...

Comment 44

[benc]

-> ftp