61301 - Strange FTP behavior/test script included

Status: VERIFIED INVALID

(Core Graveyard :: Networking: FTP, defect, P3)

Description

[Keith Briscoe]

I've noticed Mozilla nightly builds from 11/27/2000 (Linux/Win32) occasionally
stalling while accessing some FTP sites (behavior: throbber goes and goes but
the contents never appear), and suspect that this problem also exists in
Netscape 6 for Linux and Windows, because of the results of my test script.

I have written a Perl script which emulates an FTP server (runs on Linux) and
simulates the very beginning portion of an FTP session.  The behavior of Mozilla
and Netscape 6 is not consistent with any other FTP client I have tested,
including: Netscape 4.xx, IE 5.x, Linux and Win32 ftp command-lines and the gFTP
graphical Linux FTP client.

This does not appear (to me) to be related to the problem with non-passive-mode
servers because it never even gets to the point in the connection where passive
mode would be relevant.  I'm not sure what qualifies as "correct" behavior from
an FTP client, but Mozilla seems to be the odd one out in the bunch.  I think
this is a bad thing, and I hope I can help.

I'll attach the script as soon as I figure out how.

Comment 1

[Keith Briscoe]

Attachment: FTP Server Emulator script (kinda)

Comment 2

[Keith Briscoe]

Correction to script comments: Mozilla establishes a connection, but does not
send the USER command as expected.  Also, the default FTP port in the script
should be 21 for most purposes, but I had given it a higher value so that I
could run it with user permissions.

Comment 3

[Keith Briscoe]

Attachment: A better script

Comment 4

[Keyser Sose]

I have seen this as well. Good testcase included. Marking NEW.

Comment 5

[neeti]

--> ftp

Comment 6

[Keith Briscoe]

I'm suggesting a target milestone of 1.0 for this one, but I'll leave it to the
Netscape folks to make it official.  Feel free to disagree.  Mozilla does
occasionally flake out on an FTP connection, and I haven't yet found a pattern
in servertypes/versions that trigger the bug.  The only consistent failure is in
the testcase; wish I had better info.  I think it'd be a good thing to fix
before 1.0.

Bug confirmed to still exist in 2001-05-09 builds.

Comment 7

[rohan.hart]

Under IRIX this happens for every ftp site. Build is 2001052521

I also get the occasional...
Gdk-CRITICAL **: file gdkwindow.c: line 716: assertion `window != NULL' failed.
popping up just before the...
Error loading URL ftp://ftp.mozilla.org/ : 2152398850

Testing Keith's script I get a connection but no response whether or not I
specify a user as part of the ftp URL. However no password box appears either
unlike actual ftp sites.

Comment 8

[Keith Briscoe]

Okay, I think I may be on to something with this one.  I used a packet sniffer
('cause I can't read the networking code) and I think I know what's happening:

1) Mozilla establishes the connection and immediately sends the USER/PASS
sequence before it is requested.

2) The FTP server flushes its input buffer (DOH!), prompts for a login, and
waits for a response.

3) Mozilla happily waits for the FTP server to respond to its login request
while the FTP server waits for Mozilla to send a login request.  Whammo!

Anyway, I think this explains why MOST FTP servers seem to work okay and just
the occasional one freaks out.  You could probably call this a server error, but
I'm not thinking there's enough of a solid "standard FTP behavior" ruleset to be
able to say servers shouldn't flush their buffers.  Since all other browsers
wait for a prompt before sending USER/PASS, I think Mozilla should do the same.
 I'm off to modify my script and test this puppy without the "autoflush" line.

Comment 9

[Keith Briscoe]

Okay, I'm on crack.  Removing the autoflush line was not the magic solution. 
However, the packet sniffer did yield some interesting stuff:

Mozilla DOES seem to send the USER/PASS information at some point in the
connection, to the correct port.  The FTP server simply fails to read it.  This
could be because of buffer flushing, the server waiting for a linefeed
character, or some other such issue, but whatever problem these servers have,
the test script also has it.  My own abilities to test this have been defeated
by sleepiness.

Comment 10

[Keith Briscoe]

Wow, I think I've solved this one.  Really, this time for sure.  If you replace
all the newlines (\n) in the test script with CR-LFs (\r\n), Mozilla starts
working again.  So it seems that whenever Mozilla gets a line terminated with a
plain \n, it could potentially stall.  I don't know how this is handled
code-side, so it could be scattered in many different places.

In particular, I think the place this is probably getting triggered is in README
files that are spammed out as you CD to directories.  If the files are
UNIX-style and contain plain linefeeds, maybe this is it.  There's probably
hundreds of places where this could happen though in addition to here.  The FTP
protocol is such a mess.  I have no clue if plain linefeeds are even within the
specs, but it's a decent idea to support them anyway from a compatibility point
of view.

Comment 11

[rohan.hart]

Confirmed - I've extended the script to cover more of the FTP conversation and
replacing any of the \r\n with plain \n leaves Mozilla eternally waiting.
IRIX build 2001081322

Comment 12

[rohan.hart]

Attachment: Extended script

Comment 13

[rohan.hart]

Sorry, you'll have to change the #! in that script - I forgot to change it back
from the nonstandard perl5 installation location on this machine.

Comment 14

[Keith Briscoe]

If it matters--it looks like \r\n IS defined as the proper end-of-line sequence
in the relevant RFCs.  Plain \n is out-of-spec, which is probably why Mozilla
only fails on the odd server these days.  Still, I'd recommend that changes be
implemented which allow servers sending \n to work, as there's no reason I can
see for NOT doing it, and every other FTP client (including Netscape 4.x) allows
this already.

Also, plain \n signals may creep into FTP sessions via README files as I
mentioned earlier, even if the FTP server uses \r\n to properly terminate its
responses.

Comment 15

[Doug Turner (:dougt)]

reassigning to [e-mail].

Comment 16

[Bradley Baetz (:bbaetz)]

I'll look into this one.

Do you have a server which breaks, though, rather than a test script?

Comment 17

[Keith Briscoe]

Now that I know what the problem is, here's my revised opinion.  This is a
*potential* compatibility problem, but until an FTP site is found which triggers
this bug, this bug can be de-prioritized.

The test script AFAIK is the ONLY thing that fails reliably due to this bug. 
The test script was created for troubleshooting FTP problems that may or may not
have been related to this bug.  Those bugs (such as MS proxy problems) may be
fixed independently, and are much higher priority because they have known
servers that fail.

Still, I have a little worry about this bug because I'm not sure how README
files with plain newlines may be treated by various FTP servers.  So identical
server software could have no problems if its README files were created in
WordPad, but may choke if its README files were written in EMACS.

Comment 18

[Bradley Baetz (:bbaetz)]

Well, unless you know of a real server, I'm going to mark this as invalid.

Comment 19

[benc]

VERIFIED: although the test script suff is interesting, I'll probably go back
and look at using it if possible.

Comment 20

[benc]

-testcase, but moved to 'making test' list.