66873 - SYST error handling

Status: VERIFIED FIXED

(Core Graveyard :: Networking: FTP, defect)

Description

[Doug Turner (:dougt)]

FROM EMAIL:

Since the release of Netscape 6, we've been receiving complaints from our 
customers that Netscape users cannot access WFTPD servers.  We can 
reproduce, but not explain, the problem.  It seems that Netscape sends a 
PASV command, to which WFTPD responds correctly, and listens on the correct 
port.  But then Netscape doesn't ever seem to open the connection.  I've 
even run a network monitor on the transaction, and it's clear that Netscape 
isn't sending even the first SYN.

Comment 1

[HJ]

Do you have an example server adress to test? I like to take a look with a
protocol analyser.

Comment 2

[Doug Turner (:dougt)]

You can download their server here:

http://www.wftpd.com

Comment 3

[tenthumbs]

Does this happen with just Windows clients or does it occur with Linux clients?
There are some broken firewalls out there that screw up replies to PASV requests
under certain circumstances. The packets have bad checksums so, on Linux at least, they
never get to the client.

Comment 4

[Alun Jones]

I've personally only tested it between two Windows machines.  Note - there is 
no firewall present, no filtering, no NAT - just a hub, and two 10BaseT cables, 
between the two computers.  The network monitor showed that the PASV command 
was sent, and received, appropriately, but that Netscape didn't try the 
connection.  WFTPD is small - download it and try it.

Comment 5

[tenthumbs]

Do you know if the packet actually made it to the client? A network sniffer may
show a packet arriving but the system may drop it on the floor.

Comment 6

[Doug Turner (:dougt)]

lets stop guessing as to what it can be and lets get a trace of exactly what 
is happening.  I hope to do this in the next week or so.  If you want to get a 
jump on this, enable nspr logging in a debug build (not sure if it works in a 
opt build anymore):

set NSPR_LOG_MODULES=nsFTPProtocol:5
set NSPR_LOG_FILE=nspr.log

and attach this log.  It should be fairly easy to see what is happening.

Comment 7

[Alun Jones]

Okay - did that.

Here's what would appear to be the salient part of the log, suggesting that the 
problem is in "trying to send" the NLST command (which _does_ make it to the 
server and get responded to):

1292[ec98f8]: 2986de0 Writing "CWD /
"
1292[ec98f8]: SUCCEEDED
1292[ec98f8]: 2986de0 Process() - READ_BUF - read "250 "/" is current directory
" (30 bytes)
1292[ec98f8]: 2986de0 Process() - S_LIST -
1292[ec98f8]: 2986de0 Writing "NLST
"
1292[ec98f8]: FAILED
1292[ec98f8]: 2986de0 Process() - ERROR
1292[ec98f8]: ~nsFtpConnectionThread() for ftp://localhost/

Comment 8

[HJ]

I did test this on a WinNT4 system with build 2001021206 and did
ftp://[ipnumber], so both WFTPD and Mozilla are installed on the same system.
Direct after typing ftp://[ipnumber] the CPU % hogs up to 100% but WSFTP Pro,
Netscape 4.76, Netscape 6.0 and MSIE worked fine here. 

I did this basic test first to make sure that no other network related problems
are causing this strange behavior. 

Doug, did you change anything ftp related after build 2001021206?

Comment 9

[Doug Turner (:dougt)]

the 100% peg is another bug which will be fixed when I land a branch later this 
week.

We are probably messing up the parse of the SYST response.  We basically have to 
get the server's response when we issue the SYST command.  Right now the parsing 
is quite weak.  

Comment 10

[Doug Turner (:dougt)]

From rfc959:

SYSTEM (SYST)
This command is used to find out the type of operating system at the server.  
The reply shall have as its first word one of the system names listed in the 
current version of the Assigned Numbers document [4].

In RFC 1700, "WIN32" is the proper string to be checking for if we are 
interested in a windows OS based ftp server.  

In the current impl, we were looking for "windows" case insensitive.  

I will attach a patch which fixes this.

Comment 11

[Doug Turner (:dougt)]

Attachment: Fixes SYST parsing

Comment 12

[Doug Turner (:dougt)]

Attachment: Fixes windows ls parsing and protects against buffer overruns in EPLF

Comment 13

[Doug Turner (:dougt)]

jud can I get an r=
mscott can I get an sr=

Comment 14

[Judson Valeski]

regarding the SYST parsing. you're negating the possibility of "windows" being
in the string. shouldn't that be a if "windows" || "WIN32" ?

Comment 15

[Doug Turner (:dougt)]

Right.  That was the intent.  According to the rfc's, checking for WIN32 is all
we should have to do.  Do you know if there are dumb windows ftp services that
doesn't respond correctly to the SYST command?

Comment 16

[Judson Valeski]

you know better than that :-).
a) never use the acronym "RFC" in the context of FTP ;-)
b) yes, there are servers that spit back "windows". Let's just do a case
in-sensative compare for "win" in the string. If that doesn't work, then lest do
the if "windows" || "WIN32"

Comment 17

[Alun Jones]

Regardless - this bug was presumably caused by a lack of a "default" item for 
when the SYST response gives something you aren't expecting.  This time it's 
WFTPD - what happens next time you run up against a server whose SYST response 
isn't one that you've coded for?  A good solution would at least do _something_ 
in such a case, and assuming Unix-style is perhaps the most likely thing to 
work.

Comment 18

[Doug Turner (:dougt)]

if don't know how to handle it, maybe the correct thing to do is to alert the 
user that the service is not supported.

Comment 19

[Doug Turner (:dougt)]

Attachment: Patch two for syst parsing.

Comment 20

[Doug Turner (:dougt)]

got a r=gagan.

mscott, can you review?

Comment 21

[Scott MacGregor]

sr=mscott

Comment 22

[Doug Turner (:dougt)]

thanks.  Fixed checked in.

Comment 23

[benc]

So I should be checking against UNIX and Windows, and failing gracefully against
VMS?

Anyone have sample URL's to these servers?

Comment 24

[benc]

VERIFIED:
functional test shows this to be consistent across Mac OS X+9, Win32, and Linux.

Reopen if I missed anything.