131163 - Caudium FTP server, second try

Status: VERIFIED INVALID

(Core Graveyard :: Networking: FTP, defect)

Description

[Thomas Arendsen Hein]

With mozilla up to 0.9.9 I have problems accessing subdirs on servers which are
using the Caudium web server for ftp access, e.g. ftp://ftp.intevation.de/ and
ftp://ftp.caudium.net/

If you access these URLs, then click on different subdirs, some work, others not.
I don't have any problems with other clients in active and passive mode (except
Galeon, which is using mozilla's libraries, too).

Here is a snap of the server logfile:
aktaia.intevation.org - - [15/Mar/2002:14:52:23 +0100] "LOGIN Anonymous FTP" 200
0 "-" "ftp"
aktaia.intevation.org - - [15/Mar/2002:14:52:23 +0100] "LOGIN
Anonymous%20User:mozilla%40example.com FTP" 200 0 "-" "ftp"
aktaia.intevation.org - - [15/Mar/2002:14:52:24 +0100] "STAT / FTP" 405 0 "-" "ftp"
aktaia.intevation.org - - [15/Mar/2002:14:52:24 +0100] "CWD / FTP" 200 33 "-" "ftp"
aktaia.intevation.org - - [15/Mar/2002:14:52:24 +0100] "LIST / FTP" 200 606 "-"
"ftp"
aktaia.intevation.org - - [15/Mar/2002:14:52:33 +0100] "STAT /users/ FTP" 405 0
"-" "ftp"
aktaia.intevation.org - - [15/Mar/2002:14:52:34 +0100] "CWD /users/ FTP" 200 39
"-" "ftp"
aktaia.intevation.org - - [15/Mar/2002:14:52:34 +0100] "LIST /users/ FTP" 200
331 "-" "ftp"

The directory listing still shows the root directory. If I click on a different
subdirectory, mozilla shows "Index of ftp://ftp.intevation.de/users/", but
without contents.

This problem occurs on Debian Woody and RedHat 7.2, from different networks, one
having a Linux 2.2 firewall, the other having a 2.4 iptables firewall.
Caudium versions are 1.0.34 and current 1.2.

Comment 1

[Bradley Baetz (:bbaetz)]

Probably a dupe of 129811, from network traces

*** This bug has been marked as a duplicate of 129811 ***

Comment 2

[Bradley Baetz (:bbaetz)]

Actually, this isn't the same bug, since that server seems to resue teh data
connection even if I tell it to ABOR first. Could be a server bug, I guess.

Comment 3

[Bradley Baetz (:bbaetz)]

Actually, this isn't the same bug, since that server seems to resue teh data
connection even if I tell it to ABOR first. Could be a server bug, I guess.

Comment 4

[Bradley Baetz (:bbaetz)]

Note that in ncftp (with set passive on), doing:

open ftp.caudium.net
get <some-file-which-doesn't-exist> (or ls <non-existant-file>
ls loki

hangs with the same problem mozilla has. So unless someone can suggest a way of
doing this without running into problems (the other way arround has the same
issue, so don't suggest that), and tearing down the connection entirely on file
not found (Actually, we currently do that anyway. Thats not the point here,
though), this bug is INVALID

Comment 5

[Thomas Arendsen Hein]

You're right. The 'ls nonexisting' will screw up the current Caudium 1.2 and the
stable 1.0.34 even from within the same subnet.

I filed this bug to the Caudium bug tracker:
http://sourceforge.net/tracker/?func=detail&aid=530674&group_id=8825&atid=108825

Comment 6

[Bradley Baetz (:bbaetz)]

Bug 129811 is now in, and we still hang with this problem.

server bug; INVALID

Comment 7

[Thomas Arendsen Hein]

The Caudium team told me that they fixed every bug they know of. Other ftp
clients work perfectly now, but mozilla versions 0.9.5 to 1.0rc1 still have
problems.
It even got worse: Before their bugfix some directories failed to load, now I
can't access ftp servers who run a current Caudium installation.

You can test it at ftp://ftp.intevation.de/ and ftp://akira.oav.net:2100/ (both
run Caudium 1.2.5).

Is it still a server problem? If yes, can anyone assist them finding the bug?
The bug is still filed at the Caudium bug tracker:
http://sourceforge.net/tracker/?func=detail&aid=530674&group_id=8825&atid=108825

Comment 8

[Xavier Beaudouin]

Attachment: Galeon TCPFlow between Caudium Ftp and Galeon

This files shows a typical dialog between Gecko and a Caudium FTP.

Comment 9

[Xavier Beaudouin]

Attachment: tcpdump trace for previous file.

Here is the tcpdump flow.

Comment 10

[Xavier Beaudouin]

Attachment: Mozilla and Pure-FTPD flow...

Now with pure ftpd.

Comment 11

[Xavier Beaudouin]

There were a bug on Caudium's FTP... We have corrected it. Thanks... 

Now dialog between server and ftp is ok, but it seems that the FTP engine of
Mozilla/Gecko is ***REALLY** sensitive to the contents of the output of LIST
command.

We have found that the line "total xxxx" seems to block mozilla from parsing the
LIST output...

A corrected version is now at ftp://akira.oav.net:2100/, but it seems that some
mozilla seems to block again on that listing...

Here is the test we made : 

Works with :

- Mozilla 0.9.8 (Linux bin) on freebsd/4.5 with linux emulation
- FreeBSD's command line ftp
- Windows 2K MSIE 6.0.2600.0000 (Win 2K Server / French edition)
- Squid 2.4 on FreeBSD/Alpha (with Mozilla as client or Netscape or Galeon)
- ncftp 3.1.2 on FreeBSD
- Netscape Communicator 4.79/Linux over FreeBSD Linux binary compat
- ncftp from redhat 6.1
- Netscape 4.72 on redhat 6.1
- Omniweb 4.1b4 on MacOS X
- iCab 2.7.1 on MacOS X
- Opera 5.0b3.333 on MacOS X
- MSIE 5.1.4 on MacOS X
- ncftp 2.4.3 on MacOS X

Doesn't work with :
- Mozilla 0.9.9 on linux
- Mozilla 0.9.9.2002050810 Linux binaries on Freebsd/4.5 with linux bin emulation
- Mozilla 1.0 RC1 / RC2 (Debian/woody or MacOS X)
- Galeon (debian/woody one) 
- Chimera 0.2.7 (mozilla-based browser on MacOS X)

So it seems there is something that is not expected to be sent to Mozilla
Engine.... Can you help us to fix that.... ? because as far as we can see this
is more LIST parsing that doesn't like caudium's output...

Thanks,
/Xavier

Comment 12

[Xavier Beaudouin]

OOps I forget to tell you to have a look to the 3 files I've attached to this bug...

Thanks.
/Xavier

Comment 13

[Marek Habersack]

I have been playing with galeon/mozilla and various caudium ftp servers today
logging the transactions using tcpflow. As a comparison I used servers using
vsFTPD and pure-ftpd. Results were consistent in the sense that all of the
servers (including Caudium) were sending very similar data. I have checked the
output from Caudium against the RFC and found no deviations from it. Line
endings, responses etc. were all correct. The only thing I noticed was an output
from netstat (I ran the tests on Debian/Sid w. 2.4.19-pre8-aa2 kernel) where I
found that the receive queue for the galeon/mozilla data connection still
contains 1 packet while the socket itself is in the CLOSED_WAIT state.
Galeon/mozilla is blocked on poll() as evidenced by the top output. At the same
time, all the data sent from caudium was received by galeon/mozilla. Looking at
the Mozilla FTP protocol code it seems that it works correctly by receiving all
the data + the trailing CRLF and yet... one packet remains in the queue. As
Xavier said, all the other tested FTP clients (BSD ftp, ncftp, midnight
commander, konqueror and others) work just fine with Caudium - so either it is a
subtle bug in Caudium induced by Mozilla, or the reverse. I'm out of ideas about
what to do right now...

Comment 14

[Matthias Versen [:Matti]]

Attachment: Mozilla FTP log from ftp://akira.oav.net:2100/

Note : Mozilla never stopped (I clicked Stop)

Comment 15

[Bradley Baetz (:bbaetz)]

No, the problem is still the ftp server:

We send: PASV
Server: 227 Entering Passive Mode. (195,154,210,129,255,145)
Mozilla connects to port 32800 on the server
...
Mozilla: RETR /
Server: 550 /: not a plain file.
Mozilla: PASV
Server: 227 Entering Passive Mode. (195,154,210,129,254,70)
Mozilla connects to port 32801
Mozilla closes the connection on port 32800
...
Mozilla: LIST
Server: 150 Opening ASCII mode data connection for /usr/bin/ls
Server sends out data on the now-closed port 32800, while mozilla waits on port
32801.

This behaviour changed in order to keep compatability with other servers - see
bug 129811 and others.

Caudium is lying to us, by telling us which PASV connection is the latest, but
then sending to a previous one.

Comment 16

[Marek Habersack]

Thanks to the last Bradley's comment I have just fixed the bug in the Caudium
FTP service for all versions in the Caudium CVS (1.0.51, 1.2.5 and 1.3.2). The
problem was that Caudium was reusing the fd allocated for the first PASV issued
by Mozilla (which session was never actually used) instead of using the correct,
matching, fd as accepted for the 2nd PASV session. Thanks a million for the
hint, Bradley! Please consider the bug as closed. Thanks again.

p.s. actually, why is Mozilla issuing two PASV and not using one of them?

Comment 17

[Bradley Baetz (:bbaetz)]

Because some servers (ftp.asus.com is one I recall) will close the PASV
connection after the LIST fails. ns4 works because it falls back to PORT, which
we don't support.

The post-098 behaviour was to deal with servers who send the same port in
response to the PASV, but want a new connection opened (and those that was the
first connection to be usd, too.

The ftp standard is a mess ;)

not a mozilla bug -> INVALID

Comment 18

[benc]

VERIFIED/IN.