195242 - Chimera sends HTTP commands to FTP proxy when accessing FTP URLs

Status: VERIFIED WORKSFORME

(Camino Graveyard :: General, defect)

Description

[Matt Meissner]

User-Agent:       Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.0.1) Gecko/20030225 Chimera/0.6+
Build Identifier: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.0.1) Gecko/20030225 Chimera/0.6+

When downloading an ftp:// URL, Chimera uses the FTP proxy defined in the System
Preference pane.  I believe that field is reserved for FTP-specific proxies
("user@host", "site hostname", etc.) though I can't find a reference on Apple's
site to confirm that. In practice it is -- no other browser on the Mac has this
behavior.  IE, OmniWeb, and Safari all send ftp:// URLs to the proxy defined as
the HTTP proxy, and FTP-specific apps like Fetch use the FTP Proxy settings. 
Mozilla doesn't use the system proxy configuration, so it is not affected.

This problem cropped up in the nightly builds following 0.6 release, but I'm not
certain exactly when.

Reproducible: Always

Steps to Reproduce:
1.  In system prefs, set ftpproxy and http proxy to two different systems. 
Ideally, the FTP proxy would be an FTP-specific proxy, and the HTTP proxy would
be a typical HTTP proxy
2. Start Chimera.  Attempt downloading an ftp:// URL.
3.  Start Safari, IE, or Omniweb.  Attempt to download the same ftp:// URL.
Actual Results:  
The download will fail in Chimera because the proxy doesn't understand the
request.  It will succeed in other browsers because they are using the HTTP proxy.

Expected Results:  
Chimera should have used the HTTP proxy to download the ftp:// URL.

Comment 1

[Matt Meissner]

Sorry, just double-checked the behavior of other browsers -- I was incorrect.

OmniWeb shows the same sympton as Chimera.  

Safari sends ftp:// URLs to the default ftp:// handler.  

IE works as described, however.

Chimera used to work as IE did -- and I still believe it is the correct behavior.

Comment 2

[Greg K.]

Are you sure FTP URLs shouldn't use the FTP proxy? Sounds pretty straightforward
to me.

Comment 3

[Matt Meissner]

Well, I _THINK_ ftp URLs should go to the HTTP proxy -- let me explain.

When I try to access ftp://ftp.mozilla.org/ for example, Chimera sends this to
the FTP proxy:

GET ftp://ftp.mozilla.org/ HTTP/1.1
Host: ftp.mozilla.org
User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.0.1)
Gecko/20030225 Chimera/0.6+
Accept:
text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,video/x-mng,image/png,image/jpeg,image/gif;q=0.2,text/css,*/*;q=0.1
Accept-Language: en-us, en;q=0.50
Accept-Encoding: gzip, deflate, compress;q=0.9
Keep-Alive: 300
Proxy-Connection: keep-alive


So it's actually making an HTTP request to my FTP proxy.  If Chimera is going to
make an HTTP request, I think it should go to the HTTP proxy -- my FTP proxy
doesn't understand HTTP requests.

Comment 4

[Greg K.]

Ah, so it sounds like the problem isn't that Chimera wrongly uses the FTP proxy,
it's that it's sending HTTP commands to the FTP proxy. Revising summary.

What does a dedicated FTP client send to the FTP proxy?

Comment 5

[Matt Meissner]

Well that's where it gets tricky.  Fetch supports 8 different FTP-proxy
protocols.  ncftp supports 7.  That's probably part of the reason IE simply uses
the HTTP proxy and Safari punts on ftp URLs.  And to make matters worse, the
Network prefs pane has no place to specify which proxy protocol to use.  Fetch
uses Internet Config to set and retrieve that info.

I'm no expert, but I believe the only way the protocols differ is in their login
mechanism and the initiation of the connection ("SITE hostname",
"user@hostname", etc.).  After the initial connection is made to the proxy, and
the proxy has connected to the remote host, I think you just send straight-up
FTP commands on the control connection and the proxy simply forwards your
commands to the remote server.  The only trick in my experience is that all data
connections must be made in passive mode -- the proxy has to do a little bit of
address translation and stuff, but it works.

For kicks, here's a Fetch transcript using my proxy type ("SITE hostname"):

Connecting to ftpproxy.myhost.com port 21 (2/28/03 12:06:29 PM)
220-***************************************************************************
220-*                                                                         *
220-*  This is a private machine and network storing proprietary information. *
220-*  Access must be approved by "the management of this organization".      *
220-*  Users may not copy, add, delete, modify or use information on this     *
220-*  system or network without express permission of "the management of     *
220-*  this organization".                                                    *
220-*                                                                         *
220-*  External users are being monitored. Unauthorized attempts to login     *
220-*  are recorded. Intruders will be prosecuted.                            *
220-*                                                                         *
220-***************************************************************************
220 [002-0018] ftpproxy.myhost.com FTP proxy 5.0.0 ready.
USER meissnem SITE
230- user meissnem logged in.
230 [002-0024] Specify Remote Destination with: quote site hostname
SITE ftp.mozilla.org
220-(    [002-0059] Firewall connected to ftp.mozilla.org (64.12.168.243).)
220-(220 ftpmoz.newaol.com FTP server (SunOS 5.8) ready.)
220 [002-0060] login with: user name
ADAT
500 [002-0132] Command not understood.
USER anonymous
331 Guest login ok, send your complete e-mail address as password.
PASS
230 Guest login ok, access restrictions apply.
SYST
215 UNIX Type: L8 Version: SUNOS
PWD
257 "/" is current directory.
MACB ENABLE
500 [002-0132] Command not understood.
PWD
257 "/" is current directory.
CWD /
250 CWD command successful.
PWD
257 "/" is current directory.
PASV
227 Entering Passive Mode (xxx,xxx,xxx,xx,132,2)
LIST
150 Opening ASCII mode data connection for /bin/ls.
total 568
drwxr-xr-x   7 root     other       4096 Feb 14 12:13 .
drwxr-xr-x   7 root     other       4096 Feb 14 12:13 ..
-r--r--r--   1 root     other        351 Aug  8  2000 Welcome
lrwxrwxrwx   1 root     other          8 Nov 20 11:32 bin -> /usr/bin
dr-xr-xr-x   2 root     other       4096 Feb 14 12:13 dev
dr-xr-xr-x   2 root     other       4096 Feb 14 12:13 etc
dr-xr-xr-x   3 root     sys         4096 Feb 14 12:13 opt
drwxrwxr-x  16 22       22          4096 Feb 28 13:22 pub
dr-xr-xr-x   5 root     other       4096 Feb 14 12:13 usr
226 ASCII Transfer complete.
QUIT
221 Goodbye.

Comment 6

[Mike Pinkerton (not reading bugmail)]

what happens when you try this with a trunk build?

Comment 7

[Mike Pinkerton (not reading bugmail)]

please reopen if this is still an issue on the trunk

Comment 8

[benc]

V/wfm.

HTTP Proxies are actually URL proxies. All commands they receive are GET <URL>

The URL scheme can actually be anything, as long as the proxy supports it. HTTP, HTTPS (not SSL), CONNECT (which can tunnel SSL-based stuff like HTTPS, SIMAP, LDAP over SSL, etc), FTP, Gopher, WAIS.

In this light, "we" have always thought of each type of proxy as describing "the proxy you want URLs of this particular scheme to go to (using URL proxy).

This was the general technology direction set by Netscape (because it made a "Web Proxy" product. IE and other browsers seem to have follwed this design.

The pre-existing proxy technologies (most commonly used by ftp clients) is completely separate, and obviated by this browser-proxy style.

As for Apple's intent on this matter, I cannot tell. The version of Safari I have seems to send ftp: URLs to the default handler, which on my system is... Camino.

I don't know what IE for Mac does. 

By coincidence, the Apple provided list matches the main list supported in the manual proxy UI of core. I don't know if we know what to do w/ the entries that are in Apple's list, but not in Necko's manual UI, and I probably won't have time to figure that out anytime soon.

As I've mentioned in other bugs, I also cannot seem to find any detailed documentation from Apple on their true intent for these settings.