74546 - SOCKS : V5 implementation broken (no connectivity)

Status: VERIFIED FIXED

(Core :: Networking, defect)

Description

[[e-mail]]

OVERVIEW:
While helping bbaetz try to get Gopher over SOCKS to work this weekend, we found 
that many builds do not seem to have SOCKS working.

To my knowledge, I am aware of all SOCKS issues, which you could roughly find a 
summary of at:
http://www.packetgram.com/pktg/mozilla/bugzilla/socksdigest.html

The big problem is that it seems that ALL milestone releases fail to make 
successful SOCKS connetions. This is NOT the partial connecctivity problem 
described in bug 48357.

I am using a SOCKS V5 compliant server from NEC, and it works with AIM and NIM's 
SOCKS V5 mode. I also have started gathering packet trace data, which I will add 
when I get more time.

STEPS:
1- Create new profile
2- Set proxy to "manual" and enter a SOCKS proxy hostname and port.
3- Open new window (or restart browser). Observe the home page fails to load w/ 
error:

"The connection was refused when attempting to contact www.news.com."

EXPECTED BEHAVIOR: 
SOCKS connection should transparently occur and load the page.

NOTES:
Entering a http proxy forces the client to select the http proxy and connect.
Removal of the SOCKS proxy entry allows the client to connnect.

Windows, Netscape 6.0 (RTM).

I'll be adding reports of other builds later today.

Comment 1

[neeti]

Gagan, any comments on this one?

Comment 2

[Gagan]

cc'ing Jeremy-- he is so looking forward to commenting on this bug :) Knock 
yourself out Jeremy.

Comment 3

[[e-mail]]

Here are the verbose snoop traces.

A failed connection:

http://www.packetgram.com/pktg/mozilla/bugzilla/74546/failure.txt

A quick look shows that after the three-way handshake, packet 4 comes from the 
client and says: 

TCP:        .... ...1 = Fin

I <think> this was a trace of Mozilla 0.8, it might be of NS 6.0.1, but they 
seem to have the same behavior.

A working connection:

http://www.packetgram.com/pktg/mozilla/bugzilla/74546/worksforme.txt

You can see the 1st SOCKS packet (#4) does not "FIN" on the working connecction, 
instead it does a PUSH.

TCP:        .... 1... = Push

Unfortunately, snoop recognizes the snoop connection via the port number, but 
was not kind enough to display the payload when in binary (so you can't see the 
handshake). I need to dig up the original .snoop files and post them as well. 
(thanks to bbaetz for pointing this out...)

Comment 4

[benc]

QA to me.
clarified subject.

Comment 5

[Jeremy Loeb (gone)]

I think everyone would be happier if SOCKS protocol support were included in 
6.5. Jeff Hooker and Eclient have offered up their engineering and QA resources 
to help get this implemented. CC:ing Jeff and Eric Paczewitz on this bug. Please 
take a look at the bug history, so we can come to an agreement on work to be 
done. Perhaps a meeting is in order? ;)

Comment 6

[lchiang]

Need to see if this will be a large landing.   If so, we have to plan this
landing with chofmann.

Comment 7

[Bradley Baetz]

Has anyone got real packet traces for this (showing binary data)? For all we
know, it could just be that particular server not liking the fact that we don't
(according to the comments in the code) advertise supporting the mandatory
authentication scheme (GSSAPI) (or, in fact, any authentication scheme)

Since this was checked into the tree, it must have worked at some point, on at
least one type of server.

benc - you verified bug 16103 (the "support SOCKS" bug) on Mar 12, but the url
you give just gives a file not found error. What server did you use for that?

Comment 8

[benc]

CONFIRMED: broken on commercial Mac (2001-04-30-??-Mtrunk)
setting platform to all/os to all.

loeb: lets call this meeting ASAP.

bbaetz: I might have the full packet traces, I will look for them. as an
external contributor, I never thought anyone would ask for the original snoop
files for an article like this. Based on my recollection, I do not think there
is additional useful data in those traces. At this stage, somebody needs to find
out why we tell the stack to FIN out the connection.

The URL moved to:

http://www.packetgram.com/pktg/docs/net/snoop/socksclientversion.html

This was working in the commercial Win32 daily builds. When I was in IM QA, I
had configured an evaluation copy of NEC eBorder. I used several builds, and
actually dogfooded off of SOCKS-only for a day or two. I also used netstat to
verify that this worked. That server was available on the public net, and I
hijacked it at night to do the verification. 

That server (socks.packetgram.com) is currently down (trial expired), and kerz
has inherited the job of getting a valid license. 

Comment 9

[[e-mail]]

<putting on my hat as the sysadmin of *.packetgram.com, which provides test
servers for mozilla.org>

The eBorder server mentioned above was available because it was an evaluation
copy. The licensed copy may or may not be deployed publicly (because IM QA does
not have a testing requirement for a publicly available SOCKS server).

I am currently investigating alternatives for network standards qa (tever + me
doing my day job + other contributors).

It is a reasonably well-known fact that there are SOCKS servers available on the
AOL/Netscape internal network. Yet the case remains for a publicly available
SOCKS server.

1- external contributors need it. bbaetz has mentioned to me on more than one
occassion that the lack of a publicly-available, working SOCKS server has
blocked his development of gopher support of SOCKS. 

2- the test servers should reside publically. Netscape could consider giving
limited network access to individuals that really need access to existing
internal servers. Presumably, this is administratively difficult and not cost
effective. Legally, this could be problematic if someone worked for a competitor
in a day job and contributed at night.

3- Simple access to a SOCKS V5 server is unsufficient for the testing needs of
this project. Direct control is needed so the server is configured correctly for
a given test. System level access is needed for analyzing complex problems (such
as gathering the trace data mentioned above). Vendor technical support will be
needed. NEC was kind enough to provide technical support for the evaluation
copy, which answered several nagging questions about version interoperability.

The general criteria for testing the browser SOCKS V5 implementation suggest
that Netscape Proxy Server is the ideal candidate. 

1- It does not support SOCKS V4. eBorder does, and version support is not
configurable, which makes testing connections to single version-only server
impossible. (This sounds irrational, but are discussed in the other version
related bugs I have filed) 

2- Administravia barriers are limited. Netscape Proxy Server 3.5 is already
deployed on proxy.packetgram.com, which is publicly available. I will be testing
the SOCKS server as soon as time permits, and will report back here.

Comment 10

[Bradley Baetz]

I found http://www.inet.no/dante/, which is a BSD-style licenced socks server -
I'll try it in a couple of hours.

What was the date that you know that this last worked?

I want the binary data so that we can see what the client and server are telling
each other. Note that the trace can work both ways - I can trace it (and did so
earlier, but didn't save the traces), as long as I have somewhere to connect to.

Re point 2 - I've heard rumours of a bandwidth limiting external proxy server
that people with cvs accounts have access to. I don't know anything more about
it though, or whether it actually exists.

Comment 11

[Bradley Baetz]

OK, I'm now using dante locally as a socks (v5) server without any problems in
mozilla. So socks is working, at least in the general case.

Comment 12

[benc]

Nominating nsbeta1, nscatfood.

Comment 13

[Steve Meredith (gone)]

Between 89206, which is fixed, and 65583, for which I have a patch ready, these
SOCKS 5 issues should be fixed. 

Comment 14

[Steve Meredith (gone)]

Should be fixed by fix for 65583.

Comment 15

[benc]

VERIFIED:
This has worked for a while, including Netscape 7.0 and a couple versions of
Mozilla 1.0RCx