148707 - Proxy: non-DNS name resolution sightings

Status: RESOLVED WORKSFORME

(Core :: Networking, defect)

Description

[Peter da Silva]

At least that's what it looks like. I have it set to use my squid proxy for
everything, and it sits there 20-30 seconds with the "Resolving
www.whatever.com" message before it seems to suddenly decide to use the proxy
and things start moving quickly again.

I am behind an application-proxy-only firewall. There are no real internet
servers visible and all DNS servers configured in resolv.conf return an
immediate fail to requests. It's like Mozilla has its own DNS library and is
ignoring all my local setup,

Comment 1

[Adrian Ulrich]

wfm 2002060123

Mozilla sends this to the proxy: (symlink.ch)

> GET http://www.symlink.ch/ HTTP/1.1
> Host: www.symlink.ch
> User-Agent: Mozilla/5.0 (X11; U; Linux ppc; en-US; rv:1.0.0+) Gecko/20020602
> Accept:
text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,video/x-mng,image/png,image/jpeg,image/gif;q=0.2,text/css,*/*;q=0.1
> Accept-Language: en-us, en;q=0.50
> Accept-Encoding: gzip, deflate, compress;q=0.9
> Accept-Charset: ISO-8859-1, utf-8;q=0.66, *;q=0.66
> Keep-Alive: 300
> Proxy-Connection: keep-alive
.. and mozilla doesn't try to resolve symlink.ch...

and btw:
> It's like Mozilla has its own DNS library and is ignoring all my local setup,
if you change anything in /etc/hosts & co. you'll have to restart mozilla :/

Comment 2

[Peter da Silva]

No, really, it's doing its own name resolution, somewhere.

More information:

1. I'm behind a proxy-only firewall.
2. I have my own DNS servers that return "no such domain" immediately when
presented with anything but "my local domain". This has been confirmed multiple
times with nslookup and dig. They don't forward and they have had all references
to the root servers surgically removed (under anesthetic, I'm quite humane).
3. I have restarted mozilla several times during this process, and in any case I
have made no changes to resolv.conf, my nameservers, hosts, &c &c &c &c...

Yes, Mozilla does eventually send the right thing to the proxy.

However, I've been experimenting further. My initial proxy settings were cloned
from the ones Netscape had been using: host name "firewall" port "1984" for
everything (well, everything but socks, of course, socks is empty), and a set of
exceptions for IP addresses the firewall doesn't know how to get to because
they're inside a deeper firewall that doesn't fully trust the outer firewall.

Now before you go all "aha" at me again, I'll add that I haven't tried to get to
any of those addresses with Mozilla.

So, I took out those IP addresses, on the assumption that maybe it's doing a
gethostbyname(target) to check it against those addresses. Nope. So, now I'm
sitting here scratching my head, checking that maybe my ponytail is tied too
tight, the usual stuff a BOFH does when things are getting weird. Anyway...

Time for the big guns. Time for tcpdump.

I've got a lot of random extraneous traffic on my local net, so I go and get the
kids computers down and all videogames suspended (and there was much wailing and
gnashing of teeth), and fire up tcpdump. And...

When I enter a name in Mozilla I get netbios traffic. Now I'm not enough of a
Windows geek to know what this traffic actually is, and I didn't pull out the
necronomicon to see... but it's a broadcast on port 138 to my local subnet with
the source name equal to my local computer name (truncated to the first word and
in UPPERCASE), and I know damn well I shouldn't be getting netbios traffic from
a FreeBSD 4.5 box. So I go check my proxy settings again and change "firewall"
to "firewall.domain.from.resolv.conf"... that is, I put in the fully resolved
domain.

Now I enter "www.advogato.com", and it gives me a brief "resolving
www.advogato.com" and looks up my firewall, then sends the proxy request.

Now, this is downright weird.

I don't have the mozilla source locally to look at (I downloaded the FreeBSD
binary from your website, and then built the necessary gnome-infested libraries
from the FreeBSD ports tree), but it sure seems like Mozilla is going "whoa,
dude, that's an unqualified name. Before I go and waste time on trying to find
its name on your undoubtedly Bill-Gates-infected network, I'll do a quick check
for a WINS server and take care of you...", instead of saying "whoa, dude,
that's an unqualified name. What's resolv.conf want me to do with unqualified
names? OK, I'll do that, hey, that works, cool..."

I could be imagining things, but that's sure what it looks like. I
double-checked in case I'd installed Samba some time and had forgotten about it,
but no, that's not it. I am... well... staggered. I can see why this kind of
thing might get into the code base, but I sure as heck didn't expect this behaviour.

So, the workaround: use a FQDN in the proxy settings.

Well, that seems to be working, anyway.

The fix: now, if it was up to me, I'd rip out any funky traditional-netscape
name resolution code and just use the local operating system's "gethostbyname".
OK, OK, I know that some systems (Solaris, for example) have some pretty ugly
code in there, but at least give me a prefs option to say "no, I am using the
holy anointed gethostbyname from the pure source of all good bits, BSD and BIND,
and by all that's holy, by the name of McKusick and the name of Vixie I enjoin
thee, THOU SHALT USE IT".

OK? Is this thing on?

Comment 3

[Adrian Ulrich]

So much text :)

> So, the workaround: use a FQDN in the proxy settings.

Ok.. now i can see the problem (2002060123 linux ppc):

- Entering 'localhost' and 'ibook' as proxy doesnt work -> mozilla just sends
NOTHING

- But '127.0.0.1' or 'ibook.lunix' does work..


(OS -> ALL?? Is this a unix-only bug? Has win32 also this problem?)

-> NEW
..and i am confused that i couldn't find a dup for this..

Comment 4

[Darin Fisher]

ok, so mozilla simply calls 'gethostbyname' on unix systems.  it calls
'gethostbyname' to lookup the proxy servers address.  if that is what is taking
so long then i'm not sure what can be done about it.  does ping have similar
problems with non-FQDN's?

Comment 5

[Peter da Silva]

Of course I can't completely rule out a bug in my DNS setup. But...

If there's a problem with my DNS setup it's sufficiently obscure that it hasn't
erupted in the past few years. I have two local DNS servers running Bind 8, each
set up without the root name cache (so unknown domains return failure
immediately), resolv.conf looks like this:

domain  in.taronga.com
search in.taronga.com dmz.taronga.com
nameserver 127.0.0.1

Everything else on my system resolves the firewall's name perfectly well,
including Netscape 4.

You absolutely sure Mozilla's just calling the normal gethostbyname() for the
proxy? I tried looking through the source online but glimpse isn't really the
best tool for tracing lots of method calls and message passing to helper apps
and things...

Comment 6

[Darin Fisher]

netwerk/dns/src/nsDnsService.cpp calls PR_GetIPNodeByName which calls
gethostbyname_r under linux.  you'd have to check the source code for
PR_GetIPNodeByName to see what it resolves to under FreeBSD.

Comment 7

[Peter da Silva]

Well, it doesn't look there's any paths that lead to hell or Microsoft there,
but I would imagine that any suspect code would be further back up the tree then
the DNS resolver, but I can't trace the twisty little maze of method calls. I've
spent too much time digging through it already (look at the timestamps) and I'm
no closer to figuring out where the actual proxy name resolution happens.

Comment 8

[Kris Gale]

I can verify this happens on FreeBSD, but I'm not using any proxy configuration
whatsoever.  As such, I haven't been able to use the workaround mentioned above.
Any domain name I type in for a particular session hangs for 30 seconds or so
with the status bar reading "Resolving host [whatever]..."  No other process
that does DNS resolution has ever had this problem.  Once the host name is
looked up, everything works fine, until I travel off to another host or close
Mozilla.

A coworker of mine had exactly the same issue, also on FreeBSD.

Here's all of my version info:

/home/kris $ uname -rsv
FreeBSD 4.5-STABLE FreeBSD 4.5-STABLE #0: Wed Apr 24 14:54:33 MST 2002

/home/kris $ mozilla --version
Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.0.0) Gecko/20020607 <developer
build>

/home/kris $ pkg_info | grep -i mozilla
mozilla-1.0,1       The open source, standards compliant web browser
mozilla-embedded-1.0,1 The embeddable component of mozilla communicator
web-surfbo
mozilla-headers-1.0,1 Header files for mozilla communicator web-surfboard

Comment 9

[Darin Fisher]

-> gordon (DNS)

does anyone build debug?  if so, you can try enabling the following environment
variables to try to see what mozilla is doing.

  setenv NSPR_LOG_MODULES nsHttp:5,nsSocketTransport:5
  setenv NSPR_LOG_FILE /tmp/moz.log

if you upload /tmp/moz.log to this bug report, we might be able to get to the
bottom of this bug.  thx!

Comment 10

[benc]

I'll look into this soon.

Comment 11

[Andrei Boros]

Same behaviour on Mozilla 1.6 on both windows 95 and windows 98. 

Status bar shows "Resolving host ..." before it sends the request to the proxy
server. (and the corresponding delay. I've taken the DNS server offline and
Mozilla waits a looong time for the name query to timeout).

Comment 12

[benc]

I had a strange linux experience where the same thing happened, I had crippled
DNS by hacking resolv.conf and rebooting, (so no OS caches either). If I tried
to look up hostnames, there would be a strange, long pause, and then a connection.

This doens't happen on my Mac when I trace, but I've seen the same symptoms.

Comment 13

[Gennady Feldman]

I have been experiencing the same thing here on Win2k behind HTTP proxy server.
There's local DNS servers running. I usually open a bunch of tabs when i start
Firefox and it does take a small delay (which actually tends to vary?) before I
get the HTTP Proxy Authentication window. Never seen this in IE where the Proxy
authentication is almost instant. 

Comment 14

[tarvid]

I have the same problem using Firefox 2.0.0.3 on WinXP behind MS ISA server - it freezes for the 30secs time period.

I discovered that this problem occurs if:
1) "autodetect proxy settings" is on
2) and netbios is enabled
3) and non existing host is contacted

the tcpdump shows that before the browser sends the request to the proxy server it tries to detect the domains IP. DNS fails immediately but netbios request are sent to the network many times until the resolving after about 30secs expires. Then the proxy is contacted and "page not found" from proxy received.

The solution is to disable netbios on the network device interface or manually enter proxy settings.