Closed Bug 222031 Opened 17 years ago Closed 17 years ago

OSX getaddrinfo returns concatenated results from /etc/hosts and DNS, which breaks /etc/hosts based ad blocking


(NSPR :: NSPR, defect)

Not set


(Not tracked)



(Reporter: jon, Assigned: darin.moz)



(2 files, 2 obsolete files)

User-Agent:       Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.6a) Gecko/20031009 Camino/0.7+
Build Identifier: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.6a) Gecko/20031009 Camino/0.7+

Before I just upgraded Camino from about a month ago nightly build, it was
properly using the OSX lookupd to find hosts. Now, it seems it is using its own
mechanism to find hosts.

The way I know this is not working in Camino and my machine is working is that
Safari works properly.

There is bug #194476, but that is a different problem so don't close this as a
duplicate of that one.

Here is my lookupd hosts configuration:

lookupd -configuration

LookupOrder: CacheAgent FFAgent DNSAgent
ValidateCache: NO
_config_name: Host Configuration
name: hosts

Here is my /etc/hosts localhost broadcasthost
::1             localhost

Now, when I go to using this version of Camino, I see the ads
which are served by Prviously it didn't show any ads. If I
use Safari, I don't see the ads.

I also modified my lookupd configuration and removed DNSAgent to confirm that
lookupd is properly reading its configuration and that is true. It is as if
Camino isn't using lookupd at all!

I also just upgraded to OSX 10.2.8 and thought it might be something with the
new version of OSX, but the lookupd version is the same as what is on 10.2.6.
The fact that Safari works also confirms that it isn't a difference in OSX
versions that is causing the problems.

I'm a bit surprised no one else has reported this issue.

Reproducible: Always

Steps to Reproduce:
see above
Actual Results:  
I saw the ads.

Expected Results:  
Don't show me the ads. =)
Severity: blocker → major
darin? did anything change?
Assignee: pinkerton → darin
Component: General → Networking
Product: Camino → Browser
QA Contact: benc
Version: unspecified → Trunk
I would love it if someone would try to just duplicate the problem and see if
they are having it as well.

It is very easy to do:

what versions of gecko are we comparing here?  was the old version of camino
using the 1.0 branch builds?  i haven't been following camino, so i don't know
what a one month old build means.

also i have no access to a mac with which to test this.  the patch for bug
205726 recently went into the tree.  that should not have had too much affect on
the OSX build, though possibly it means that we are now calling getaddrinfo
instead of gethostbyname on OSX.
adding a line like this to /etc/hosts

works fine (i.e., blocks URLs from loading from that host) with yesterday's
trunk build under linux (RH9).

again, without access to a mac, there's not much i can do with this bug.  it
might be useful if someone provided a mozilla socket log.  there are
instructions on this site which explain how to capture this log:

all i need is to see NSPR_LOG_MODULES=nsSocketTransport:5

exerpts from the log file:

29141392[1c23b60]:   trying address:
29141392[1c23b60]:   connection failed! [reason=804b000d]
29141392[1c23b60]: nsSocketTransport::RecoverFromError [this=2df1770 state=3
29141392[1c23b60]:   trying again with next ip address
29141392[1c23b60]:   trying address:


mozilla-trunk> host has address

so, the conclusion i'd draw is that OSX is giving the application both
and as possible choices for  since in
reality servers can be multi-homed, there is nothing mozilla can or should do to
work around this behavior.  i'd suggest filing a bug against the Apples since
this is clearly a system issue.

most likely we are now calling getaddrinfo instead of gethostbyname on OSX, and
that probably explains the difference in behavior.  we are calling getaddrinfo
in order to support IPv6, and moreover it is the preferred way to do host

i think this bug report is INVALID.
Closed: 17 years ago
Resolution: --- → INVALID
heh.. "the Apples".. not sure where that came from.  i meant, "Apple" of course ;)
I understand that none of you seem to be OSX users (bah! =)), but I think that
you should switch back to using gethostbyname on OSX for three reasons:

#1. the number of people using IPv6 at this point is minimal compared with the
number still using IPv4. As with browser compatibility wars, sometimes it is
best to compromise with what the end user sees and wants as part of the user

#2. This broke functionality of the browser that is extremely useful. I don't
want to see ads and I really like this browser and don't want to have to switch
back to using Safari. Put yourself in my seat. I also don't expect that this is
going to be the last time we hear about this issue. As soon as all of the OSX
users out there who use a hosts file for blocking ads get the new version of
Mozilla, they are going to ask the same questions as I am.

#3. According to the OSX man page for getaddrinfo on 10.2.8, the implementation
is still experimental and non-standard and therefore can be taken as something
to not rely on as the preferred way to do things:

     This implementation is still very experimental and non-standard.  The
     current implementation assumes a one-to-one relationship between inter-
     faces and links, which is not necessarily true according to the specifi-

I will file a bug report with Apple about this issue, however, I think that
until that bug is fixed (does anyone have a copy of Panther to try?), the
previous 'working' behavior should be restored on OSX. Maybe even have Mozilla
conditionally use getaddrinfo only on 10.3 and on 10.2 go back to using

Thank you for your consideration on this matter.
Resolution: INVALID → ---
I filed this as problem ID 3454903 with Apple.
jon: ok, maybe there is something we can do...

hmm.. i wonder if other BSDs behave similarly.

wtc: do you have any thoughts on this bug?  apparently it seems that getaddrinfo
returns results from both /etc/hosts and DNS.  this breaks the ad blocking
technique of listing ad servers under to bypass DNS lookup.  i suspect
OSX is broken in doing this, but maybe there is no promise that it will not
query both /etc/hosts and DNS.  apparently, if we switch to calling
gethostbyname, this problem goes away.  that makes me suspect that the behavior
getaddrinfo is unintended.

one solution is probably to define _PR_INET6_PROBE, so that we will only call
getaddrinfo on IPv6 enabled clients.
Summary: Camino no longer uses lookupd to find dns entries → OSX getaddrinfo returns concatenated results from /etc/hosts and DNS, which breaks /etc/hosts based ad blocking
I think this pretty much says, can I get you guys to put it back the way
it was on 10.2 now?


------ Forwarded Message
From: Marc Majka <>
Date: Fri, 17 Oct 2003 09:55:45 -0700
Subject: Re: getaddrinfo bug

The implementation of getaddrinfo in Mac OS X 10.2 was written to 
support our initial version of IPv6.  It ended up getting used a lot 
more than we had expected.  This was good news, in that it showed that 
there is a lot of code ready to take advantage of IPv6.  We've done 
lots of work to improve IPv6 in 10.3.  Included in that effort was an 
overhaul of getaddrinfo and getnameinfo.  getaddrinfo no longer 
searches all sources for information (e.g. /etc/hosts + NetInfo + DNS). 
  In 10.3 it follows lookupd's host search order and stops at the first 
"match".  It's also faster and takes more advantage of local caching to 
gain better performance.

Marc Majka
darwin-development mailing list |
Do not post admin requests to the list. They will be ignored.

------ End of Forwarded Message
We can certainly determine the runtime OS version
and use getaddrinfo() only if we are on Mac OS X
10.3 or later, and use gethostbyname() if we are
on 10.1 and 10.2.  Could one of you ask Apple to
review this decision?

Does anyone know what is the version of Darwin in
Mac OS X 10.3?  This is the output of the "uname -r"
uname -r returns: "7.0.0"
--> nspr
Component: Networking → NSPR
Ever confirmed: true
Product: Browser → NSPR
Version: Trunk → 4.2
What I just got back from Apple on the subject...

now i'm curious when this will get fixed. =)

------ Forwarded Message
Date: Tue, 21 Oct 2003 19:50:39 GMT
Subject: Re: Bug ID #3454903: getaddrinfo lookup behavior is not the same as
with gethostbyname

Please include the line below in follow-up emails for this request.

Follow-up:  2999907

Re: Bug ID #3454903:  getaddrinfo lookup behavior is not the same as with

Hello Jon,

Thank you for taking the time to bring this issue to our attention. 
Engineering has reviewed your report and has determined that this is
behavior intended by design and is working correctly per engineering

Below, we have provided comments from engineering to help further
explain this conclusion:

>From your bug report, it sounds as if you were trying to block ads in a 
web browser by putting bogus entries in /etc/hosts.  The Mac OS X 10.2 
implementation of getaddrinfo consulted both /etc/hosts and DNS.  This 
meant that the idea of blocking unwanted banner advertising by screening 
out unwanted hosts with bogus host entries in /etc/hosts did not work.  The
behavior of getaddrinfo changed in Mac OS X 10.3 so that it simply followed
the lookupd host search order and stopped at the first entry.  This means that 
in Mac OS X 10.3, one can block access to certain hosts with bogus hosts entries. 

We hope this information is helpful.  Please do not hesitate to contact
us should you wish to discuss this matter further.

Thank you for your time and effort.  Your help is greatly appreciated.

Best Regards,

Maurice Cusseaux
Apple Developer Connection
For follow-ups please contact
Attached patch v1 patch (obsolete) — Splinter Review
simple patch to disable getaddrinfo on OSX.  this means that gethostbyname will
be called instead, which at the moment it treated as non-threadsafe under OSX. 
this is not true for OSX 10.2 and above.  so, i'm not entirely happy with this
patch alone.  i'm also not sure if OSX will return IPv6 addresses via
gethostbyname.	in which case this patch is definitely not good enough.
I think we want to use getaddrinfo on 10.3 (Darwin kernel 7.0) and higher. (10.3
has been released, in case you didn't know.) It's only 10.2 that has this
problem, correct? Let's not hamstring future users. Ideally, Mozilla should
automagically use gethostbyname on 10.0 - 10.2.x and getaddrinfo on 10.3 and up.
That means a runtime initialization probe, I think. Perhaps too complicated. A
better compile-time-oriented patch would do something like this:

ed: yes, i agree.. i'm not happy with the v1 patch either.  i just put it up as
an option.
A possible solution would be to use getipnodebyname instead of gethostbyname. It
supports IPv6 and IPv4-mapped addresses, is thread safe, and supports the
AI_ADDRCONFIG flag. See bug 68796 comment #43.
What we really need is a mechanism that specifically supports ad blocking.
Besides people that are hacking their /etc/hosts files, is anyone else having a
problem with this behavior?

I've always disliked this user hacking of /etc/hosts. If we want to support ad
blocking in this situation, can't we just decide to cancel the request for a
hostname if the IP address is

I think this is network-incoherent, at least I've never seen any technical
discussion that explains why an address is the whole IPv4 network should be
allowed in /etc/hosts. TCP/Illustrated says that this cannot be a valid
desintation address. I'm sure that is what people intend when they hack their
/etc/hosts files, but do we want to encourage this behavior? Do we know other
software will handle this bogus entry correctly?
Ha! the crux of the problem! =)

It would be great to have a simple list that says:

Hosts to block access to:

It should be available via scripting as well so that outside applications can
add/remove items from the list easily.

Note that one benefit of using /etc/hosts over just having an application level
blocking mechanism is that it works in all of the browsers that I use, not just
Camino. So, maybe coming up with another central database that all browsers
agree on using would be better. Until then /etc/hosts is gonna have to be that
database. is my list...=) it is amazing how much faster web pages load and how
much **** i just don't see almost making using a browser on a
non-configured machine unbearable...
Attached patch v2 patch (obsolete) — Splinter Review
Attachment #134432 - Attachment is obsolete: true
Attachment #136387 - Flags: review?(wchang0222)
Attached patch v2.1 patchSplinter Review
revised patch.	wtc noted that _darwin.h needs to enable _PR_INET6_PROBE for
target platforms less than 10.3 not less than 10.2.
Attachment #136387 - Attachment is obsolete: true
Attachment #136387 - Flags: review?(wchang0222)
Attachment #136390 - Flags: review?(wchang0222)
Looks pretty good, Darin. Forgive the following newbie questions: What is the
current value of MACOS_DEPLOYMENT_TARGET and how is its value
assigned/determined? Will the runtime check you've added in the patch to ptio.c
be done just once or will that code be run with every connection made?
MACOS_DEPLOYMENT_TARGET is set by NSPR's configure script.  there is an argument
that can be passed to NSPR's configure script to alter the value.  the default
value appears to be 10.1.

this runtime check will only happen once at startup.
Comment on attachment 136390 [details] [diff] [review]
v2.1 patch

r=wtc.	I like this patch!
Attachment #136390 - Flags: review?(wchang0222) → review+
Target Milestone: --- → 4.5
Comment on attachment 136390 [details] [diff] [review]
v2.1 patch

would be good to get this in for beta.
Attachment #136390 - Flags: approval1.6b?
Attachment #136390 - Flags: approval1.6b? → approval1.6b+
fixed on NSPR trunk and NSPRPUB_4_2_CLIENT_BRANCH.  this fix will appear in
mozilla 1.6 beta.
Closed: 17 years ago17 years ago
Resolution: --- → FIXED
sadly, i'm on 10.3 now so this issue doesn't bug me personally anymore, but
thank you for fixing this! i think all the 10.2.x people will be eternally
grateful and it is the right thing to do anyway.

another satisified customer. =)
VERIFIED: fixed.
Added a entry to /etc/hosts.
Correct behavior would be to block hostname.

Mozilla 1.6a - incorrect behavior.
Mozilla 1.6f - correct behavior.
You need to log in before you can comment on or make changes to this bug.