621429 - Proxy configuration : "Use system proxy settings" assumes no proxy if IE is set to auto-detect itself.

Status: RESOLVED FIXED

(Core :: Networking: Proxy, defect, P3)

Description

[emmanuel]

User-Agent:       Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b8) Gecko/20100101 Firefox/4.0b8
Build Identifier: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b8) Gecko/20100101 Firefox/4.0b8

Trying to setup automatic proxy discovery through WPAD. Firefox's default setting on Windows, "Use system proxy settings" does not seem to understand that it needs to perform auto-detection if Internet Explorer is set to "Automatically detect settings".

Reproducible: Always

Steps to Reproduce:
1. Set up a working WPAD environment (I used both DHCP and DNS)
2. In Internet explorer, set the proxy settings to auto-detect (do not provide actual values, do not specify the script's url, only check "Automatically detect settings" - this is the default in Vista / 7)
3. Set firefox to use system proxy settings (default in newer versions)
4. Restart firefox and test whether a proxy is in use.
Actual Results:  
Firefox tries to connect directly

Expected Results:  
Firefox should try auto-discovering the proxy settings using wpad.

I don't think there is an error in my wpad setup as IE and Chrome work fine and Firefox too if i set it to auto-discover the proxy settings. However this is not the default setting and I cannot enforce it on my network (roaming users, guest laptops, etc...).
My DNS suffix is ".local" which I thought my cause issues if firefox was blacklisting anything that is not a sub-domain (cf wpad.com) however in this case forcing the auto detection would not work either, would it ?

This is my PAC file, very simple as you can see :

function FindProxyForURL(url, host)
{
// If URL has no dots in host name, send traffic direct.
	if (isPlainHostName(host))
		return "DIRECT";

// If IP address is internal or hostname resolves to internal IP, send direct.
	var resolved_ip = dnsResolve(host);
	if (isInNet(resolved_ip, "10.0.0.0", "255.0.0.0") ||
		isInNet(resolved_ip, "172.16.0.0",  "255.240.0.0") ||
		isInNet(resolved_ip, "192.168.0.0", "255.255.0.0") ||
		isInNet(resolved_ip, "127.0.0.0", "255.255.255.0"))
		return "DIRECT";
					
// All other traffic uses below proxies, in fail-over order.
	return "PROXY proxy.local:3128; DIRECT";
}

Comment 1

[Jo Hermans]

I think I found it, but I don't think it will be easily fixed.

The code in <http://mxr.mozilla.org/mozilla-central/source/toolkit/system/windowsproxy/nsWindowsSystemProxySettings.cpp> is not looking at the PAC-file that might have been found by the WPAD code. That is, it will only use a static proxy (ProxyServer key) or a static PAC-file (AutoConfigURL key). I don't see a way to detect that a WPAD server should be used, and a quick look in my registry (HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings\) doesn't reveal anything to me.

According to <http://technet.microsoft.com/en-us/library/cc302643.aspx>, the PAC-file (if there is one) can be found in HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings\Connections . Look in the DefaultConnectionSettings and SavedLegacySettings registry-entries.

But I don't think that will work too, as those entries are filled in by Internet Explorer (or the DLL actually, which is used by other applications too). If no IE was running before, then there would be no PAC-file found, right ?

Comment 2

[madengineer]

I'd like to expand on this:

If I set FF to "Auto-detect proxy settings for this network" then FF will, as it says, detect the proxy and apply those settings from wpad.

If however FF is set to "Use system proxy settings" with the system set to auto, it doesn't try to do the same and autodetect settings.

FF's autodetect works fine, but doesn't apply when set to "Use system proxy settings".  I guess the logic is to copy the proxy setting, not the network settings.

Comment 3

[Jo Hermans]

The problem is that when a WPAD server is used, there is no data that can be found in the registry (PAC-file in this case), until the Windows DLL is running. Firefox uses their own networking code, so the registry will not be automatically filled in by the DLL.

Now, if Firefox can detect the *name* of the WPAD server from the IE registry, then it could query that server by itself.

Comment 4

[madengineer]

HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings\Connections\DefaultConnectionSettings

if on then autodetect

Comment 5

[Jo Hermans]

That contains a PAC file, but not my WPAD server (I sometimes toggle between the 2 different settings).

It probably included in one of the non-readable bytes of that structure. At first i thought it was in the 5th byte, but that seems to be a simple ID (incrementing every time you change the configuration).

Comment 6

[madengineer]

 select case lcase(status)
   case "on"    binaryVal(8) = binaryVal(8) OR 8        'Force Autodetect on
   case "off"    binaryVal(8) = binaryVal(8) XOR 8        'Force Autodetect off
   case "show"    wscript.echo "Automatically detect is set to " & ((binaryVal(8) and 8) = 8)
   case else    wscript.echo "Invalid parameter - IEautomaticallydetect  on, off or show"
 end select

Tested and works for wpad autodetect

i should refer  'howeasyisthat.com'

Comment 8

[alex_mayorga]

I think the version here should change to Trunk as I still see this problem on Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:8.0a1) Gecko/20110814 Firefox/8.0a1 ID:20110814030749

Comment 9

[Gwhynott]

Hello,

Just want to confirm this issue still exists in 10.0.2.  The default setting of "use system settings" does not work as expected when the windows internet control panel has 'auto detect' selected.  

   We configure our browsers via WPAD which is passed to clients both by DHCP and DNS and is part of the AD GP (added as an attempt to correct when we ran into this issue).    Every browser works out of the box except for FF.  (which is a bit painful when you have 700 clients do deal with)

Comment 10

[Olo]

It's a pain in the ass and it still occurs in Firefox 10.0.4 ESR. We configure WPAD only through DHCP as it is the only reasonable scenario.

Comment 11

[Masatoshi Kimura [:emk]]

WinInet provides an official way to obtain the WPAD setting.

Comment 12

[Olo]

Could someone take a look at this one - <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=356831" title="Bug 356831">Bug 356831</a> - Proxy autodiscovery doesn't check DHCP (option 252)?

Comment 13

[Masatoshi Kimura [:emk]]

I tried implementing this, but it caused a xpcshell-test crash on because WPAD is enabled by default on windows.
IE fallbacks to manual PAC url if WPAD server was not available.
https://tbpl.mozilla.org/?tree=Try&rev=7f56ae134b63

Comment 14

[fukawi2]

This bug is showing itself in our environment (identical to the OP)

This is nearly 2 years old with no progress?? We need to use automatic detection for laptop users; explicitly setting the WPAD means they will not have connectivity outside the office, and setting no proxy will not work inside the office.

And for whatever reason, Firefox keeps defaulting back to "Use System Settings" instead of staying on Auto Detect (separate issue that I need to figure out as a work around, but "Use System Settings" should work for WPAD environments.

Comment 16

[Rand Dow [:randix]]

nsProtocolProxyService::PrefsChanged() asks for the system proxy settings, but then does not properly evaluate them

I would expect that the configuration: "Use system proxy settings", would then look at all of the system proxy settings and use them, i.e.,
0) if no proxies, then we use no proxies 
1) specific http, ftp, … settings, 
2) if the system is set to a specific PAC file, then load that, and
3) if the system proxy setting is set to autodiscovery (WPAD), then we would do the WPAD autodiscovery ourselves.

Patrick McManus <[e-mail]>:

http://mxr.mozilla.org/mozilla-central/source/netwerk/base/public/nsISystemProxySettings.idl#17

your choices are either to use the PACURI in that interface or call systemProxySettings->GetProxyForURI() and let the OS do all the work, I would think in this case we would be doing the latter and the OS should be doing the wpad if configured to do so. At least through this interface we don't have any insight whether or not the syste mproxy settings is set to do autodiscovery.

you might inquire with biesi - I think he wrote this code.

Randall Dow <[e-mail]>:

We might consider a re-design of what we expect this to do.

Comment 17

[Josh Aas]

Reassigning to rdow given no activity from the current owner.

Comment 18

[Rand Dow [:randix]]

Attachment: Patch to pass systems settings for Autodetect through to proxy discovery

This makes windows work the same as osx: GetPACURI returns the WPAD_URL if the systems settings were set for auto discovery and firefox settings are to track the system settings.

Comment 19

[Masatoshi Kimura [:emk]]

Comment on attachment 706551 [details] [diff] [review]
Patch to pass systems settings for Autodetect through to proxy discovery

>+    // Bug 621429: test added, then Value union contains no string..
>+    // when this function called for INTERNET_PER_CONN_AUTODISCOVERY_FLAGS
>+    if (!(aFlags & PROXY_TYPE_AUTO_DETECT)) {
>+        aValue.Assign(options[1].Value.pszValue);
>+        GlobalFree(options[1].Value.pszValue);
>+    }
This will cause memory leak because aFlags can contain PROXY_TYPE_AUTO_DETECT even if other options are enabled. Windows system proxy settings can enable multiple options simultaneously.

>+        rv = ReadInternetOption(INTERNET_PER_CONN_AUTODISCOVERY_FLAGS, flags, buf);
Don't call the function again. |flags| already have an appropriate value. It is a waste of time.
Thus you don't have to change inside ReadInternetOption either.

Have you ever run the patch on tryserver? I bet it will fail (see comment #13).

Comment 20

[Rand Dow [:randix]]

The current problem is, as I see it, that there is no interface to find out from nsProtocolProxyService.cpp to the OS dependent code to find out if autodetection for WPAD should be done, if that's what the IE settings are. And we do our own WPAD auto detection, Win32 won't do it for us.  Thus is my proposal to change the Win32 GetPACURL() to return the path WPAD_URL if Autodetect is set and there is no other URL. Yes, :emk, I agree, this must be done correctly without memory leaks. This would make it work the same as OSX, which works for me in all the permutations of "Auto Proxy Discovery" and "Automatic Proxy Configuration". I.e., when Fx is set to use "Use system proxy settings". I will experiment more with the exact results coming from ReadInternetOption() and try to make a fuller picture of the possible results coming from the InternetQueryOptionW() call.
Comments?

Comment 21

[Masatoshi Kimura [:emk]]

I'm fine if the memory leak is fixed and the redundant call is removed.
Could you attach a new patch?

Comment 22

[Mozilla RelEng Bot]

Try run for bf80c5769cfb is complete.
Detailed breakdown of the results available here:
    https://tbpl.mozilla.org/?tree=Try&rev=bf80c5769cfb
Results (out of 6 total builds):
    exception: 1
    success: 2
    warnings: 3
Builds (or logs if builds failed) available at:
http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/[e-mail]-bf80c5769cfb

Comment 23

[Patrick McManus [:mcmanus]]

(In reply to Rand Dow [:randix] from comment #20)
> 
> And we do our own WPAD auto detection, Win32 won't do it for us.  Thus is my
> proposal to change the Win32 GetPACURL() to return the path WPAD_URL if
> Autodetect is set and there is no other URL.

I might be reading this wrong (I haven't read the patch) so please bear with me.

But that changes the default behavior, right? On windows we would use the system settings and if no proxy was configured we would then immediately do wpad whether or not windows was actually configured to use wpad? That seems like a giant security hole. (i.e. wpad can easily be exploited, we need to have it opted into at either the firefox or the OS level.)

am I reading that wrong?

Comment 24

[Masatoshi Kimura [:emk]]

Apparently I won the bet :)
Automated-tests try to connect local WebServer for some tests. WPAD effectively screw those tests by returning some values from GetPACURI().
WPAD is enabled on Windows by default (hence on test machines).

(In reply to Patrick McManus [:mcmanus] from comment #23)
> But that changes the default behavior, right? On windows we would use the
> system settings and if no proxy was configured we would then immediately do
> wpad whether or not windows was actually configured to use wpad? That seems
> like a giant security hole. (i.e. wpad can easily be exploited, we need to
> have it opted into at either the firefox or the OS level.)

The behavior is almost the same as "Auto-detect proxy settings for this network." So I don't think it introduces a new danger.

Comment 25

[Rand Dow [:randix]]

Attachment: Return the WPAD_URL if PROXY_TYPE_AUTO_DETECT and no other URL.

This is a simpler and safer patch. It *only* returns the WPAD_URL if Windows system settings has that flag set and there is no other proxy autoconfig URL set.

Comment 26

[Patrick McManus [:mcmanus]]

(In reply to Masatoshi Kimura [:emk] from comment #24)
>
> WPAD is enabled on Windows by default (hence on test machines).

So I've been asking around on this exact point and I've heard mixed things. Are you 100% sure (I'd love to get a certain data point) - it would be great to understand for sure; thank you!

> The behavior is almost the same as "Auto-detect proxy settings for this
> network." So I don't think it introduces a new danger.

For the sake of argument, let's assume WPAD is a security risk that should be explicitly opted into (we can validate/invalidate that presumption separately).

Is it correct then to say that this patch exposes us to that risk on a default windows system (use system settings - default windows settings) where current Firefox does not have that exposure? But its also correct to say that the new risk is what IE and Chrome are already experiencing in their default windows configs?

Comment 27

[Masatoshi Kimura [:emk]]

(In reply to Patrick McManus [:mcmanus] from comment #26)
> Is it correct then to say that this patch exposes us to that risk on a
> default windows system (use system settings - default windows settings)
> where current Firefox does not have that exposure?

Correct. It is the very purpose of this bug to follow the default Windows settings.

> But its also correct to
> say that the new risk is what IE and Chrome are already experiencing in
> their default windows configs?

I've just confirmed IE and Chrome use the proxy indicated by WPAD by default.
Microsoft have even released a security update to mitigate the risk.
http://technet.microsoft.com/en-us/security/bulletin/ms09-008

Comment 28

[Patrick McManus [:mcmanus]]

Comment on attachment 707903 [details] [diff] [review]
Return the WPAD_URL if PROXY_TYPE_AUTO_DETECT and no other URL.

r=me but checkin should be dependent on a sec-review to see how they feel about the changes.

Comment 29

[Daniel Veditz [:dveditz]]

The whole premise of this bug is wrong. Use the System Proxy settings means use a proxy set up on that system. WPAD lives on the network, not the system, and that is what the "Auto-detect proxy settings for THIS NETWORK" (emphasis mine) is for.

Automatically grabbing some random WPAD file is full of fail and has been exploited for years. Yes, this is annoying for enterprises who generally know how to set up WPAD safely (keep it in the network!), but the vast majority of our users are NOT in corporate environments and http://wpad/wpad.dat could be anything and anywhere.

There's a bug open that we don't correctly discover WPAD from DHCP (when in auto-detect mode!), maybe that's the bug you want? See bug 356831

Security issues with WPAD are not obscure and easily exploited. Metasploit has a module for the script kiddies
http://www.netresec.com/?page=Blog&month=2012-07&post=WPAD-Man-in-the-Middle

Researchers have found over a hundred wpad.TLD domains registered, and 60+ return sometimes suspicious stuff for wpad.dat
http://allievi.sssup.it/techblog/?p=142

(In reply to Masatoshi Kimura [:emk] from comment #27)
> Microsoft have even released a security update to mitigate the risk.
> http://technet.microsoft.com/en-us/security/bulletin/ms09-008

That was a server fix, which helps people who manage corporate networks. Then the employees take their WPAD-searching laptops to Starbucks and get Pwned anyway.

If the result of this patch is that someone brings home a new Windows laptop, installs Firefox, and without making any changes to Windows or Firefox default settings it pings http://wpad/wpad.dat then this patch fails security review. Somewhere an administrator needs to have opted the machine or Firefox into WPAD as a conscious setting change either locally or pushed via remote enterprise config software.

Comment 30

[Masatoshi Kimura [:emk]]

-> WONTFIX per sec-review comment. Firefox doesn't consider WPAD settings as a part of "System Proxy settings".

Comment 31

[fukawi2]

This is not an acceptable answer; the option in FF is clearly suggesting that whatever the system settings are, they will be honored. If you are refusing to make it behave this way, then at minimum you need to alter the option so there is not the expectation that it will work this way.

Is the next step to refuse to honor WPAD URL's that aren't inside the clients configured domain name? You either honor the setting wholey, honestly and completely, or remove the option.

As a side note, despite the security issues you've raised, WPAD is a legitimate and very useful procedure. Certainly in our environment, we can NOT explicitly set a WPAD URL for our laptop users since they are unable to access internal servers when outside the office.

Not fixing this BUG is leaving FF in a broken state for businesses, and leaving misleading configuration options within FF.

Comment 32

[Daniel Veditz [:dveditz]]

There is a Firefox setting, right next to the current default, that says "Auto-detect proxy settings for this network". That does exactly what it says and is what you're asking for. If the user sees that his Firefox is NOT set to "Auto-detect" and yet it auto-detects anyway I think they'd be surprised.

There is apparently a bug in discovering settings from DHCP rather than DNS but that is a different issue (bug 356831).

Comment 33

[Rand Dow [:randix]]

Having just been through the entire relevant code in depth, it should be noted that setting the Firefox proxy setting to "Autodetect proxy settings for this network" causes the exact same behavior: the http://wpad/wpad.dat is searched in the current local domain. This change would simply cause the system settings to propagate, which they do for everything else, but autodetect. Firefox on OSX also behaves this way, this patch would bring the Windows Firefox in-line with the other platforms, and in-line with at least IE, Chrome, and Safari. (The system settings do propagate on the other platforms and browsers.)

Either this bug should be fixed, or the UI changed and make it clear to users that Mozilla considers the implementation of RFC 3040, section 6.4, http://www.ietf.org/rfc/rfc3040.txt , to be too risky. And we should remove the implementation for the other platforms.

Comment 34

[Patrick McManus [:mcmanus]]

Hey Dan, I'm pretty confident there is no confusion about which bug # is being addressed here - which is not to say every one is in favor of the approach.

As you can see from the comment trail, I basically agree with your assessment - however I think comment 31 does a pretty good job of representing the counter argument - the firefox definition of "use system settings" works differently in this case than other applications on the system which leads to surprises, frustrations, and probably the occasional lost user who exists in a legitimate wpad environment.

Can you think of any middle ground? I know the history of "allow this potentially dangerous operation, OK?" dialogs is poor.. but is there anything else or is a unknown-ca like click through sufficient given chrome and ie behavior?

Comment 35

[Rand Dow [:randix]]

(In reply to Daniel Veditz [:dveditz] from comment #29)
> If the result of this patch is that someone brings home a new Windows
> laptop, installs Firefox, and without making any changes to Windows or
> Firefox default settings it pings http://wpad/wpad.dat then this patch fails
> security review. Somewhere an administrator needs to have opted the machine
> or Firefox into WPAD as a conscious setting change either locally or pushed
> via remote enterprise config software.

The default of Firefox upon install is "No proxy", thus, someone would indeed have to make a conscious setting change. It is extremely unlikely that anyone who currently uses WPAD will have the "Use system settings" button set, since it doesn't currently work on Windows. Thus, it is unlikely that anyone would get bitten by an update, either.

Comment 36

[Patrick McManus [:mcmanus]]

(In reply to Rand Dow [:randix] from comment #35)

> The default of Firefox upon install is "No proxy", thus, someone would

I'm fairly certain this is incorrect. I just tried a fresh profile and it is set to "use system settings" which was my understanding.

Comment 37

[Rand Dow [:randix]]

I'm not sure about the default upon install. I was relying on memory from my many fresh installs. 

If it is "use system settings" as default, is that good? I would prefer "no proxy" forcing initial installations to visit the settings if they must.

Comment 38

[Daniel Veditz [:dveditz]]

(In reply to Rand Dow [:randix] from comment #35)
> The default of Firefox upon install is "No proxy", thus, someone would
> indeed have to make a conscious setting change.

Not according to https://mxr.mozilla.org/mozilla-central/source/modules/libpref/src/init/all.js#1257 or any of my local copies of Firefox that I've tried. All have network.proxy.type "5" (Use system settings).

(In reply to Rand Dow [:randix] from comment #33)
> setting the Firefox proxy setting to "Autodetect proxy settings
> for this network" causes the exact same behavior: the http://wpad/wpad.dat
> is searched in the current local domain.

Yes. If the user chooses "Autodetect" we should honor that. Hopefully they know what they're doing.

> Firefox on OSX also behaves this way, this patch would bring the
> Windows Firefox in-line with the other platforms,

Nice... I wonder how many of us were pwned this way at the last BlackHat.

> The system settings do propagate on the other
> platforms and browsers.

Is the default system setting to search for wpad on the network? If not then I'm OK with that-- only users who have opted into a potentially dangerous configuration are at risk. I'll check my mac when I get home.

> Either this bug should be fixed, or the UI changed and make it clear to
> users that Mozilla considers the implementation of RFC 3040, section 6.4,
> http://www.ietf.org/rfc/rfc3040.txt, to be too risky.

Take any RFC from 11 years ago with a huge grain of salt -- we've learned a lot since then and many specs from that era have been obsoleted. RFC 3040 itself is not a spec, it's a collection of references. The closest to a WPAD spec we have is http://tools.ietf.org/html/draft-ietf-wrec-wpad-01 which was an informational work-in-progress from 1999, but the working group disbanded without ever seeing it to official adoption. de facto it's what we've got to work with, but with a history like that you can't just implement it the way it's written just because it's written down somewhere.

In any case I never said WPAD was too risky to implement. I'm saying it's too risky to have as the default behavior, a call we've made in the past (bug 310331, bug 421490).

I'm also concerned that if we're hardcoding "http://wpad/wpad.dat" then we're not really obeying the system settings, because IE might be getting the URL from DHCP and it might not be that. It's a good, standard, guess--but just a guess. Is there any way to actually query the system rather than replicate the auto-discovery on our own?

Comment 39

[Rand Dow [:randix]]

Very good comments, thanks!

(In reply to Daniel Veditz [:dveditz] from comment #38)
> Nice... I wonder how many of us were pwned this way at the last BlackHat.

Good point.

> Is the default system setting to search for wpad on the network? If not then
> I'm OK with that-- only users who have opted into a potentially dangerous
> configuration are at risk. I'll check my mac when I get home.

Not as far as I know. I have myself never seen WPAD used in the wild, and I spent many years consulting for many large companies in Europe.

> Take any RFC from 11 years ago with a huge grain of salt ....
> 
> In any case I never said WPAD was too risky to implement. I'm saying it's
> too risky to have as the default behavior, a call we've made in the past
> (bug 310331, bug 421490).

Agreed. And as Patrick pointed out, IF firefox is installed on a system that is configured for WPAD usage, we would get that then by our default. However, as pointed out above, that might be wished for. This is a tough call. We don't do the DHCP auto detect (as you pointed out), and I don't see how to tell if Windows is doing DNS or DHCP autodetect, when it does so.

> I'm also concerned that if we're hardcoding "http://wpad/wpad.dat" then
> we're not really obeying the system settings, because IE might be getting
> the URL from DHCP and it might not be that. It's a good, standard,
> guess--but just a guess. Is there any way to actually query the system
> rather than replicate the auto-discovery on our own?

I looked into that, but couldn't find any way to do so. 

So my only concern here is that we should become consistent across platforms. Your call on which way to go.

Comment 40

[Masatoshi Kimura [:emk]]

(In reply to Rand Dow [:randix] from comment #39)
> > Is the default system setting to search for wpad on the network? If not then
> > I'm OK with that-- only users who have opted into a potentially dangerous
> > configuration are at risk. I'll check my mac when I get home.
> 
> Not as far as I know.

It contradicts my observation.

> I have myself never seen WPAD used in the wild, and I
> spent many years consulting for many large companies in Europe.

Windows searches non-existing WPAD server by default even if WPAD is not used in the network. It's the problem.

Comment 41

[Masatoshi Kimura [:emk]]

(In reply to Daniel Veditz [:dveditz] from comment #38)
> I'm also concerned that if we're hardcoding "http://wpad/wpad.dat" then
> we're not really obeying the system settings, because IE might be getting
> the URL from DHCP and it might not be that. It's a good, standard,
> guess--but just a guess. Is there any way to actually query the system
> rather than replicate the auto-discovery on our own?

I found WinHttpDetectAutoProxyConfigUrl function.
WinHttpGetIEProxyConfigForCurrentUser will also be useful which retrieves IE settings without loading wininet.

Comment 42

[Patrick McManus [:mcmanus]]

Dan, any ideas on paragraph 3 of comment 34?

The fact that we have different behavior than pretty much all other applications that honor the system settings on windows is a problem of some sort.

Rand, you say that OSX already works this way (honors wpad).. but isn't it true that WPAD is not part of the default system settings on OSX the way it is on windows? What Dan and I are worried about here is that with no config from the admin we suddenly start doing WPAD which is easily spoofed..

Comment 43

[Rand Dow [:randix]]

Patrick, I have turned on the OSX System Preferences->Network->Advanced, Proxies tab,  "Auto Proxy Discovery". This turns on searching for a "wpad.dat" file. My DNS server includes in /etc/hosts:

192.168.1.21	wpad.localdomain wpad

(192.168.1.21 is running an apache with a file "wpad.dat" in the Documents directory.)

Immediately, these entries start appearing in /var/log/apache2/access_log:

192.168.1.21 - - [06/Feb/2013:20:39:33 -0800] "GET /wpad.dat HTTP/1.1" 200 8394
192.168.1.21 - - [07/Feb/2013:09:36:01 -0800] "GET /wpad.dat HTTP/1.1" 200 8394

and when I turn off the "Auto Proxy Discovery" that stops.

Thus, I conclude that, yes, OSX honors wpad in the good old RFC way. Perhaps unfortunately. I certainly agree with the worries -- I have two points: 1) we should be consistent across platforms (perhaps, it is probably only Windows that ever uses WPAD auto discovery -- anyone agree, or say otherwise?) and 2) we should certainly document (in the settings UI) that we do things differently, if we are going to. I agree that it is safer to not use wpad. (As an aside, I worked for many, many large companies in Europe, none of them used WPAD discovery, they *all* used a proxy.pac URL.) But this bug is an indication that at least one person wishes we'd do this differently.

I have worked on a fix, as suggested by Masatoshi (Comment 41), and I think it would work. But I am not familiar enough with the windows build system to get it to work. If someone could pick it up and integrate it, we might have an acceptable and safe fix. Here is my proposal: (I am not adding it as a patch, if someone wants to integrate, just cut and paste from here.)

Two additional files (to avoid include conflicts -- including wininet.h and winhttp.h conflicts for me):

1) nsWinHttpDetectAutoProxyConfigUrlWrapper.h
---------------------------
/* -*- Mode: C++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
/* This Source Code Form is subject to the terms of the Mozilla Public
 * License, v. 2.0. If a copy of the MPL was not distributed with this
 * file, You can obtain one at http://mozilla.org/MPL/2.0/. */


nsresult WinHttpDetectAutoProxyConfigUrlWrapper(nsAString& aValue);
---------------------------

2) nsWinHttpDetectAutoProxyConfigUrlWrapper.cpp
---------------------------
/* -*- Mode: C++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
/* This Source Code Form is subject to the terms of the Mozilla Public
 * License, v. 2.0. If a copy of the MPL was not distributed with this
 * file, You can obtain one at http://mozilla.org/MPL/2.0/. */

#include <windows.h>
#include <ras.h>
#include <winhttp.h>

#include "mozilla/Attributes.h"
#include "mozilla/Util.h"
#include "nsISystemProxySettings.h"
#include "nsIServiceManager.h"
#include "mozilla/ModuleUtils.h"
#include "nsPrintfCString.h"
#include "nsNetUtil.h"
#include "nsISupportsPrimitives.h"
#include "nsIURI.h"

#include "nsWinHttpDetectAutoProxyConfigUrlWrapper.h"

nsresult WinHttpDetectAutoProxyConfigUrlWrapper(nsAString& aValue)
{
    LPWSTR *ppwszAutoConfigUrl;
        
    if (!WinHttpDetectAutoProxyConfigUrl(WINHTTP_AUTO_DETECT_TYPE_DHCP | WINHTTP_AUTO_DETECT_TYPE_DNS_A, 
                                         ppwszAutoConfigUrl)) {

        if (GetLastError() == ERROR_WINHTTP_AUTODETECTION_FAILED) {

fprintf(stderr, "randix: AutoDetectionFailed\n");
        } else {
fprintf(stderr, "randix: internal error or memory\n");

        }

        return NS_ERROR_FAILURE;
    }

    aValue.Assign((PRUnichar *)(ppwszAutoConfigUrl));
    GlobalFree(ppwszAutoConfigUrl);

    return NS_OK;
}
-----------------------------------

and 3) possible patch to other files, included in the next comment.

Comment 44

[Rand Dow [:randix]]

Attachment: A possible patch.

This completes the patch proposed in the above Comment.
I need help from a Windows firefox expert to complete this.

Comment 45

[Patrick McManus [:mcmanus]]

(In reply to Rand Dow [:randix] from comment #43)
> Patrick, I have turned on the OSX System Preferences->Network->Advanced,
> Proxies tab,  "Auto Proxy Discovery". This turns on searching for a
> "wpad.dat" file.

I don't know any other way to say it differently, so I'll say the same thing again: this is OK because that is not default behavior. The Admin turned on that feature (in your case via OS X System Prefs->network->etc..) and obviously wants to use wpad. Nobody got owned at blackhat on os x because of this because nobody was silly enough to configure the OS to do wpad..

but according to the comment trail here the Windows difference is that it simply could do WPAD without the admin doing anything to explicitly turn it on. That's how it comes out of the box. The difference is huge from a security pov. OTOH other applications seem to honor that as a default, so doing something different is a usability problem.

I'm hoping a sec-review will suggest some alternate path. (a doorhanger or something for example)

Comment 46

[Masatoshi Kimura [:emk]]

Comment on attachment 711450 [details] [diff] [review]
A possible patch.

>+#include "nsWinHttpDetectAutoProxyConfigUrlWrapper.h"
Did you forget hg add?

Unfortunately WinHttp was not better than WinInet, according to Chromium developers.
https://code.google.com/p/chromium/issues/detail?id=12189

Comment 47

[Daniel Veditz [:dveditz]]

(In reply to Patrick McManus [:mcmanus] from comment #34)
> Can you think of any middle ground?

I can't. As long as the default windows action includes WPAD we can't safely include that in "use system settings" on windows. We can't really annoy users by asking in any sensible way -- users who have and want WPAD will be annoyed at the constant (daily?) requests, and casual home users won't know what the dialog means if they ever do get it in an attack situation.

Can we somehow make use of the Windows concept of what kind of network you're on, "home", "work", or "public"? WPAD for a home or public network isn't safe (I'm sure there's some geek somewhere who uses wpad at home, but that guy is capable of changing the existing auto-detect pref); trying WPAD only on a "work" network limits the potential for attacks.

Comment 48

[Patrick McManus [:mcmanus]]

(In reply to Daniel Veditz [:dveditz] from comment #47)
> (In reply to Patrick McManus [:mcmanus] from comment #34)
> > Can you think of any middle ground?
> 
> I can't. 

[..]
> network you're on, "home", "work", or "public"? 

you say you can't, then you make a great suggestion! :)

Rand, what do you think about doing that?

Comment 49

[Rand Dow [:randix]]

(In reply to Patrick McManus [:mcmanus] from comment #48)
> [..]
> > network you're on, "home", "work", or "public"? 
> [..] 
> Rand, what do you think about doing that?

IIRC Windows does not automatically reconfigure between "home", "work" and "public". These settings basically set default firewall rules, and are only changed manually. This needs to be checked.

I see four possible resolutions for this bug:

1) Change the default Network Settings to NOT be "Use system default." IMO we SHOULD NOT change that.
2) Take the security risk of implementing my patch. IMO (now) we SHOULD NOT do that; in the sense of "improving the web" WPAD should probably just die. As noted, AFAIK, most big companies use Manual Proxy Configuration.
3) Accept this current implementation. We should be aware (and document) that the platforms behave differently (OSX and Linux require user settings changes to work, but then they do work, Windows Firefox cannot be made to work currently with the Auto Proxy Discovery setting. I presume that the mobile platform also cannot be coerced to do auto-proxy discovery.) IMO we SHOULD document that behavior in the Windows Settings UI. See Bug 841046.
4) Close with WONTFIX. IMO we SHOULD NOT do that.

Comment 50

[Mike]

The default of 'Use System Proxy Settings' fails for me here whilst using a transparent proxy (shorewall/squid/squidguard/dansguardian) meaning I would have to manually change Firefox on every pc for every user (roaming profiles) so it's bye bye firefox, hello IE. Bummer!

All the systems' settings are actually set to 'Use No Proxy' but that makes no difference.

Comment 51

[Hale]

The same problem. Tried versions 21-24. FF just doesn't try to autodetect when autodetection is set in Windows.

Why don't it use "WinHTTP Web Proxy Auto-Discovery Service", which is designed specially for that case? Ignoring this major problem for more than 4 years is ridiculous!

Comment 52

[Hale]

(In reply to Mike from comment #50)
> The default of 'Use System Proxy Settings' fails for me here whilst using a
> transparent proxy (shorewall/squid/squidguard/dansguardian) meaning I would
> have to manually change Firefox on every pc for every user (roaming
> profiles) so it's bye bye firefox, hello IE. Bummer!
> 
> All the systems' settings are actually set to 'Use No Proxy' but that makes
> no difference.

lately, FireFox got so many ignored bugs that raising version numbers looks like a bad joke from developers. They ignore Flash compatibility problems, (Even Youtube often crashes or losses TCP connection), they ignore rudimentary tab functionality that is useless without tons of plugins. There's no text/image accelerator. Printing doesn't give correct printouts. And finally, network part of the engine is a total mess.

Comment 53

[Masatoshi Kimura [:emk]]

(In reply to Hale from comment #51)
> Why don't it use "WinHTTP Web Proxy Auto-Discovery Service", which is
> designed specially for that case? Ignoring this major problem for more than
> 4 years is ridiculous!

Chromium team already tried WinHTTP, but the result was not better than the WinInet one, so they backed to use WinInet.

Comment 54

[Hale]

(In reply to Masatoshi Kimura [:emk] from comment #53)
> Chromium team already tried WinHTTP, but the result was not better than the
> WinInet one, so they backed to use WinInet.

Does this mean WinHTTP doesn't work, or Chromium team doesn't work?

Comment 55

[Hale]

(In reply to Rand Dow [:randix] from comment #49)
> (In reply to Patrick McManus [:mcmanus] from comment #48)
>  AFAIK, most big companies use Manual Proxy Configuration.

Hmm, yes. They do this. But with Internet Explorer via AD policies. ;-)
Do you suggest to close the Mozilla project ? Just because IE can be configured and FF is just a pain in the ***?

Comment 56

[Michal Bruncko]

Just found this bug and it is very annoying. As per my understanding "Use system proxy settings" in Firefox Network settings does not do what it wanted to do. I found this problem in my environment and wondering where the hell could be the problem... and now I found this bug report.

It is really too hard to make another condition for "Use system proxy settings" that
 if ("ProxyEnable==0" AND ( not ProxyServer or ProxyServer=="") AND AutoConfigProxy=="wininet.dll") then follow same behavior like with "Auto-Detect Proxy settings for this Network"?

Currently I have to run vbs script as Log-On script which changes Proxy settings for existing Firefox user profile in "prefs.js".

Thanks for more attention on this problem.

Comment 57

[Michal Bruncko]

I have one question about WPAD discovery. Why are you still talking about "http://wpad/wpad.dat". AFAIK windows are using DNS discovery only in case that:
- primary DNS suffix exists (this is available if to windows clients is a part of AD, or manually configured), or
- connection-specific DNS suffix exists (available using DHCP)
If there is no any mentioned DNS suffix available, the WPAD discovery fails - http://technet.microsoft.com/en-us/library/cc713344.aspx - section DNS .

So in summary I am suggesting:
- implement WPAD discovery within "Use system proxy settings" if this configured in System settings as well, BUT
- use it only in case, that some of DNS prefixes are available in system - like described in that technet article.

Comment 58

[Vitaly]

Same problem with Windows 7 x86 and Firefox 36.0.1
All other browsers use Internet Explorer Auto configure checkbox, Firefox is not.

Comment 59

[Raul Valencia Thomas]

Same problem with Windows 7 Professional 64-bit, and Firefox 40.0.2

Other browsers like Internet Explorer or Chrone use Internet Options -> Connections tab -> LAN settings -> Use a proxy server for your LAN and the Bypass proxy server box for local address and works properly. 

In Firefox, works the proxy but doesn't works with the Bypass proxy server checkbox activate in the OS and you need to config all the URL manually: Tools -> Options -> Advanced -> Network tab -> Settings and adds the "No Proxy for:"

That's is not the way is you need to configure a Group policy.

Comment 60

[u550515]

It appears that the developers are not willing to go further. I respect developers' seriousness about security. However, why doesn't Firefox include an option under "Use system proxy settings" to allow taking IE/system auto detect setting? This option can be disabled by default, thus still requires manual change, meaning no new risk. Though manual change doesn't satisfy some administrators, it's an acceptable compromise to honor all system proxy settings which in many cases is the only way to work.

Comment 61

[Pablo Varela]

Very annoying. At least they could add a setting so "auto dettect proxy settings" can be the default for every new profile, so you don't get an annoying call every time a new user wants to use firefox.

Comment 62

[Will Manley]

Can we get some sort of agreed resolution for the Firefox community. For this issue it would make sense for "Auto detect proxy settings" to be the default on a new install. Not only that if the "Use system proxy settings" has been set then they should be inheriting whatever has been setup within IE which is not working currently. Either way a developer needs to spend some time on this to fix it.

Could we please get this fixed up once and for all on this particular issue? At least it would stop users from getting annoyed with this bug and moving to another browser.

Comment 63

[Will Manley]

Without being disrespectful to the hard working developers who provide their free time, has anything been discussed about getting this resolved since my last post? This is a browser fundamental.

Comment 64

[Daniele "Mte90" Scasciafratte]

any news for that ticket?

Comment 65

[Firefox Bug Husbandry Bot]

Bulk change to priority: https://bugzilla.mozilla.org/show_bug.cgi?id=1399258

Comment 66

[Jason Straight]

I just spent the better part of a day trying to get wpad to work. Resorted to running wireshark and sniffing to realize that FF is not even looking for wpad when "use system settings" is chosen (Both Linux and Windows).

I admin for a few schools who require a filtering proxy and this bug means BYOD's that run FF won't work on the network, as 80 and 443 are blocked.

Other browsers work as expected on such networks, I think, after 7 years, something should be done with FF on this issue. Especially since I expect its popularity to be on the climb soon.

If FF is expected to "just work" then its default needs to be set to auto configure.

Maybe a warning to the user on startup, or first page, that notifies them a proxy has been auto configured, with an option to never show this warning again for this network. It could match the network by ip, domain, gateway, proxy server, wpad file, etc.

Comment 67

[Ju]

Hi,

I'm experiencing this bug too, and am quite happy to find thi bug because it explains why I'm having os much problems configuring proxy settings...

The problem is still present with recent versions of Firefox (50+), this bug is years old, isn't there any way to fix this ? 
If it can't, maybe put a small warning under the "uses system proxy settings" notifying it doesn't work with auto-detect ?

Having the possibility to use the system settings seems like a great idea, but is finally disappointing... and I have to resort to configure both the system AND firefox.

Comment 69

[Bala]

[Tracking Requested - why for this release]:

Release Note Request (optional, but appreciated)
[Why is this notable]:
[Affects Firefox for Android]:
[Suggested wording]:
[Links (documentation, blog post, etc)]:

Comment 70

[Bala]

"Use system proxy settings" assumes no proxy if IE is set to auto-detect itself. Please help in fixing the issue and it occurred recently. I need to set auto proxy in firefox to have internet connection.

Comment 71

[Jason]

(In reply to Bala from comment #70)

"Use system proxy settings" assumes no proxy if IE is set to auto-detect itself. Please help in fixing the issue and it occurred recently. I need to set auto proxy in firefox to have internet connection.

You can set the auto detect proxy in firefox, it just will not honor the system-wide setting to do so. I've read that it's a "won't fix" issue, because of security implications, which I agree with. However, I wish they would add something to that line that notifies people that "use system settings" does not work with auto-detect, only with manually entered proxy settings.

Comment 73

[BugBot [:suhaib / :marco/ :calixte]]

The bug assignee didn't login in Bugzilla in the last 7 months.
:dragana, could you have a look please?
For more information, please visit auto_nag documentation.

Comment 74

[zapy85]

Any news about this?
My company is using the admx files and have set the option " use system default " but it does not seem to auto-detect it when using a online PAC.
However it does default back to a local PAC file when I'm not at the office and FF then starts working again.

Comment 75

[BugBot [:suhaib / :marco/ :calixte]]

The severity field for this bug is relatively low, S3. However, the bug has 4 duplicates and 14 votes.
:kershaw, could you consider increasing the bug severity?

For more information, please visit auto_nag documentation.

Comment 76

[BugBot (nomail) [:suhaib / :marco/ :calixte]]

The last needinfo from me was triggered in error by recent activity on the bug. I'm clearing the needinfo since this is a very old bug and I don't know if it's still relevant.

Comment 77

[Randell Jesup [:jesup] (needinfo me)]

Dan, Valentin - We should make a final determination about what our position on this issue is. I see the options as:

• WONTFIX (due to security issues, leave everything as-is)
• Automatic use of WPAD in this case (likely no due to security)
• Additional preferences option
• Modify description in the UI of the current options
• something else?
  Has there been any change in what Chrome/Edge do with regards to WPAD and their UI/options?

Comment 78

[Valentin Gosu [:valentin] (he/him)]

IMO we've had the behaviour of only using WPAD (autodetect) when specifically instructed to do so for a long while, so changing that behaviour is potentially a security issue.
I think the way forward is either to wontfix or to add an additional checkbox whether to use wpad when system settings are set up so (default to false).

Comment 79

[zapy85]

(In reply to zapy85 from comment #74)

Any news about this?
My company is using the admx files and have set the option " use system default " but it does not seem to auto-detect it when using a online PAC.
However it does default back to a local PAC file when I'm not at the office and FF then starts working again.

Ill just reply to myself here and add that my issue were maybe not at all the auto-detect feature, however i cant verify it as we also found out that my work was trying out ipv6 and we had some issues with that as well when it came to the proxy scripts.

https://bugzilla.mozilla.org/show_bug.cgi?id=558253
And that is something i think FF dev should consider and implementing as ipv6 grows.

About the original issue i could just agree with everyone that lifts the security concerns and ask of FF to add more info to the settings available in the UI for better understanding (as i guess its explained in the manual a bit more detailed) and add the checkbox maybe hidden in about:config for those that want to use it.

Comment 80

[Daniel Veditz [:dveditz]]

+1 to what Valentin said: I think we need an extra pref. We would also want to expose that pref through to Group Policy.

Comment 81

[Randell Jesup [:jesup] (needinfo me)]

Ok, so it sounds like the best way forward is a preference which defaults to false. This will need front-end changes and strings, and as Dan says it needs to be exposed to Group Policy for enterprise users. FYI mkaply

Comment 82

[Mike Kaply [:mkaply]]

Should be straight forward to update policy once it's done. I'll keep an eye out.

Comment 83

[Valentin Gosu [:valentin] (he/him)]

This may be a good project for Dylan.

Comment 84

[Dylan]

Attachment: Bug 621429 - Added button to allow WPAD while using system proxy settings. r=kershaw

Comment 85

[BMO Automation]

Moving bug to Core/Networking: Proxy.

Comment 86

[Dylan]

Attachment: Bug 621429 - added check to only show the Use system Web Proxy Auto-Discovery setting checkbox on windows. r=kershaw

Depends on D183429

Comment 87

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/57b8feadc072
Added button to allow WPAD while using system proxy settings. r=necko-reviewers,fluent-reviewers,settings-reviewers,kershaw,bolsson,Gijs
https://hg.mozilla.org/integration/autoland/rev/efe1984e433c
added check to only show the Use system Web Proxy Auto-Discovery setting checkbox on windows. r=kershaw,settings-reviewers,mossop

Comment 88

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/mozilla-central/rev/57b8feadc072
https://hg.mozilla.org/mozilla-central/rev/efe1984e433c

Comment 89

[Ryan VanderMeulen [:RyanVM]]

Do we want to call this out in the Fx125 relnotes? Please add a relnote-firefox nomination if so.

Comment 90

[Dylan]

Release Note Request (optional, but appreciated)
[Why is this notable]: Users can now use WPAD while configured to system proxy settings.
[Affects Firefox for Android]: No
[Suggested wording]: Added proxy feature to use WPAD while configured to system proxy settings.
[Links (documentation, blog post, etc)]:

Comment 91

[Ryan VanderMeulen [:RyanVM]]

Added to the Fx125 relnotes.

Firefox now provides an option to enable Web Proxy Auto-Discovery (WPAD) while configured to use system proxy settings.