1028195 - Windows Proxy Setting "Bypass proxy server for local addresses" is ignored by Firefox when using "Use system proxy settings"

Status: RESOLVED FIXED

(Core :: Networking: HTTP, defect)

Description

[SomeNerd]

User Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; Touch; .NET4.0E; .NET4.0C; Tablet PC 2.0; .NET CLR 3.5.30729; .NET CLR 2.0.50727; .NET CLR 3.0.30729; BRI/2; Zune 4.7; InfoPath.3; rv:11.0) like Gecko

Steps to reproduce:

In Windows I've activated in Internet Options, Connections, LAN settings "Bypass proxy server for local addresses" which adds "<local>" at the end of the registry key "HKEY_CURRENT_USER\SOFTWARE\MICROSOFT\WINDOWS\CURRENTVERSION\Internet Settings\ProxyOverride"
Then I activated in Firefox Options, Advanced, Network, Connection Settings "Use system proxy settings"
(see actual / expected results) 
---
Important:
Firefox itself knows <local> if you use "Manual proxy configuration" see https://bugzilla.mozilla.org/show_bug.cgi?id=44980#c30 and https://developer.mozilla.org/en-US/docs/Proxies_in_Necko#Proxies_and_local_hosts 
but it's ignored if it's in the OS list of proxy exceptions rather than the FF own exception list.


Actual results:

non-fqdn websites like http://intranet or http://server are forwarded to the proxy


Expected results:

non-fqdn websites like http://intranet or http://server are not forwarded to the proxy

Comment 1

[Matthias Versen [:Matti]]

I found only bug 72444

Comment 2

[Jon]

I have encountered this issue for years and can confirm it happens for me at work. If I use the "use system proxy settings" option, non-fqdn intranet pages go to the proxy even if the Microsoft option to bypass the proxy for these sites is enabled. If I change Firefox to manual proxy settings and convert the Microsoft proxy bypass list from the semicolon-separated, * wildcard version into the comma-separated CIDR one that Firefox prefers, I can add <local> to the list to get the desired functionality (see bug 72444). 

If the "Use system proxy settings" option were changed to use the <local> functionality when the equivalent Microsoft setting is enabled, it should resolve this bug.

Comment 3

[Gary Chen [:xeonchen]]

(In reply to Jon from comment #2)
> I have encountered this issue for years and can confirm it happens for me at
> work. If I use the "use system proxy settings" option, non-fqdn intranet
> pages go to the proxy even if the Microsoft option to bypass the proxy for
> these sites is enabled. If I change Firefox to manual proxy settings and
> convert the Microsoft proxy bypass list from the semicolon-separated, *
> wildcard version into the comma-separated CIDR one that Firefox prefers, I
> can add <local> to the list to get the desired functionality (see bug
> 72444). 
> 
> If the "Use system proxy settings" option were changed to use the <local>
> functionality when the equivalent Microsoft setting is enabled, it should
> resolve this bug.

Thanks for the comment, Jon, very helpful!

Looks like we have some inconsistent here:
[1] thinks "<local>" is for localhost/127.0.0.1, but [2] checks for non-IP, non-dotted hosts.

Daniel, do you think we should take the solution from [2] and maybe add support for LAN IPs?

[1] https://dxr.mozilla.org/mozilla-central/rev/af8a2573d0f1e9cc6f2ba0ab67d7a702a197f177/toolkit/system/windowsproxy/nsWindowsSystemProxySettings.cpp#137
[2] https://dxr.mozilla.org/mozilla-central/rev/af8a2573d0f1e9cc6f2ba0ab67d7a702a197f177/netwerk/base/nsProtocolProxyService.cpp#729

Comment 4

[Daniel Stenberg [:bagder]]

I think both code paths should use the setting the same way as the current differences are more confusing than anything else.

Isn't the missing part here only that the [2] case needs to also say that host names without dots are to be considered to be local (in addition to "localhost" and "127.0.0.1")?

Maybe [::1] should be respected too while at it?

Comment 5

[Jon]

(In reply to Gary Chen [:xeonchen] from comment #3)
> (In reply to Jon from comment #2)
> > I have encountered this issue for years and can confirm it happens for me at
> > work. If I use the "use system proxy settings" option, non-fqdn intranet
> > pages go to the proxy even if the Microsoft option to bypass the proxy for
> > these sites is enabled. If I change Firefox to manual proxy settings and
> > convert the Microsoft proxy bypass list from the semicolon-separated, *
> > wildcard version into the comma-separated CIDR one that Firefox prefers, I
> > can add <local> to the list to get the desired functionality (see bug
> > 72444). 
> > 
> > If the "Use system proxy settings" option were changed to use the <local>
> > functionality when the equivalent Microsoft setting is enabled, it should
> > resolve this bug.
> 
> Thanks for the comment, Jon, very helpful!
> 
> Looks like we have some inconsistent here:
> [1] thinks "<local>" is for localhost/127.0.0.1, but [2] checks for non-IP,
> non-dotted hosts.
> 
> Daniel, do you think we should take the solution from [2] and maybe add
> support for LAN IPs?
> 
> [1]
> https://dxr.mozilla.org/mozilla-central/rev/
> af8a2573d0f1e9cc6f2ba0ab67d7a702a197f177/toolkit/system/windowsproxy/
> nsWindowsSystemProxySettings.cpp#137
> [2]
> https://dxr.mozilla.org/mozilla-central/rev/
> af8a2573d0f1e9cc6f2ba0ab67d7a702a197f177/netwerk/base/nsProtocolProxyService.
> cpp#729

I believe the comment in [1] (and associated logic) is incorrect while [2] is valid, at least with respect to how the relevant Windows setting works. The Windows proxy override list is stored in the following registry value: HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows\CurrentVersion\Internet Settings\ProxyOverride
If you check the "Bypass proxy server for local addresses" box in Internet Options, it appends "<local>" to this registry value. So, <local> is not meant to signify only localhost. The relevant if statement in [1] (line 152) should probably return true for any host which does not contain a period as well, similar to [2].

Comment 6

[Gary Chen [:xeonchen]]

Attachment: Bug 1028195 - make proxy bypass rule consistent;

Review commit: https://reviewboard.mozilla.org/r/110662/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/110662/

Comment 7

[Daniel Stenberg [:bagder]]

Comment on attachment 8834824 [details]
Bug 1028195 - make proxy bypass rule consistent;

https://reviewboard.mozilla.org/r/110662/#review111956

If you're fine with the little question about IPv6 addresses, then I'm all good!

::: netwerk/base/nsProtocolProxyService.cpp:731
(Diff revision 1)
>      }
>  
>      // Don't use proxy for local hosts (plain hostname, no dots)
> -    if (!is_ipaddr && mFilterLocalHosts && !host.Contains('.')) {
> +    if ((!is_ipaddr && mFilterLocalHosts && !host.Contains('.')) ||
> +        host.EqualsLiteral("127.0.0.1") ||
> +        host.EqualsLiteral("::1")) {

As long as you're *sure* the IPv6 version is compared this way, then I'm fine with it. I mean IPv6 literal addresses are written as '[address]' in URLs and I haven't checked if those brackets have been removed or not...

::: toolkit/system/windowsproxy/nsWindowsSystemProxySettings.cpp:161
(Diff revision 1)
> +                if (!isIpAddr && !host.Contains('.')) {
>                      return true;
> +                }
> +
> +                if (host.EqualsLiteral("127.0.0.1") ||
> +                    host.EqualsLiteral("::1")) {

The IPv6 comment from above goes for this as well.

Comment 8

[SomeNerd]

Guys, please keep in mind, that not only 127.0.0.1 but all addresses in 127.0.0.0/8 are loopback addreses (see https://en.wikipedia.org/wiki/Localhost#Name_resolution)

Comment 9

[Jon]

I think the ip address/loopback considerations are outside the scope of this Microsoft setting and the <local> functionality.  Really, the if statement at nsWindowsSystemProxySettings.cpp[152] should *only* be checking if host contains '.', and not even checking for loopback because the override value being checked there is the literal "<local>". So, if I understand it correctly, it will only land there if "<local>" exists in the override list, which is unrelated to loopback/localhost as discussed. I think this means the loopback check here is superfluous. Initially, I was being cautious by trying to suggest the minimum effective change.

I think nsProtocolProxyService.cpp[729] (referenced as [2] above) also handles this part correctly because it specifically ignores ip addresses for the <local> check with !is_ipaddr (it uses https://dxr.mozilla.org/mozilla-central/source/nsprpub/pr/src/misc/prnetdb.c#2217 which does check for ipv6 addresses as well)

Comment 10

[Jon]

I found some documentation from Microsoft supporting the idea that their setting only applies to host names and not ip addresses: https://support.microsoft.com/en-us/help/262981/internet-explorer-uses-proxy-server-for-local-ip-address-even-if-the-bypass-proxy-server-for-local-addresses-option-is-turned-on 
under "Cause"

Comment 11

[Gary Chen [:xeonchen]]

Comment on attachment 8834824 [details]
Bug 1028195 - make proxy bypass rule consistent;

https://reviewboard.mozilla.org/r/110662/#review112216

::: netwerk/base/nsProtocolProxyService.cpp:731
(Diff revision 1)
>      }
>  
>      // Don't use proxy for local hosts (plain hostname, no dots)
> -    if (!is_ipaddr && mFilterLocalHosts && !host.Contains('.')) {
> +    if ((!is_ipaddr && mFilterLocalHosts && !host.Contains('.')) ||
> +        host.EqualsLiteral("127.0.0.1") ||
> +        host.EqualsLiteral("::1")) {

Yes, in this function the brackets are removed.

Comment 12

[Gary Chen [:xeonchen]]

(In reply to Jon from comment #10)
> I found some documentation from Microsoft supporting the idea that their
> setting only applies to host names and not ip addresses:
> https://support.microsoft.com/en-us/help/262981/internet-explorer-uses-proxy-
> server-for-local-ip-address-even-if-the-bypass-proxy-server-for-local-
> addresses-option-is-turned-on 
> under "Cause"

I agree, so the case would be like the following table, right?

--------------------------------------
|            | localhost | 127.0.0.1 |
|------------+-----------+-----------|
|   bypass   |   direct  |   proxy   |
|------------+-----------+-----------|
| not bypass |   proxy   |   proxy   |
--------------------------------------

I'm just curious about how IE/Edge behave like of the given cases?

Comment 13

[Gary Chen [:xeonchen]]

Comment on attachment 8834824 [details]
Bug 1028195 - make proxy bypass rule consistent;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/110662/diff/1-2/

Comment 14

[Jon]

Actually, Microsoft always bypasses the proxy for localhost and 127.0.0.1 regardless of the other settings we've discussed. For example, if you have an empty bypass list, both localhost and 127.0.0.1 will still bypass the proxy (even if you do *not* have the 'bypass proxy for local addresses' box checked).

After some research though, I did find another case: adding <-loopback> to the bypass list for Windows overrides the default behavior and causes localhost and 127.0.0.1 to not bypass (i.e. they do go through the proxy). 
**Note that the default behavior and the <-loopback> setting only affect "localhost" and 127.0.0.1 specifically. Other loopback ip addresses go through the proxy unless explicitly added to the bypass list.

So, I believe and expanded version of your chart should look like this:
--------------------------------------------------------------------------------------------
|                  | localhost | 127.0.0.1 | 127.0.0.9 |  MyHost*  | MyHost.com| 10.0.0.5  |
|------------------+-----------+-----------+-----------|-----------+-----------+-----------|
|   bypass local   |   direct  |   direct  |   proxy   |  direct   |  proxy    |  proxy    |
|------------------+-----------+-----------+-----------+-----------+-----------+-----------|
| not bypass local |   direct  |   direct  |   proxy   |  proxy    |  proxy    |  proxy    |
--------------------------------------------------------------------------------------------
*Note that the only item that changes is "MyHost" because it is not an IP, is not loopback, and it does not contain '.'

And here is a similar chart showing the <-loopback> functionality:
--------------------------------------------------------
|                  | localhost | 127.0.0.1 | 127.0.0.9 | 
|------------------+-----------+-----------+-----------|
|   <-loopback>    |   proxy   |   proxy   |   proxy   | 
|------------------+-----------+-----------+-----------|
|  no <-loopback>  |   direct  |   direct  |   proxy   | 
--------------------------------------------------------

I tested all of these cases on Windows 7 using IE11.

Comment 15

[Gary Chen [:xeonchen]]

Comment on attachment 8834824 [details]
Bug 1028195 - make proxy bypass rule consistent;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/110662/diff/2-3/

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/48e6afc85032
make proxy bypass rule consistent; r=bagder

Comment 17

[Gary Chen [:xeonchen]]

(In reply to Jon from comment #14)
> Actually, Microsoft always bypasses the proxy for localhost and 127.0.0.1
> regardless of the other settings we've discussed. For example, if you have
> an empty bypass list, both localhost and 127.0.0.1 will still bypass the
> proxy (even if you do *not* have the 'bypass proxy for local addresses' box
> checked).
> 
> After some research though, I did find another case: adding <-loopback> to
> the bypass list for Windows overrides the default behavior and causes
> localhost and 127.0.0.1 to not bypass (i.e. they do go through the proxy). 
> **Note that the default behavior and the <-loopback> setting only affect
> "localhost" and 127.0.0.1 specifically. Other loopback ip addresses go
> through the proxy unless explicitly added to the bypass list.
> 
> So, I believe and expanded version of your chart should look like this:
> -----------------------------------------------------------------------------
> ---------------
> |                  | localhost | 127.0.0.1 | 127.0.0.9 |  MyHost*  |
> MyHost.com| 10.0.0.5  |
> |------------------+-----------+-----------+-----------|-----------+---------
> --+-----------|
> |   bypass local   |   direct  |   direct  |   proxy   |  direct   |  proxy 
> |  proxy    |
> |------------------+-----------+-----------+-----------+-----------+---------
> --+-----------|
> | not bypass local |   direct  |   direct  |   proxy   |  proxy    |  proxy 
> |  proxy    |
> -----------------------------------------------------------------------------
> ---------------
> *Note that the only item that changes is "MyHost" because it is not an IP,
> is not loopback, and it does not contain '.'
> 
> And here is a similar chart showing the <-loopback> functionality:
> --------------------------------------------------------
> |                  | localhost | 127.0.0.1 | 127.0.0.9 | 
> |------------------+-----------+-----------+-----------|
> |   <-loopback>    |   proxy   |   proxy   |   proxy   | 
> |------------------+-----------+-----------+-----------|
> |  no <-loopback>  |   direct  |   direct  |   proxy   | 
> --------------------------------------------------------
> 
> I tested all of these cases on Windows 7 using IE11.

Thank you Jon, I've created bug 1338963 for the "<-loopback>" part.

Comment 18

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/48e6afc85032

Comment 19

[stealthplayerx]

in firefox 52 (32bit) Windows, it is ignoring everything in "No proxy for" even in Manual proxy.

Comment 20

[Gary Chen [:xeonchen]]

Comment on attachment 8834824 [details]
Bug 1028195 - make proxy bypass rule consistent;

[Approval Request Comment]
If this is not a sec:{high,crit} bug, please state case for ESR consideration:
User impact if declined: proxy rule implementation changed, user might get unexpected behavior when setting specific rule of proxy, which also breaks the gtest in bug 1334443.
Fix Landed on Version: 54
Risk to taking this patch (and alternatives if risky): no obvious risk
String or UUID changes made by this patch: no

See https://wiki.mozilla.org/Release_Management/ESR_Landing_Process for more info.

Comment 21

[Ryan VanderMeulen [:RyanVM]]

Ritu, any chance you can take a look at this nomination soonish? ESR52 has broken GTests at the moment from bug 1334443 and I'd prefer to not disable the newly-added tests from that bug if we can just take this patch instead. Looks like it'd actually be useful for enterprise environments anyway.

Comment 22

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

Comment on attachment 8834824 [details]
Bug 1028195 - make proxy bypass rule consistent;

This fixes an automation test failure, ESR52.2+

Comment 23

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-esr52/rev/69a5ca2bf867

Comment 24

[Michal Čaplygin [:myf]]

(In reply to SomeNerd from comment #8)

Guys, please keep in mind, that not only 127.0.0.1 but all addresses in
127.0.0.0/8 are loopback addreses (see
https://en.wikipedia.org/wiki/Localhost#Name_resolution)

TL;DR: 127.0.0.2 mysteriously works in Chrome, but not in Firefox (and pre-chromium Edge).

Not sure this is right tree to bark at, but isn't possible this was not addressed after all?

---

System info:

• Windows 10 v1709
• Manual System proxy:
  • "Bypass proxy server for local addresses" checked,
  • "No proxy for ..." exceptions list does not include explicit loopback IPs (i.e. no 127.0.0.1;...;127.255.254.255 in there).
• Listen 80 in running local Apache HTTPD.
• Each browser uses system proxy.

Observations:

1. Visiting 127.0.0.1 works in "every browser".

2. Visiting 127.0.0.2 works in Chromium based browsers only.

3. Adding 127.0.0.2 to "No proxy for" exceptions list makes it work in Firefox and pre-Chromium Edge. (In par with aforementioned MS support article)

Conclusions

1. According that MS support article even 127.0.0.1 should not work in case of proxy and not in Exceptions List. (I guess 127.0.0.1 and localhost are probably so standard exceptions they didn't have them mentioned in that article.)

2. It seems that Chromium-based browsers behaves as if entire loopback range had exception and is never passed to proxy, what makes sense. Is it a violation of some rule Firefox obeys? Or does Firefox something wrong here?

---

(For record, I wasn't aware of this at all, I just stumbled upon terse comment under SO answer telling ~ "you can use 127.0.0.1, 127.0.0.2, 127.0.0.3 etc... for localhost")