Windows Proxy Setting "Bypass proxy server for local addresses" is ignored by Firefox when using "Use system proxy settings"

RESOLVED FIXED in Firefox -esr52

Status

()

defect
RESOLVED FIXED
5 years ago
2 years ago

People

(Reporter: somenerd, Assigned: xeonchen)

Tracking

(Blocks 1 bug)

30 Branch
mozilla54
x86_64
Windows 8.1
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox-esr52 fixed, firefox54 fixed)

Details

(Whiteboard: [proxy][necko-active])

Attachments

(1 attachment)

User Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; Touch; .NET4.0E; .NET4.0C; Tablet PC 2.0; .NET CLR 3.5.30729; .NET CLR 2.0.50727; .NET CLR 3.0.30729; BRI/2; Zune 4.7; InfoPath.3; rv:11.0) like Gecko

Steps to reproduce:

In Windows I've activated in Internet Options, Connections, LAN settings "Bypass proxy server for local addresses" which adds "<local>" at the end of the registry key "HKEY_CURRENT_USER\SOFTWARE\MICROSOFT\WINDOWS\CURRENTVERSION\Internet Settings\ProxyOverride"
Then I activated in Firefox Options, Advanced, Network, Connection Settings "Use system proxy settings"
(see actual / expected results) 
---
Important:
Firefox itself knows <local> if you use "Manual proxy configuration" see https://bugzilla.mozilla.org/show_bug.cgi?id=44980#c30 and https://developer.mozilla.org/en-US/docs/Proxies_in_Necko#Proxies_and_local_hosts 
but it's ignored if it's in the OS list of proxy exceptions rather than the FF own exception list.


Actual results:

non-fqdn websites like http://intranet or http://server are forwarded to the proxy


Expected results:

non-fqdn websites like http://intranet or http://server are not forwarded to the proxy
I found only bug 72444
Component: Untriaged → Networking: HTTP
Product: Firefox → Core
Whiteboard: [necko-backlog]
Whiteboard: [necko-backlog] → [proxy][necko-backlog]
I have encountered this issue for years and can confirm it happens for me at work. If I use the "use system proxy settings" option, non-fqdn intranet pages go to the proxy even if the Microsoft option to bypass the proxy for these sites is enabled. If I change Firefox to manual proxy settings and convert the Microsoft proxy bypass list from the semicolon-separated, * wildcard version into the comma-separated CIDR one that Firefox prefers, I can add <local> to the list to get the desired functionality (see bug 72444). 

If the "Use system proxy settings" option were changed to use the <local> functionality when the equivalent Microsoft setting is enabled, it should resolve this bug.
(In reply to Jon from comment #2)
> I have encountered this issue for years and can confirm it happens for me at
> work. If I use the "use system proxy settings" option, non-fqdn intranet
> pages go to the proxy even if the Microsoft option to bypass the proxy for
> these sites is enabled. If I change Firefox to manual proxy settings and
> convert the Microsoft proxy bypass list from the semicolon-separated, *
> wildcard version into the comma-separated CIDR one that Firefox prefers, I
> can add <local> to the list to get the desired functionality (see bug
> 72444). 
> 
> If the "Use system proxy settings" option were changed to use the <local>
> functionality when the equivalent Microsoft setting is enabled, it should
> resolve this bug.

Thanks for the comment, Jon, very helpful!

Looks like we have some inconsistent here:
[1] thinks "<local>" is for localhost/127.0.0.1, but [2] checks for non-IP, non-dotted hosts.

Daniel, do you think we should take the solution from [2] and maybe add support for LAN IPs?

[1] https://dxr.mozilla.org/mozilla-central/rev/af8a2573d0f1e9cc6f2ba0ab67d7a702a197f177/toolkit/system/windowsproxy/nsWindowsSystemProxySettings.cpp#137
[2] https://dxr.mozilla.org/mozilla-central/rev/af8a2573d0f1e9cc6f2ba0ab67d7a702a197f177/netwerk/base/nsProtocolProxyService.cpp#729
Assignee: nobody → xeonchen
Flags: needinfo?(daniel)
See Also: → 72444
Whiteboard: [proxy][necko-backlog] → [proxy][necko-active]
I think both code paths should use the setting the same way as the current differences are more confusing than anything else.

Isn't the missing part here only that the [2] case needs to also say that host names without dots are to be considered to be local (in addition to "localhost" and "127.0.0.1")?

Maybe [::1] should be respected too while at it?
Flags: needinfo?(daniel)
(In reply to Gary Chen [:xeonchen] from comment #3)
> (In reply to Jon from comment #2)
> > I have encountered this issue for years and can confirm it happens for me at
> > work. If I use the "use system proxy settings" option, non-fqdn intranet
> > pages go to the proxy even if the Microsoft option to bypass the proxy for
> > these sites is enabled. If I change Firefox to manual proxy settings and
> > convert the Microsoft proxy bypass list from the semicolon-separated, *
> > wildcard version into the comma-separated CIDR one that Firefox prefers, I
> > can add <local> to the list to get the desired functionality (see bug
> > 72444). 
> > 
> > If the "Use system proxy settings" option were changed to use the <local>
> > functionality when the equivalent Microsoft setting is enabled, it should
> > resolve this bug.
> 
> Thanks for the comment, Jon, very helpful!
> 
> Looks like we have some inconsistent here:
> [1] thinks "<local>" is for localhost/127.0.0.1, but [2] checks for non-IP,
> non-dotted hosts.
> 
> Daniel, do you think we should take the solution from [2] and maybe add
> support for LAN IPs?
> 
> [1]
> https://dxr.mozilla.org/mozilla-central/rev/
> af8a2573d0f1e9cc6f2ba0ab67d7a702a197f177/toolkit/system/windowsproxy/
> nsWindowsSystemProxySettings.cpp#137
> [2]
> https://dxr.mozilla.org/mozilla-central/rev/
> af8a2573d0f1e9cc6f2ba0ab67d7a702a197f177/netwerk/base/nsProtocolProxyService.
> cpp#729

I believe the comment in [1] (and associated logic) is incorrect while [2] is valid, at least with respect to how the relevant Windows setting works. The Windows proxy override list is stored in the following registry value: HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows\CurrentVersion\Internet Settings\ProxyOverride
If you check the "Bypass proxy server for local addresses" box in Internet Options, it appends "<local>" to this registry value. So, <local> is not meant to signify only localhost. The relevant if statement in [1] (line 152) should probably return true for any host which does not contain a period as well, similar to [2].
Comment on attachment 8834824 [details]
Bug 1028195 - make proxy bypass rule consistent;

https://reviewboard.mozilla.org/r/110662/#review111956

If you're fine with the little question about IPv6 addresses, then I'm all good!

::: netwerk/base/nsProtocolProxyService.cpp:731
(Diff revision 1)
>      }
>  
>      // Don't use proxy for local hosts (plain hostname, no dots)
> -    if (!is_ipaddr && mFilterLocalHosts && !host.Contains('.')) {
> +    if ((!is_ipaddr && mFilterLocalHosts && !host.Contains('.')) ||
> +        host.EqualsLiteral("127.0.0.1") ||
> +        host.EqualsLiteral("::1")) {

As long as you're *sure* the IPv6 version is compared this way, then I'm fine with it. I mean IPv6 literal addresses are written as '[address]' in URLs and I haven't checked if those brackets have been removed or not...

::: toolkit/system/windowsproxy/nsWindowsSystemProxySettings.cpp:161
(Diff revision 1)
> +                if (!isIpAddr && !host.Contains('.')) {
>                      return true;
> +                }
> +
> +                if (host.EqualsLiteral("127.0.0.1") ||
> +                    host.EqualsLiteral("::1")) {

The IPv6 comment from above goes for this as well.
Attachment #8834824 - Flags: review?(daniel) → review+
Guys, please keep in mind, that not only 127.0.0.1 but all addresses in 127.0.0.0/8 are loopback addreses (see https://en.wikipedia.org/wiki/Localhost#Name_resolution)
I think the ip address/loopback considerations are outside the scope of this Microsoft setting and the <local> functionality.  Really, the if statement at nsWindowsSystemProxySettings.cpp[152] should *only* be checking if host contains '.', and not even checking for loopback because the override value being checked there is the literal "<local>". So, if I understand it correctly, it will only land there if "<local>" exists in the override list, which is unrelated to loopback/localhost as discussed. I think this means the loopback check here is superfluous. Initially, I was being cautious by trying to suggest the minimum effective change.

I think nsProtocolProxyService.cpp[729] (referenced as [2] above) also handles this part correctly because it specifically ignores ip addresses for the <local> check with !is_ipaddr (it uses https://dxr.mozilla.org/mozilla-central/source/nsprpub/pr/src/misc/prnetdb.c#2217 which does check for ipv6 addresses as well)
I found some documentation from Microsoft supporting the idea that their setting only applies to host names and not ip addresses: https://support.microsoft.com/en-us/help/262981/internet-explorer-uses-proxy-server-for-local-ip-address-even-if-the-bypass-proxy-server-for-local-addresses-option-is-turned-on 
under "Cause"
Comment on attachment 8834824 [details]
Bug 1028195 - make proxy bypass rule consistent;

https://reviewboard.mozilla.org/r/110662/#review112216

::: netwerk/base/nsProtocolProxyService.cpp:731
(Diff revision 1)
>      }
>  
>      // Don't use proxy for local hosts (plain hostname, no dots)
> -    if (!is_ipaddr && mFilterLocalHosts && !host.Contains('.')) {
> +    if ((!is_ipaddr && mFilterLocalHosts && !host.Contains('.')) ||
> +        host.EqualsLiteral("127.0.0.1") ||
> +        host.EqualsLiteral("::1")) {

Yes, in this function the brackets are removed.
(In reply to Jon from comment #10)
> I found some documentation from Microsoft supporting the idea that their
> setting only applies to host names and not ip addresses:
> https://support.microsoft.com/en-us/help/262981/internet-explorer-uses-proxy-
> server-for-local-ip-address-even-if-the-bypass-proxy-server-for-local-
> addresses-option-is-turned-on 
> under "Cause"

I agree, so the case would be like the following table, right?

--------------------------------------
|            | localhost | 127.0.0.1 |
|------------+-----------+-----------|
|   bypass   |   direct  |   proxy   |
|------------+-----------+-----------|
| not bypass |   proxy   |   proxy   |
--------------------------------------

I'm just curious about how IE/Edge behave like of the given cases?
Actually, Microsoft always bypasses the proxy for localhost and 127.0.0.1 regardless of the other settings we've discussed. For example, if you have an empty bypass list, both localhost and 127.0.0.1 will still bypass the proxy (even if you do *not* have the 'bypass proxy for local addresses' box checked).

After some research though, I did find another case: adding <-loopback> to the bypass list for Windows overrides the default behavior and causes localhost and 127.0.0.1 to not bypass (i.e. they do go through the proxy). 
**Note that the default behavior and the <-loopback> setting only affect "localhost" and 127.0.0.1 specifically. Other loopback ip addresses go through the proxy unless explicitly added to the bypass list.

So, I believe and expanded version of your chart should look like this:
--------------------------------------------------------------------------------------------
|                  | localhost | 127.0.0.1 | 127.0.0.9 |  MyHost*  | MyHost.com| 10.0.0.5  |
|------------------+-----------+-----------+-----------|-----------+-----------+-----------|
|   bypass local   |   direct  |   direct  |   proxy   |  direct   |  proxy    |  proxy    |
|------------------+-----------+-----------+-----------+-----------+-----------+-----------|
| not bypass local |   direct  |   direct  |   proxy   |  proxy    |  proxy    |  proxy    |
--------------------------------------------------------------------------------------------
*Note that the only item that changes is "MyHost" because it is not an IP, is not loopback, and it does not contain '.'

And here is a similar chart showing the <-loopback> functionality:
--------------------------------------------------------
|                  | localhost | 127.0.0.1 | 127.0.0.9 | 
|------------------+-----------+-----------+-----------|
|   <-loopback>    |   proxy   |   proxy   |   proxy   | 
|------------------+-----------+-----------+-----------|
|  no <-loopback>  |   direct  |   direct  |   proxy   | 
--------------------------------------------------------

I tested all of these cases on Windows 7 using IE11.
Blocks: 1338963
Pushed by gachen@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/48e6afc85032
make proxy bypass rule consistent; r=bagder
(In reply to Jon from comment #14)
> Actually, Microsoft always bypasses the proxy for localhost and 127.0.0.1
> regardless of the other settings we've discussed. For example, if you have
> an empty bypass list, both localhost and 127.0.0.1 will still bypass the
> proxy (even if you do *not* have the 'bypass proxy for local addresses' box
> checked).
> 
> After some research though, I did find another case: adding <-loopback> to
> the bypass list for Windows overrides the default behavior and causes
> localhost and 127.0.0.1 to not bypass (i.e. they do go through the proxy). 
> **Note that the default behavior and the <-loopback> setting only affect
> "localhost" and 127.0.0.1 specifically. Other loopback ip addresses go
> through the proxy unless explicitly added to the bypass list.
> 
> So, I believe and expanded version of your chart should look like this:
> -----------------------------------------------------------------------------
> ---------------
> |                  | localhost | 127.0.0.1 | 127.0.0.9 |  MyHost*  |
> MyHost.com| 10.0.0.5  |
> |------------------+-----------+-----------+-----------|-----------+---------
> --+-----------|
> |   bypass local   |   direct  |   direct  |   proxy   |  direct   |  proxy 
> |  proxy    |
> |------------------+-----------+-----------+-----------+-----------+---------
> --+-----------|
> | not bypass local |   direct  |   direct  |   proxy   |  proxy    |  proxy 
> |  proxy    |
> -----------------------------------------------------------------------------
> ---------------
> *Note that the only item that changes is "MyHost" because it is not an IP,
> is not loopback, and it does not contain '.'
> 
> And here is a similar chart showing the <-loopback> functionality:
> --------------------------------------------------------
> |                  | localhost | 127.0.0.1 | 127.0.0.9 | 
> |------------------+-----------+-----------+-----------|
> |   <-loopback>    |   proxy   |   proxy   |   proxy   | 
> |------------------+-----------+-----------+-----------|
> |  no <-loopback>  |   direct  |   direct  |   proxy   | 
> --------------------------------------------------------
> 
> I tested all of these cases on Windows 7 using IE11.

Thank you Jon, I've created bug 1338963 for the "<-loopback>" part.
https://hg.mozilla.org/mozilla-central/rev/48e6afc85032
Status: UNCONFIRMED → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla54
in firefox 52 (32bit) Windows, it is ignoring everything in "No proxy for" even in Manual proxy.
Comment on attachment 8834824 [details]
Bug 1028195 - make proxy bypass rule consistent;

[Approval Request Comment]
If this is not a sec:{high,crit} bug, please state case for ESR consideration:
User impact if declined: proxy rule implementation changed, user might get unexpected behavior when setting specific rule of proxy, which also breaks the gtest in bug 1334443.
Fix Landed on Version: 54
Risk to taking this patch (and alternatives if risky): no obvious risk
String or UUID changes made by this patch: no

See https://wiki.mozilla.org/Release_Management/ESR_Landing_Process for more info.
Attachment #8834824 - Flags: approval-mozilla-esr52?
Ritu, any chance you can take a look at this nomination soonish? ESR52 has broken GTests at the moment from bug 1334443 and I'd prefer to not disable the newly-added tests from that bug if we can just take this patch instead. Looks like it'd actually be useful for enterprise environments anyway.
Flags: needinfo?(rkothari)
Comment on attachment 8834824 [details]
Bug 1028195 - make proxy bypass rule consistent;

This fixes an automation test failure, ESR52.2+
Flags: needinfo?(rkothari)
Attachment #8834824 - Flags: approval-mozilla-esr52? → approval-mozilla-esr52+
See Also: → 1381144
You need to log in before you can comment on or make changes to this bug.