1041511 - Can't access 'localhost:port' while on a remote page once bug 354493 was landed

Status: RESOLVED FIXED

(Core :: DOM: Navigation, defect)

Description

[Daniel Gherasim]

All of our mozmill tests that was accessing localhost pages started to fail once bug 354493 was landed.

We can manually reproduce this:
Pre-requirements: a local apache server installed.

1. Change port of localhost in apache to 8181
2. Try to access "localhost:8181" while already connected to a web page (like www.google.com). This doesn't work! Get's redirected to www.localhost.com:8181 which is not found.
3. Open about:blank or any local page, then open "localhost:8181". This works!

Steve, any idea how your patch could affect here ?

Comment 1

[Steve Workman [:sworkman] (INACTIVE)]

Looking into this now. Able to locally reproduce the behavior as you describe above. More later when I know what's happening.

Comment 2

[Steve Workman [:sworkman] (INACTIVE)]

Attachment: WIP 1.0 Fix private navigations

I have a patch that works locally - passes all NetworkZonePolicy automated tests and allows navigating from public to private documents in the address bar. I've pushed to try - let me know if it works for you too.

https://tbpl.mozilla.org/?tree=Try&rev=fbcf5e00bde0

Comment 4

[Steve Workman [:sworkman] (INACTIVE)]

Attachment: v1.0 Fix private navigations

Amended a comment in the patch, and added LOAD_INITIAL_DOCUMENT_URI cases to the test script.

Pat, can you take a look here, please? I think checking LOAD_INITIAL_DOCUMENT_URI should do the trick, right? Let me know.

https://tbpl.mozilla.org/?tree=Try&rev=b75986b7b0e3

Comment 5

[Boris Zbarsky [:bzbarsky]]

Comment on attachment 8459837 [details] [diff] [review]
v1.0 Fix private navigations

LOAD_INITIAL_DOCUMENT_URI is set on all "first-party" loads, no?

So if you have a web page with <a target="subframe" href="whetever"> clicking that link will do a load with that flag set.

That said, it looks like you're still checking _ancestors_ of the docshell.  But then why LOAD_INITIAL_DOCUMENT_URI and not LOAD_DOCUMENT_URI?  the latter is what SetPrivateNetworkPermission uses...

Comment 6

[Daniel Gherasim]

Hey Steve, tested with your build from tbpl & it's working fine now.

Comment 7

[Steve Workman [:sworkman] (INACTIVE)]

(In reply to Boris Zbarsky [:bz] from comment #5)
> Comment on attachment 8459837 [details] [diff] [review]
> v1.0 Fix private navigations
> 
> LOAD_INITIAL_DOCUMENT_URI is set on all "first-party" loads, no?
> 
> So if you have a web page with <a target="subframe" href="whetever">
> clicking that link will do a load with that flag set.
> 
> That said, it looks like you're still checking _ancestors_ of the docshell.

Yup - this should (and seems to from local testing) block subframes of public pages from loading private docs.

> But then why LOAD_INITIAL_DOCUMENT_URI and not LOAD_DOCUMENT_URI?  the
> latter is what SetPrivateNetworkPermission uses...

Indeed. LOAD_DOCUMENT_URI works too. I was focusing too much on the "INITIAL" part. Working on updating the patch and test now.

Comment 8

[Steve Workman [:sworkman] (INACTIVE)]

Attachment: v1.1 Fix private navigations

-- Use LOAD_DOCUMENT_URI instead of LOAD_INITIAL_DOCUMENT_URI. (Verified manually as well as in test script).
-- Update test script to cover new behavior, i.e. top-level doc loads should always have permission to load from private networks; doc loads in sub-frames are dependent on the permission of the owning document.
-- In relevant test cases, verify loadgroup permissions with both a doc-load channel and a non-doc-load channel.

https://tbpl.mozilla.org/?tree=Try&rev=8e3a63e3afd7

Comment 14

[Gary [:streetwolf52]]

I ran the try build of this patch and it fixed my problem as mentioned here: https://bugzilla.mozilla.org/show_bug.cgi?id=1042559#c2

Comment 15

[Steve Workman [:sworkman] (INACTIVE)]

Thanks for the r+ Pat, and Gary for verifying the build.

Mozilla-inbound is currently closed, so I will try to land this later. Once it has baked on mozilla-central for a couple of days, I will request aurora-approval.

Comment 16

[Steve Workman [:sworkman] (INACTIVE)]

Landed on inbound: https://hg.mozilla.org/integration/mozilla-inbound/rev/a377b855c360

Comment 17

[Bill Gianopoulos [:WG9s]]

The patch here does NOT seem to fix the issue. I still have the problem that on my local network when i go to http://www.wg9s.com/mozilla/firefox/  where www.wg9s.com resolves to 192.168.1.20 it loads OK but each time I hit F5 to refresh I get a different number of missing images.

I also changed the importance to major I would call this a blocker if the regressor were not identified so I could just back that out.  This prevents me form doing a lot of my local testing.

Comment 18

[Bill Gianopoulos [:WG9s]]

Changing this to a blocker the patch for bug 354493 just needs to be backed out.

Comment 19

[Bill Gianopoulos [:WG9s]]

This should be easy to test just add 24.60.111.126 to the IPs restricted by bug 354493 and then go to http://www.wg9s.com/mozilla/firefox/ and hit F5 repeatedly and see that the images do not load reliably.

Comment 20

[Bill Gianopoulos [:WG9s]]

I suspect the patch is for bug 354493 is just broken is some way that it does not work at all correctly with cached objects.

Comment 21

[Bill Gianopoulos [:WG9s]]

Can we just get this backed out until it works correctly?

Comment 23

[Roman R.]

The port isn't necessary for the bug to show up. I get the same problem with just localhost.

Comment 24

[Steve Workman [:sworkman] (INACTIVE)]

(In reply to Bill Gianopoulos [:WG9s] from comment #17)
Hi Bill,

> The patch here does NOT seem to fix the issue. I still have the problem that
> on my local network when i go to http://www.wg9s.com/mozilla/firefox/  where
> www.wg9s.com resolves to 192.168.1.20 it loads OK but each time I hit F5 to
> refresh I get a different number of missing images.

This is a little different to what this bug is about - this bug is about localhost not loading, but you seem to be describing issues with resources from local networks. I know 354493 restricted both of those, but I'd like a second bug to keep the conversations clear and separate. Can you file a new bug and describe the issue there, please?

Also, please describe the makeup of the site you are testing. I would like to know if there is an iframe or frame loaded publicly - if so, images in those frames will be blocked. Do you have different sources for the images? Are some loaded from public addresses and some locally? Are the local ones visible?

This information would be very helpful in the other bug.

> I also changed the importance to major I would call this a blocker if the
> regressor were not identified so I could just back that out.  This prevents
> me form doing a lot of my local testing.

If this is primarily for testing purposes and not consumer-facing, you can disable the pref 'network.zonepolicy.enabled'.

Comment 25

[Bill Gianopoulos [:WG9s]]

T His is a normal Apache site fits a mess of files  in a a file view really nothing strange. I really don't understand what you think I need to provide about how this is other than a normal thing to do  just using a 192.168.1.20 ip address is only thing different.

Comment 26

[Bill Gianopoulos [:WG9s]]

So this prevents me form actually distributing my builds properly to to othher users so is why I decided it is a blocker.

Comment 28

[Patrick McManus [:mcmanus]]

steve is headed offline for a few days, but he asked me to disable the feature if turning the pref off helps bill. That way steve can look at it when he come back.

Bill - can you go to about:config and set network.zonepolicy.enabled to false, restart the browser, and let me know if things work correctly? If they do I'll push that pref change to the main repo when I come back to the keyboard in a bit.

Comment 29

[Patrick McManus [:mcmanus]]

comment 28

Comment 30

[Marc Auslander]

I opened one of the dups and the pref "fixes" my problem.  I can access local pages from tabs visiting internet pages.

Comment 31

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/a377b855c360

Comment 32

[Peja Stija]

Have to agree with Bill,
the try-build fixed almost all proxy related problems, but i've just run into the same problems again.
Fore example on the site www.bfbc2.eu/en/ps3/stats/Bullethead___
having a lot of images cached on a local proxy, when accessing the page, none of the images get requested from the local proxy.

Comment 33

[Bill Gianopoulos [:WG9s]]

(In reply to Steve Workman [:sworkman] Offline until 7/30 from comment #24)

> If this is primarily for testing purposes and not consumer-facing, you can
> disable the pref 'network.zonepolicy.enabled'.

Changing that preference did not help my issue at all.

Comment 35

[Patrick McManus [:mcmanus]]

(In reply to Bill Gianopoulos [:WG9s] from comment #33)
> (In reply to Steve Workman [:sworkman] Offline until 7/30 from comment #24)
> 
> > If this is primarily for testing purposes and not consumer-facing, you can
> > disable the pref 'network.zonepolicy.enabled'.
> 
> Changing that preference did not help my issue at all.

interesting. that obviously slightly reduces the chances that a backout will help you. So I'm going to make a try build of 1041511 and 354493 backed out and only push it to the main trees after you confirm. Thanks.

stay tuned.

Comment 36

[Patrick McManus [:mcmanus]]

this is a backed out build on try.. let me know if it solves the problem
https://tbpl.mozilla.org/?tree=Try&rev=481ece1facc2

Comment 37

[Marc Auslander]

As of the 7/24 nightly my version of this issue is gone - I can now visit a local address from an internet tab.

Comment 38

[Bill Gianopoulos [:WG9s]]

(In reply to Patrick McManus [:mcmanus] from comment #36)
> this is a backed out build on try.. let me know if it solves the problem
> https://tbpl.mozilla.org/?tree=Try&rev=481ece1facc2

I will try this when I get home from work.

Comment 39

[Patrick McManus [:mcmanus]]

(In reply to Bill Gianopoulos [:WG9s] from comment #38)
> (In reply to Patrick McManus [:mcmanus] from comment #36)
> > this is a backed out build on try.. let me know if it solves the problem
> > https://tbpl.mozilla.org/?tree=Try&rev=481ece1facc2
> 
> I will try this when I get home from work.

thanks.. please also double check the latest nightly to make sure it doesn't help you (e..g comment 37)

Comment 40

[Bill Gianopoulos [:WG9s]]

(In reply to Patrick McManus [:mcmanus] from comment #39)
> (In reply to Bill Gianopoulos [:WG9s] from comment #38)
> > (In reply to Patrick McManus [:mcmanus] from comment #36)
> > > this is a backed out build on try.. let me know if it solves the problem
> > > https://tbpl.mozilla.org/?tree=Try&rev=481ece1facc2
> > 
> > I will try this when I get home from work.
> 
> thanks.. please also double check the latest nightly to make sure it doesn't
> help you (e..g comment 37)

I tried today;s nightly and no difference.  I also tried toggling the network.zonepolicy.enabled prefernce with today's nightly still no go.

I am unable to reproduce the issue with the try build that does the backout, so the backout definitely resolves the issue I see.

To make it clear the page in question always loads with all images on initial load it is when i click on the reload icon that the issue occurs each time I click I get different results.  Sometimes all images sometime no images sometimes different images are absent.

Comment 41

[Bill Gianopoulos [:WG9s]]

I also tried the following, which I think should ahve disabled all of this code but obviously does not:

diff --git a/netwerk/base/src/nsLoadGroup.cpp b/netwerk/base/src/nsLoadGroup.cpp
--- a/netwerk/base/src/nsLoadGroup.cpp
+++ b/netwerk/base/src/nsLoadGroup.cpp
@@ -1093,24 +1093,26 @@ nsresult nsLoadGroup::MergeLoadFlags(nsI
 
     outFlags = flags;
     return rv;
 }
 
 NS_IMETHODIMP
 nsLoadGroup::GetAllowLoadsFromPrivateNetworks(bool *aAllowed)
 {
-    *aAllowed = mAllowLoadsFromPrivateNetworks;
+//    *aAllowed = mAllowLoadsFromPrivateNetworks;
+    *aAllowed = true;
     return NS_OK;
 }
 
 NS_IMETHODIMP
 nsLoadGroup::SetAllowLoadsFromPrivateNetworks(bool aAllowed)
 {
-    mAllowLoadsFromPrivateNetworks = aAllowed;
+//    mAllowLoadsFromPrivateNetworks = aAllowed;
+     mAllowLoadsFromPrivateNetworks = true;
     return NS_OK;
 }
 
 // nsLoadGroupConnectionInfo
 
 class nsLoadGroupConnectionInfo MOZ_FINAL : public nsILoadGroupConnectionInfo
 {
     ~nsLoadGroupConnectionInfo() {}

Comment 42

[Ray Satiro]

Bug #1042497 that I filed was marked as a duplicate of this bug. The issue I had is not fixed though. In the latest nightly Firefox is still unable to find the proxy server in a tab where the proxy is set after it's opened. The proxy is listening on localhost. 

The message is different now. It says "The proxy server is refusing connections". If I open a new tab and go to the same site it works though.

gecko.mstone = 34.0a1
gecko.buildID = 20140724030201

Comment 43

[Alice0775 White]

Please uplift the patch to Aurora33.012

Comment 44

[Patrick McManus [:mcmanus]]

backed out as:

remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/973c6a19f142
remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/96339f5d6392

I'm going to leave it as fixed because the root cause 354493 was backed out too.

Comment 45

[Ray Satiro]

(In reply to Ray Satiro from comment #42)
> Bug #1042497 that I filed was marked as a duplicate of this bug. The issue I
> had is not fixed though. In the latest nightly Firefox is still unable to
> find the proxy server in a tab where the proxy is set after it's opened. The
> proxy is listening on localhost. 
> 
> The message is different now. It says "The proxy server is refusing
> connections". If I open a new tab and go to the same site it works though.
> 
> gecko.mstone = 34.0a1
> gecko.buildID = 20140724030201

Oops, nevermind, the proxy server wasn't running. The new tab pulled from cache I assume. I retested and can no longer reproduce bug #1042497.

Comment 46

[Bill Gianopoulos [:WG9s]]

(In reply to Patrick McManus [:mcmanus] from comment #44)
> backed out as:
> 
> remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/973c6a19f142
> remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/96339f5d6392
> 
> I'm going to leave it as fixed because the root cause 354493 was backed out
> too.

But the backout has not yet landed on mozilla-central it seems.  Otherwise there is something really odd going on because it fails with today's nightly.

Comment 47

[Peja Stija]

Still experiencing the same as in comment #32

Comment 48

[Bill Gianopoulos [:WG9s]]

(In reply to Peja Stija from comment #47)
> Still experiencing the same as in comment #32

Did you read my previous comment?  The backout did not make it into today's nightly.  It should be in tomorrow's.

Comment 49

[Bill Gianopoulos [:WG9s]]

(In reply to Bill Gianopoulos [:WG9s] from comment #40)
> (In reply to Patrick McManus [:mcmanus] from comment #39)
> > (In reply to Bill Gianopoulos [:WG9s] from comment #38)
> > > (In reply to Patrick McManus [:mcmanus] from comment #36)
> > > > this is a backed out build on try.. let me know if it solves the problem
> > > > https://tbpl.mozilla.org/?tree=Try&rev=481ece1facc2
> > > 
> > > I will try this when I get home from work.
> > 
> > thanks.. please also double check the latest nightly to make sure it doesn't
> > help you (e..g comment 37)
> 
> I tried today;s nightly and no difference.  I also tried toggling the
> network.zonepolicy.enabled prefernce with today's nightly still no go.
> 
> I am unable to reproduce the issue with the try build that does the backout,
> so the backout definitely resolves the issue I see.
> 
> To make it clear the page in question always loads with all images on
> initial load it is when i click on the reload icon that the issue occurs
> each time I click I get different results.  Sometimes all images sometime no
> images sometimes different images are absent.

I should have mentioned that a shift-reload "fixes" it.

Comment 50

[Bill Gianopoulos [:WG9s]]

This might mean that loading the page out of cache and  because the modified since request ways it has not changed does not do the allow access to the restricted networks.

But I have the while idea of how is this supposed to work with cache anyway.

I have a portable laptop.  when it is on my home network it uses the local RFC 1918 address to get to my server because that is what the local DNS server returns, yet when I am elsewhere it uses internet DNS and gets the Internet accessible IP so the cache might have pages loaded from a different IP than what DNS is going to return currently.  Does that screw this up also (that is not the case I have now as I have cleared the whole cache and only have stuff loaded when at home in it now and still have issues, just asking)

Comment 51

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/973c6a19f142

Comment 52

[Dave Garrett]

Just for the record, an additional error case which I hit today on Aurora: this breaks attempts to load the Plex Media Manager in an existing tab, as that application accesses things via a localhost page. (Plex is an app used to stream video from a computer to a settop box) There's probably other software that does this too, so it's not just routers, intranets, and people with test servers.

Comment 54

[Sylvestre Ledru [:Sylvestre]]

Patrick or Steve, 33 is still marked as affected. A backout missing in aurora?

Comment 55

[Patrick McManus [:mcmanus]]

(In reply to Sylvestre Ledru [:sylvestre] from comment #54)
> Patrick or Steve, 33 is still marked as affected. A backout missing in
> aurora?

This was a follow-on patch to a bug that has since been backed out of aurora before the follow-on made it there too. This patch only needed to be backed of moz-central.

So we're good.

Comment 56

[Sylvestre Ledru [:Sylvestre]]

Thanks. Clearing also Steve n-i and untracking it.

Comment 59

[Ryan]

I filed bug https://bugzilla.mozilla.org/show_bug.cgi?id=1044564

This bug seems to bee fixed in the latest builds. For me at least.