1313641 - System proxy Settings ignored by JavaApplets when e10s is enabled

Status: RESOLVED INCOMPLETE

(Core Graveyard :: Plug-ins, defect, P2)

Description

[Markus Kopp]

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:49.0) Gecko/20100101 Firefox/49.0
Build ID: 20160916101415

Steps to reproduce:

Start Firefox, launch the sample Applet by Oracle: http://docs.oracle.com/javase/tutorial/deployment/applet/getStarted.html
 This did not happen with earlier Firefox Versions, with Internet Explorer is also works. When explicitly Configuring Java to use our proxy.pac File the proxy is used.


Actual results:

Applet does not launch. When tracing the Java Applet i can see in the Applet Console direct Connection to the Oracle Webseite.


Expected results:

Java should use the proxy.pac set by the system. This did not happen with earlier Firefox Versions, with Internet Explorer everything works. When explicitly Configuring Java to use our proxy.pac File is also works.

Comment 1

[Loic]

Is it reproducible by just setting a proxy in Firefox "network" options? Or do I need to use your .pac file?

Comment 2

[Markus Kopp]

I do not know. I have set the .pac file in Firefox, this does not work also. Only setting the Proxy explicit in Java Config works. Is there a way i can found out more about the way Firefox passes the Proxy Config/System Config to Java? Would love to help finding the cause for this.

Comment 3

[Benjamin Smedberg]

jduell, I'd like the networking team to own this if possible. NPAPI exposes the proxy settings to Java through the NPN_GetValue(NPNURLVProxy) API, through this chain of functions:

http://searchfox.org/mozilla-central/rev/f5c9e9a249637c9abd88754c8963ecb3838475cb/dom/plugins/base/nsNPAPIPlugin.cpp#2499
http://searchfox.org/mozilla-central/rev/f5c9e9a249637c9abd88754c8963ecb3838475cb/dom/plugins/base/nsPluginHost.cpp#603
Which then calls into nsIProtocolProxyService.

Do you know of anything changing in Firefox 49 WRT to proxy APIs? This is mainly important for Firefox 52 ESR, since that is where Java will continue to be supported for another year.

As far as I know, only the Java plugin uses this API (all other plugins do their networking using the browser stack).

Comment 4

[Jason Duell]

Gary/Patrick,

Any idea of what might have changed in our proxy stack to make this start happening?

Comment 5

[Gary Chen [:xeonchen]]

Jason, I fixed 4 proxy bugs in Firefox 49:

- Bug 1251332 is about built-in time functions behavior in PAC file.
- Bug 1233360 and bug 1258977 are related to proxy exception list.
- Bug 1250776 changed the stack size of the JavaScript VM.

I am not 100% sure any of them causes this issue, but I can check it if I can reproduce this on my platform.


Hi Markus, please correct me if I'm wrong:

1. The PAC is set in the Windows Setting.
2. Firefox is set to "use system proxy settings".
3. The applet is accessible only when the proxy server set in the PAC is used.

And I have some questions:
* Have you tried platforms other than Windows?
* What's your exception list in the proxy setting?
* Does this issue only happen when using PAC? How about set proxy directly?

Thank you.

Comment 6

[Markus Kopp]

Sorry i did not post the Operating System, but it's the one from Useragent (Windows 7).
1. Yes, Our Windows is configured to use a .pac file downloaded from an internal http URL.
2. Yes, Firefox is configured to use System settings (and this works).
3. Yes, the Java machine can not load applet classes without using our proxy.
I can not try any other operating System, the behaviour is the same for all updated firefox and all Applets loaded via Proxy. I have used the Oracle Hello World as a sample, the result is the same for any applet i have tried.
I can not change the proxy Settings for my System. If i change the Java Settings to use .pac file instead of system settings it works again.

Comment 7

[Patrick McManus [:mcmanus]]

thanks gary

Comment 8

[Gary Chen [:xeonchen]]

Hi Markus,

Unfortunately I cannot reproduce your issue.
Here's my test:

1) Set system internet option to use PAC
2) In the PAC, every traffic goes through the proxy
3) Set Firefox use system proxy settings
4) Set Java to use browser setting

I use Wireshark to monitor every packet sent from my host.
Seems all packets are sent to the proxy server except DNS queries.

Then I modify 4) to make it connect directly, I can observe that the applet_HelloWorld.jar is got from 23.53.67.xx, that makes sense to me.

Since you mentioned about version 49, one possibility is that in bug 1258977, some exception handling mechanism changed, I guess your PAC may have different behavior.

I just made a build that removes bug 1258977 in [1], would you help to check if this fixes your problem?

[1] https://tinyurl.com/jlslvpa

Comment 9

[Markus Kopp]

Hi Gary,
i have tried to use Java with this Version, but the java plugin disappears with this Firefox Version (not shown in about:addons). I have even tried to reinstall java, this does not make a difference. If i launch the normal Firefox Version 49.x the java plugin is still visible in about:addons

Comment 10

[Gary Chen [:xeonchen]]

(In reply to Markus Kopp from comment #9)
> Hi Gary,
> i have tried to use Java with this Version, but the java plugin disappears
> with this Firefox Version (not shown in about:addons). I have even tried to
> reinstall java, this does not make a difference. If i launch the normal
> Firefox Version 49.x the java plugin is still visible in about:addons

I tried and I can't use Java either.

Looks like NPAPI is removed in bug 1269807?
Benjamin, do you have any idea?

Comment 11

[Benjamin Smedberg]

You could just debug/fix this against the 51 codebase.

But for 52, you can still enable non-Flash plugins by setting the pref plugin.load_flash_only to false and removing your pluginreg.dat cache in your profile.

Comment 12

[Loic]

I tried with a proxy set in a PAC file and FF52, jp2launcher.exe has been proxified properly (on Win 7), like all connections from firefox.exe.

Comment 13

[Gary Chen [:xeonchen]]

OK, I've tried several channels and releases of Firefox, here's my observation result:

- I cannot reproduce on the latest Firefox release/beta
- I can reproduce on the latest Firefox aurora/nightly

Then I found bug 436344 broke this and bug 1165286 had fixed the issue.
But I can still reproduce this issue using nightly build released on Aug. 2015.

I cannot understand the difference between release/beta and aurora/nightly until Firefox was asking me to enable e10s.

Looks like the issue only happens on e10s mode.

Nick, do you have any idea?

Comment 14

[Gary Chen [:xeonchen]]

Oh! the patch author is Patrick

Comment 15

[Gary Chen [:xeonchen]]

Markus, would you try disabling e10s?

More info can be found at https://wiki.mozilla.org/Electrolysis#Firefox_Release

Comment 16

[Gary Chen [:xeonchen]]

It happens only in e10s mode because |FindProxyForURL| is the only user of |nsProtocolProxyService| in content process.

Within |FindProxyForURL|, it calls:

1) |do_GetService(NS_PROTOCOLPROXYSERVICE_CONTRACTID)|
2) |DeprecatedBlockingResolve|

The result of (1) is always called after (2), but (2) depends on the result of (1).

Comment 17

[Markus Kopp]

Checked it twice. As soon as e10s is enabled the Proxy is ignored.

Comment 18

[Patrick McManus [:mcmanus]]

(In reply to Gary Chen [:xeonchen] from comment #16)
> It happens only in e10s mode because |FindProxyForURL| is the only user of
> |nsProtocolProxyService| in content process.
> 
> Within |FindProxyForURL|, it calls:
> 
> 1) |do_GetService(NS_PROTOCOLPROXYSERVICE_CONTRACTID)|
> 2) |DeprecatedBlockingResolve|
> 
> The result of (1) is always called after (2), but (2) depends on the result
> of (1).

I understand this is e10s only.. but I don't understand the rest of the analysis.

in any event, to fix you'll need to find a way to proxy this via synchronous ipdl to the main process. Which will be horrendous for performance, but this has always been the case because of the blocking DNS API available to PAC in JS... for regular PAC we mitigate it with a thread pool running the JS, but that can't be done out of the plugin interface.

otoh - we can remove this whole interface in 52, so maybe we ought to wontfix this corner case now for 50/51 rather than making an investment with a rather short shelf life. Benajmin, what's your opinion?

Comment 19

[Patrick McManus [:mcmanus]]

or maybe we can add the java plugin to the e10s-blocklist

Comment 20

[Markus Kopp]

I have set browser.tabs.remote.force-enable;false to explicitly disable e10s. This is is the fastest solution for me.

Comment 21

[Gary Chen [:xeonchen]]

Attachment: 0001-Bug-1313641-initiailize-nsProtocolProxyService-earli.patch

In |nsProtocolProxyService::Init| it tries to asynchronously retrieve system proxy setting for the following request. In e10s mode, every network request are sent through main process and therefore the service is never used.

But when Java requests the proxy setting by calling |nsPluginHost::FindProxyForURL|, the service is requested and initialized here, but the PAC loading is asynchronous, therefore the setting will always be retrieved after |nsPluginHost::FindProxyForURL| returned.

The simplest workaround is to create the service earlier.

Comment 22

[Patrick McManus [:mcmanus]]

this doesn't really match well with what I presumed the problem was, but if you've done the testing you've done the testing.

I'm not the right person to comment on whether that change to the plugin code is right though - I am encouraged its so small :)

Comment 23

[Benjamin Smedberg]

Comment on attachment 8808900 [details] [diff] [review]
0001-Bug-1313641-initiailize-nsProtocolProxyService-earli.patch

I'm fine with this if it works!

Comment 24

[Gary Chen [:xeonchen]]

Hi Markus,

Would you download [1] and help to see if the issue has been resolved?

[1] http://tinyurl.com/p6uybrn

Comment 25

[Gary Chen [:xeonchen]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=a4879262d2f691697a7992a2baec894e12b2f1a2

Comment 26

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/161d052b3e74
Initialize nsProtocolProxyService earlier. r=bsmedberg

Comment 27

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/161d052b3e74

Comment 28

[Markus Kopp]

Hi Gery, i sorry could not test on Friday. I have tried the Download, it did not fix the issue for me. But as i said, i will resolve the problem by deactivating e10s.

Comment 29

[Benjamin Smedberg]

Sounds like this needs to be reopened, since it doesn't actually fix the problem?

Comment 30

[Gary Chen [:xeonchen]]

(In reply to Benjamin Smedberg [:bsmedberg] from comment #29)
> Sounds like this needs to be reopened, since it doesn't actually fix the
> problem?

Yeah...that's the problem. I couldn't figure out what's the difference between Markus' platform and mine.

Marcus, would you kindly help to check the latest nightly build again?
Make sure e10s is enabled and the version is 53, and

1) in about:config, set plugin.load_flash_only to false
2) close all firefox instances, delete pluginreg.dat (located in %AppData%\Mozilla\Firefox\Profiles\<your_profile>\pluginreg.dat)

If the issue still exists, would you give me a bit more information about the situation? I have to feed this bug with something to go on.

Comment 31

[Gary Chen [:xeonchen]]

Hi Benjamin, I cannot reach Markus for a while, and I also sent him email to ask for help in Nov. 2016.

In my test environment, this patch did resolve the situation I reproduced, however, since the reporter said it didn't fix his problem, I'll consider to backout the patch, and close this bug as WONTFIX, how do you think?

Comment 32

[Markus Kopp]

Sorry, my fault.
The patches did not help in my case and i think keeping the code as clean as passible ist the best. I will create an explicit proxy configuration for Java because this is the other way to deal with the problem.

Comment 33

[Gary Chen [:xeonchen]]

(In reply to Markus Kopp from comment #32)
> Sorry, my fault.
> The patches did not help in my case and i think keeping the code as clean as
> passible ist the best. I will create an explicit proxy configuration for
> Java because this is the other way to deal with the problem.

Hi Markus, did you try the latest nightly build and still not working?

Comment 34

[Masatoshi Kimura [:emk]]

The latest Nightly no longer supports Java applet.

Comment 35

[Gary Chen [:xeonchen]]

(In reply to Masatoshi Kimura [:emk] from comment #34)
> The latest Nightly no longer supports Java applet.

1) in about:config, set plugin.load_flash_only to false
2) close all firefox instances, delete pluginreg.dat (located in %AppData%\Mozilla\Firefox\Profiles\<your_profile>\pluginreg.dat)

Comment 36

[Benjamin Smedberg]

Gary I'm fine with leaving this patch in if we think it fixes part of the issue. At this point we're only talking about this for the 52ESR release, so the value of working on this is rapidly diminishing.

Comment 37

[Benjamin Smedberg]

Going to resolve this since the chance of taking this for 52 beta/ESR is low.