1219935 - Startup very slow when set to use auto proxy configuration using pac file

Status: VERIFIED FIXED

(Core :: Networking, defect, P2)

Description

[Satheesan Nayar]

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:41.0) Gecko/20100101 Firefox/41.0
Build ID: 20151014143721

Steps to reproduce:

1.Removed proxy settings. FireFox starts quickly as it used to
2.Put the configuration back and the problem came back
3.Checked if the .pac file was fetch-able quickly and it was



Actual results:

Initial startup very slow 


Expected results:

Proxy configuration should not have affected the Firefox startup speed.

Comment 1

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Does the .pac file get fetched from a server and does that respond slowly?

Comment 2

[Satheesan Nayar]

(In reply to Sebastian H. [:aryx][:archaeopteryx] from comment #1)
> Does the .pac file get fetched from a server and does that respond slowly?

Yes, the .pac file get fetched from a server. The server response is quick

Comment 3

[Paul Oiegas [:poiegas], PM for (Ecosystem QA, Services QA, WebCompat QA, Ads QA, and Solo QA teams)]

Hello Satheesan,

Tested this scenario on Windows 7 x64 with the latest Nightly release (45.0a1) and could not reproduce it. Did you tried to reproduce this issue on a latest Firefox Nightly or Release builds with a new profile and with all addons disabled (safe mode maybe)? Maybe it's an addon that causes this problems at startup. Did you tried also to use the same proxy URL in other browsers configurations ? Maybe is something in your script that causes this.

If this still occurs, can you provide the URL for that .pac file you are using ? I am trying to reproduce this on my side with some free examples found on web and I had no success until now. Maybe providing your own will help us to find the issue.

Thanks,
Paul.

Comment 4

[Satheesan Nayar]

Hi Paul,
I tried with a new profile with the release build (version 42.0) and that seems to have fixed the problem. I'll contact my users who originally reported this problem to confirm.
Other browsers didn't have the problem with the same .pac URL

Thanks,
Satheesan

Comment 5

[Paul Oiegas [:poiegas], PM for (Ecosystem QA, Services QA, WebCompat QA, Ads QA, and Solo QA teams)]

Hi Satheesan,

I'm glad this resolved your problems. I will change the issue status to RESOLVED-Worksforme since it seems to no longer occur on latest release version. If your users are still able to reproduce this even after performing the same steps as you, feel free to reopen this issue and provide more details on how we can reproduce it.

Thanks !

Comment 6

[Satheesan Nayar]

Hi Paul,
  My users are still experiencing the problem and I looked further into it. The slowness is when the user tries to visit a page immediately after the firefox started up or if they had set a default home page.
  When I created a new profile this got masked because there was no default home page set in the new profile. So I mistook it as the problem had resolved.
  The problem still exists. The moment I set a default home page(say google.com) the problem comes back.If I don't set a default home page firefox seems to startup just fine. But if I visit a site immediately after the startup the delay is 15-20 seconds.

Thanks,
Satheesan

Comment 7

[Satheesan Nayar]

btw: my automatic proxy URL is http://www.sri.com/web-proxy/corp-proxy.pac

Comment 8

[Paul Oiegas [:poiegas], PM for (Ecosystem QA, Services QA, WebCompat QA, Ads QA, and Solo QA teams)]

Hi Satheesan,

Thanks for the automatic proxy URL. I can confirm that the issue is reproducing on my side too with this configuration but only when trying to load the first page. The browser starts normally, but when trying to load the first page (set startup page or other url), it takes like 10-15 seconds to load. But all the next pages load normally after. 
Tested on latest Nightly build 45.0a1 - 20151123030237
User Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0

Also I have tested your proxy URL with Chrome and the page loading takes like 2 seconds. So it seams that it may be a problem on our end.

Since I can reproduce it on my environment I will set the issue status to NEW so developers could look into it.

Thanks,
Paul.

Comment 9

[Satheesan Nayar]

Hi Paul,
  Thanks for the update

Satheesan

Comment 10

[Firefox Bug Husbandry Bot]

Bulk change to priority: https://bugzilla.mozilla.org/show_bug.cgi?id=1399258

Comment 11

[Gary Chen [:xeonchen]]

Attachment: http.log.0

It took longer than expected to fetch the PAC file because:

1. the PAC URL [1] is redirected to [2].
2. while it tries to download [2], [3] tries to connect to [4] (to verify certificate?) via proxy.
3. so |nsHttpChannel::ResolveProxy()| will be blocked until timeout, and in this case an empty proxy will be used temporarily, then everything goes well.

[1] http://www.sri.com/web-proxy/corp-proxy.pac
[2] https://www.sri.com/web-proxy/corp-proxy.pac
[3] https://searchfox.org/mozilla-central/rev/1285ae3e810e2dbf2652c49ee539e3199fcfe820/security/manager/ssl/nsNSSCallbacks.cpp#94
[4] http://sr.symcd.com/

Comment 12

[Gary Chen [:xeonchen]]

(In reply to Gary Chen [:xeonchen] (needinfo plz) from comment #11)
> Created attachment 8921815 [details]
> http.log.0
> 
> It took longer than expected to fetch the PAC file because:
> 
> 1. the PAC URL [1] is redirected to [2].
> 2. while it tries to download [2], [3] tries to connect to [4] (to verify
> certificate?) via proxy.
> 3. so |nsHttpChannel::ResolveProxy()| will be blocked until timeout, and in
> this case an empty proxy will be used temporarily, then everything goes well.
> 
> [1] http://www.sri.com/web-proxy/corp-proxy.pac
> [2] https://www.sri.com/web-proxy/corp-proxy.pac
> [3]
> https://searchfox.org/mozilla-central/rev/
> 1285ae3e810e2dbf2652c49ee539e3199fcfe820/security/manager/ssl/nsNSSCallbacks.
> cpp#94
> [4] http://sr.symcd.com/

Seems connections to [2] and [4] are blocked by each other.

Daniel, do you have any idea?

Comment 13

[Daniel Stenberg [:bagder]]

First: great catch Gary!

This is an interesting catch-22. The fetch done in [3] is clearly certificate/verification related and as that request is using HTTP itself, it will need a proxy for the request, thus it needs the proxy setup that Firefox is already trying to figure out by doing this HTTPS request in [2]!

I have two thoughts on this:

1 - we should make sure that while connecting to a HTTPS proxy or a PAC URL, we should probably not try to use a HTTP proxy for SSL certificate reasons since we can't use it yet

2 - simply disabling HTTP proxy use before the proxy is proven to work (ie the HTTPS proxy connection is completed or the PAC file is loaded) might not always be what users want either, since users ask Firefox to use a proxy for a reason and simply bypassing it may not be what the user wants. But failing HTTP requests before the proxy is setup may cause certificate verification side effects...

Comment 14

[Daniel Stenberg [:bagder]]

David, any thoughts on this from a larger security/overview perspective?

Comment 15

[Dana Keeler (she/her) (use needinfo) [:keeler]]

I'm sort-of surprised this hasn't been a problem before now. In any case, we should probably just fail any OCSP requests that result from trying to verify a proxy's certificate (unless there isn't an http->https proxy I guess?). OCSP is soft-fail by default, so this will let the certificate verification succeed and the proxy configuration can continue.

Comment 16

[Kershaw Chang [:kershaw]]

Attachment: Bug 1219935 - Don’t use proxy for the ocsp request if PAC is downloading.

While downloading a PAC script, we should not use proxy for the OCSP request since the proxy is not configured completely yet.
This patch simply uses |mPACMan->IsLoading| and Content-Type to do this check.

Comment 17

[Daniel Stenberg [:bagder]]

For lots of users, they can't reach the Internet without using the specified proxy. Another set of users really want us to avoid using the network outside of the specified proxy as that could potentially "leak" that data/information to the outside in an unintended way.

I think "skipping OCSP" seems wiser than "skipping the proxy" for this case.

Comment 18

[Kershaw Chang [:kershaw]]

Attachment: Bug 1219935 - Skip OCSP request if PAC download is in progress

This is a straightforward patch.
Just add a new attribute in nsIProtocolProxyService to indicate whether PAC is still loading. If yes, fail the OCSP request.

Comment 19

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/28403444666c
Skip OCSP request if PAC download is in progress r=keeler,bagder

Comment 20

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/28403444666c

Comment 21

[Mihai Boldan, Desktop QA [:mboldan]]

Managed to reproduce the issue on Firefox 45.0a1 (2015-11-23), under Windows 10x64, using the automatic proxy from Comment 7.

The issue is no longer reproducible on Firefox 66.0a1 (2019-01-23), or on Firefox 65.0RC.
Tests were executed under Windows 10x64, macOS 10.12.6 and under Ubuntu 16.04x64.