Startup very slow when set to use auto proxy configuration using pac file

VERIFIED FIXED in Firefox 65

Status

()

defect
P2
normal
VERIFIED FIXED
4 years ago
6 months ago

People

(Reporter: satheesan.nayar, Assigned: kershaw)

Tracking

({perf})

41 Branch
mozilla65
x86_64
Windows 7
Points:
---

Firefox Tracking Flags

(firefox65 verified, firefox66 verified)

Details

(Whiteboard: [proxy][necko-triaged])

Attachments

(2 attachments, 1 obsolete attachment)

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:41.0) Gecko/20100101 Firefox/41.0
Build ID: 20151014143721

Steps to reproduce:

1.Removed proxy settings. FireFox starts quickly as it used to
2.Put the configuration back and the problem came back
3.Checked if the .pac file was fetch-able quickly and it was



Actual results:

Initial startup very slow 


Expected results:

Proxy configuration should not have affected the Firefox startup speed.
Does the .pac file get fetched from a server and does that respond slowly?
(In reply to Sebastian H. [:aryx][:archaeopteryx] from comment #1)
> Does the .pac file get fetched from a server and does that respond slowly?

Yes, the .pac file get fetched from a server. The server response is quick
Hello Satheesan,

Tested this scenario on Windows 7 x64 with the latest Nightly release (45.0a1) and could not reproduce it. Did you tried to reproduce this issue on a latest Firefox Nightly or Release builds with a new profile and with all addons disabled (safe mode maybe)? Maybe it's an addon that causes this problems at startup. Did you tried also to use the same proxy URL in other browsers configurations ? Maybe is something in your script that causes this.

If this still occurs, can you provide the URL for that .pac file you are using ? I am trying to reproduce this on my side with some free examples found on web and I had no success until now. Maybe providing your own will help us to find the issue.

Thanks,
Paul.
Component: Untriaged → Networking
Flags: needinfo?(satheesan.nayar)
OS: Unspecified → Windows 7
Product: Firefox → Core
Hardware: Unspecified → x86_64
Hi Paul,
I tried with a new profile with the release build (version 42.0) and that seems to have fixed the problem. I'll contact my users who originally reported this problem to confirm.
Other browsers didn't have the problem with the same .pac URL

Thanks,
Satheesan
Flags: needinfo?(satheesan.nayar)
Hi Satheesan,

I'm glad this resolved your problems. I will change the issue status to RESOLVED-Worksforme since it seems to no longer occur on latest release version. If your users are still able to reproduce this even after performing the same steps as you, feel free to reopen this issue and provide more details on how we can reproduce it.

Thanks !
Status: UNCONFIRMED → RESOLVED
Closed: 4 years ago
Resolution: --- → WORKSFORME
Hi Paul,
  My users are still experiencing the problem and I looked further into it. The slowness is when the user tries to visit a page immediately after the firefox started up or if they had set a default home page.
  When I created a new profile this got masked because there was no default home page set in the new profile. So I mistook it as the problem had resolved.
  The problem still exists. The moment I set a default home page(say google.com) the problem comes back.If I don't set a default home page firefox seems to startup just fine. But if I visit a site immediately after the startup the delay is 15-20 seconds.

Thanks,
Satheesan
Status: RESOLVED → UNCONFIRMED
Resolution: WORKSFORME → ---
btw: my automatic proxy URL is http://www.sri.com/web-proxy/corp-proxy.pac
Hi Satheesan,

Thanks for the automatic proxy URL. I can confirm that the issue is reproducing on my side too with this configuration but only when trying to load the first page. The browser starts normally, but when trying to load the first page (set startup page or other url), it takes like 10-15 seconds to load. But all the next pages load normally after. 
Tested on latest Nightly build 45.0a1 - 20151123030237
User Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0

Also I have tested your proxy URL with Chrome and the page loading takes like 2 seconds. So it seams that it may be a problem on our end.

Since I can reproduce it on my environment I will set the issue status to NEW so developers could look into it.

Thanks,
Paul.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Hi Paul,
  Thanks for the update

Satheesan
Keywords: perf
Whiteboard: [necko-backlog]
Whiteboard: [necko-backlog] → [necko-backlog][proxy]
See Also: → 1294695, 1121443
Assignee: nobody → xeonchen
Whiteboard: [necko-backlog][proxy] → [necko-next][proxy]
Bulk change to priority: https://bugzilla.mozilla.org/show_bug.cgi?id=1399258
Priority: -- → P2
Posted file http.log.0
It took longer than expected to fetch the PAC file because:

1. the PAC URL [1] is redirected to [2].
2. while it tries to download [2], [3] tries to connect to [4] (to verify certificate?) via proxy.
3. so |nsHttpChannel::ResolveProxy()| will be blocked until timeout, and in this case an empty proxy will be used temporarily, then everything goes well.

[1] http://www.sri.com/web-proxy/corp-proxy.pac
[2] https://www.sri.com/web-proxy/corp-proxy.pac
[3] https://searchfox.org/mozilla-central/rev/1285ae3e810e2dbf2652c49ee539e3199fcfe820/security/manager/ssl/nsNSSCallbacks.cpp#94
[4] http://sr.symcd.com/
(In reply to Gary Chen [:xeonchen] (needinfo plz) from comment #11)
> Created attachment 8921815 [details]
> http.log.0
> 
> It took longer than expected to fetch the PAC file because:
> 
> 1. the PAC URL [1] is redirected to [2].
> 2. while it tries to download [2], [3] tries to connect to [4] (to verify
> certificate?) via proxy.
> 3. so |nsHttpChannel::ResolveProxy()| will be blocked until timeout, and in
> this case an empty proxy will be used temporarily, then everything goes well.
> 
> [1] http://www.sri.com/web-proxy/corp-proxy.pac
> [2] https://www.sri.com/web-proxy/corp-proxy.pac
> [3]
> https://searchfox.org/mozilla-central/rev/
> 1285ae3e810e2dbf2652c49ee539e3199fcfe820/security/manager/ssl/nsNSSCallbacks.
> cpp#94
> [4] http://sr.symcd.com/

Seems connections to [2] and [4] are blocked by each other.

Daniel, do you have any idea?
Flags: needinfo?(daniel)
First: great catch Gary!

This is an interesting catch-22. The fetch done in [3] is clearly certificate/verification related and as that request is using HTTP itself, it will need a proxy for the request, thus it needs the proxy setup that Firefox is already trying to figure out by doing this HTTPS request in [2]!

I have two thoughts on this:

1 - we should make sure that while connecting to a HTTPS proxy or a PAC URL, we should probably not try to use a HTTP proxy for SSL certificate reasons since we can't use it yet

2 - simply disabling HTTP proxy use before the proxy is proven to work (ie the HTTPS proxy connection is completed or the PAC file is loaded) might not always be what users want either, since users ask Firefox to use a proxy for a reason and simply bypassing it may not be what the user wants. But failing HTTP requests before the proxy is setup may cause certificate verification side effects...
Flags: needinfo?(daniel)
David, any thoughts on this from a larger security/overview perspective?
Flags: needinfo?(dkeeler)
I'm sort-of surprised this hasn't been a problem before now. In any case, we should probably just fail any OCSP requests that result from trying to verify a proxy's certificate (unless there isn't an http->https proxy I guess?). OCSP is soft-fail by default, so this will let the certificate verification succeed and the proxy configuration can continue.
Flags: needinfo?(dkeeler)
Assignee: xeonchen → nobody
Assignee: nobody → kershaw
Whiteboard: [necko-next][proxy] → [proxy][necko-triaged]
While downloading a PAC script, we should not use proxy for the OCSP request since the proxy is not configured completely yet.
This patch simply uses |mPACMan->IsLoading| and Content-Type to do this check.
For lots of users, they can't reach the Internet without using the specified proxy. Another set of users really want us to avoid using the network outside of the specified proxy as that could potentially "leak" that data/information to the outside in an unintended way.

I think "skipping OCSP" seems wiser than "skipping the proxy" for this case.
Attachment #9018339 - Attachment is obsolete: true
This is a straightforward patch.
Just add a new attribute in nsIProtocolProxyService to indicate whether PAC is still loading. If yes, fail the OCSP request.
Pushed by kjang@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/28403444666c
Skip OCSP request if PAC download is in progress r=keeler,bagder
https://hg.mozilla.org/mozilla-central/rev/28403444666c
Status: NEW → RESOLVED
Closed: 4 years ago9 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla65
Flags: qe-verify+

Managed to reproduce the issue on Firefox 45.0a1 (2015-11-23), under Windows 10x64, using the automatic proxy from Comment 7.

The issue is no longer reproducible on Firefox 66.0a1 (2019-01-23), or on Firefox 65.0RC.
Tests were executed under Windows 10x64, macOS 10.12.6 and under Ubuntu 16.04x64.

Status: RESOLVED → VERIFIED
Flags: qe-verify+
You need to log in before you can comment on or make changes to this bug.