1549678 - Proxy failover (from .pac) results in slow web browsing experience

Status: RESOLVED FIXED

(Core :: Networking: HTTP, defect, P2)

Description

[Oliver]

User Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:66.0) Gecko/20100101 Firefox/66.0

Steps to reproduce:

We use a "proxy.pac" file which sets a specific proxy "localhost:3128" (which is a port forwarded by putty) for specific webpages.

This proxy is only needed, if user is outside of LAN and only in that case putty is running and the "localhost:3128" port is available.

Up to Firefox version 64.0.2 this worked fine: In LAN Proxy "localhost:3128" failed, so after first try Firefox switched to DIRECT and worked with very good speed.

But starting with 65.0.0 in that case a delay is added with every network connection. Browsing is very slow now, when the first proxy is not available.

Up to now we did not roll out versions higher than 64.0.2 because of this. But now we have to because of the AddOn issue fixed in version 66.0.4... :(

The proxy.pac looks like this:

function FindProxyForURL(url, host)
{
var proxy = "PROXY localhost:3128; DIRECT";
var no_proxy = "DIRECT";

if (dnsDomainIs(host, ".xycorp")
|| shExpMatch(host,"confluence.xycorp.com")
|| shExpMatch(host,"jira.xycorp.com")
{
return proxy;
}

return no_proxy;
}

Actual results:

A delay of ~1..2 seconds is added on every connection. Web browsing feels very slow, if the first proxy configured is not available.

Expected results:

If the first proxy is not available Firefox should use the second proxy (in out case DIRECT) and work fast as usual.

Comment 1

[Oliver]

Forgot to mention:

If I set this proxy.pac as "System Proxy" I see the same behavior in Firefox. But other Browser (Explorer and Chrome) work fine and fast with that .pac file.

Comment 2

[Gingerbread Man]

It would be helpful if you could find the exact regression range.
https://mozilla.github.io/mozregression/quickstart.html

Comment 3

[Oliver]

(In reply to Gingerbread Man from comment #2)

It would be helpful if you could find the exact regression range.
https://mozilla.github.io/mozregression/quickstart.html

I can tell you, the last released version which worked was 64.0.2

The next version 65.0.0 had this problem, but it was worse than it is now in version 66.0.4, a lot slower. I did not test any versions in between.

So, it's a little complicated ;)

Regards,

• Oliver

Comment 4

[Oliver]

OK, I did the regression. What exactly do you need? ;) This?

pushlog_url: https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=04b755bbff848f7e245ae86c3fa2c5d6c6bad191&tochange=5836a60614764631436bf5030c5baa34c676c7a2

Comment 5

[Oliver]

This is the end of the log from Mozregression. If you need more info, please let me know.

2019-05-07T17:10:10: INFO : Narrowed inbound regression window from [4aeb1d18, 848152c2] (3 builds) to [4aeb1d18, 1e0f0a17] (2 builds) (~1 steps left)
2019-05-07T17:10:10: DEBUG : Starting merge handling...
2019-05-07T17:10:10: DEBUG : Using url: https://hg.mozilla.org/integration/autoland/json-pushes?changeset=1e0f0a17143f07a323d4f166327ae2c9d164d393&full=1
2019-05-07T17:10:11: DEBUG : Found commit message:
Backed out changeset e5cd93207b3a (bug 1503572) for build bustages on TelemetryHistogramEnums.h. CLOSED TREE

2019-05-07T17:10:11: DEBUG : Did not find a branch, checking all integration branches
2019-05-07T17:10:11: INFO : The bisection is done.
2019-05-07T17:10:11: INFO : Stopped

Comment 6

[Alice0775 White]

From comment #5
Regression window:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=4aeb1d18&tochange=1e0f0a17

Regressed by:634d9ca93c9491c01d901c53418288bc284567d2 Junior Hsu — Bug 1494364 - don't prune proxy if all non-direct proxies are disabled r=bagder

Comment 7

[Alice0775 White]

:junior,
Your patch seems to cause the network performance regression. Can you please look into this?

Comment 8

[Junior [inactive]]

It's on purpose. We have lots of users who needs proxy with fragile proxy. The policy is: if we found all the proxies are disabled or not available, we recognize this as fragile proxies, and try those proxies.

Comment 9

[Junior [inactive]]

Hello reporter,
Please see comment 8.
If you really want this happen, we may make it behind a pref, i.e., about:config.

However we need a use case to convince us this is the right thing to do.

Comment 10

[Oliver]

(In reply to Junior [:junior](ooo - May 28) from comment #9)

Hello reporter,
Please see comment 8.
If you really want this happen, we may make it behind a pref, i.e., about:config.

However we need a use case to convince us this is the right thing to do.

I don't understand what you did to cause this. But the solution is not working, it causes severe performance issues, when the proxy is not working and the browser uses "direct". I could do a video or you could please just try it yourself.

For example it can be seen very good in Atlassian Jira. It has a "Pull Down Menu" system, which opens up on mouse over. Without your patch I can mouse over all the menu items and they open/close without any delay. With you patch it takes up to 10 seconds until I see any reaction and every open/close operation of these menus is like that. Each and every time I try to use a menu. You can't use this web app with your patch.

Non of the other web browsers I know show this behavior.

I don't care, if you "retry" that non working proxy, but it should happen in a way which is not noticeable by the user.

There already is a setting in config, after what time a non working proxy should be tried again, why not use that? and if "direct" is one of the options, I would just leave "non working" proxies alone until the time set has run out.

Regards,

• Oliver

Comment 11

[Oliver]

Is there any chance to get this fixed?

[:junior] from comment #8)

It's on purpose. We have lots of users who needs proxy with fragile proxy. The policy is: if we found all the proxies are disabled or not available, we recognize this as fragile proxies, and try those proxies.

If "direct" is one of the options and it WORKS, I see no reason to check a non working proxy again and again!? On every single access. Causing extreme slowdown. If there is only ONE option (no "direct") sure, try it every time. Doesn't matter, cause user cannot browese any ways.

Comment 12

[Oliver]

(In reply to Junior [:junior] from comment #9)

Hello reporter,
Please see comment 8.
If you really want this happen, we may make it behind a pref, i.e., about:config.

However we need a use case to convince us this is the right thing to do.

My use case again:

if proxy is not available, user should still be able to browse fast using direct connection.

At least only try again every few minutes - right now, the non working proxy is tried again SEVERAL TIMES in certain web apps like Atlassian Jira and Confluence, causing extrem slowdown.

We stayed on Firefox 64.0.2 because of this.

Thanks,

• Oliver

Comment 13

[Valentin Gosu [:valentin] (he/him)]

As mentioned in bug 1791655 comment 2, I think we can add a pref for this, while we figure out a long term solution.

Comment 14

[Valentin Gosu [:valentin] (he/him)]

Attachment: Bug 1549678 - Add pref whether to retry failed proxies r=#necko

This is a regression from:
Bug 1494364 - don't prune proxy if all non-direct proxies are disabled

This pref preserves existing behaviour (and regression) but allows users to opt out of this behaviour which might cause slow browsing when the proxies are not responsive.

We might implement a proper fix for this problem in Bug 1791655.

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7cdc78209472
Add pref whether to retry failed proxies r=necko-reviewers,kershaw

Comment 16

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/7cdc78209472