Remove usage of proxy server from testing/profiles/perf/user.js
Categories
(Testing :: General, defect, P1)
Tracking
(firefox69 fixed)
| Tracking | Status | |
|---|---|---|
| firefox69 | --- | fixed |
People
(Reporter: whimboo, Assigned: whimboo)
References
Details
Attachments
(3 files)
As of right now the Raptor profiles get a proxy server setup for HTTP here:
https://searchfox.org/mozilla-central/source/testing/profiles/perf/user.js#79-81
Whenever I run a test against a live site, the connection cannot be made because it's routed through the proxy which doesn't run at this point.
I don't know for which reasons this proxy exists for perf tests, so maybe Andrew and Joel can give some answers to it.
Also why is the proxy only set for HTTP but not HTTPS too? Most connections nowadays run through a secure connection.
|
||
Comment 1•5 years ago
|
||
this is set for all of our testing. It appears there are prefs for ssl:
https://searchfox.org/mozilla-central/source/testing/awsy/conf/tp6-prefs.json#7
IIRC tp6/mitmproxy has to turn that off so we can use the proxy that mitmproxy provides.
|
Assignee | |
Comment 3•5 years ago
|
||
(In reply to Joel Maher ( :jmaher ) (UTC-4) from comment #1)
this is set for all of our testing. It appears there are prefs for ssl:
For which specific reason does the proxy exist for perf tests?
https://searchfox.org/mozilla-central/source/testing/awsy/conf/tp6-prefs.json#7
This is awsy only so doesn't affect Raptor.
IIRC tp6/mitmproxy has to turn that off so we can use the proxy that mitmproxy provides.
|
|
||
Comment 4•5 years ago
|
||
I gave you an example, talos uses the same prefs as well and so does raptor for non tp6. This is used to ensure that we are not sending traffic to the live internet.
|
||
Updated•5 years ago
|
|
|
Assignee | |
Comment 5•5 years ago
|
||
(In reply to Joel Maher ( :jmaher ) (UTC-4) from comment #4)
I gave you an example, talos uses the same prefs as well and so does raptor for non tp6. This is used to ensure that we are not sending traffic to the live internet.
Who/what is we? The tests or Firefox (and several services)? I just wonder because it's only port 80 but not HTTPS too, which is mostly used those days.
|
|
||
Comment 6•5 years ago
|
||
we == firefox + tests
these settings have been around for 10 years and there has not been updating. I assume updating them for ssl would be a good thing.
|
|
Assignee | |
Comment 7•5 years ago
|
||
So while investigating more it looks like that the proxy is only there so all the services whose preferences for remote URLs are modified in perf/user.js get routed to 127.0.0.1.
Note that for other test harnesses we do not have such a logic, but set the host as %(server)s in the preference value, and let the specific harness interpolate this value based on it's own locally running HTTPd instance.
I'm fairly sure that this should be adapted by any harness using those specific perf preferences. With that we won't need the proxy for port 80 nor for 443 at all.
Harnesses affected here are just Talos, and Raptor.
Joel, would that be ok with you?
|
|
||
Comment 8•5 years ago
|
||
this might work out- will we cause a crash to the browser if it tries to access external sites? I believe that is the case already, but I want to confirm.
Lastly, to turn off services via prefs is difficult- we add or edit services often and they need to be kept up between all the profiles in testing/profiles- could you file a bug so we have one place to turn off all services?
|
|
Assignee | |
Comment 9•5 years ago
|
||
(In reply to Joel Maher ( :jmaher ) (UTC-4) from comment #8)
this might work out- will we cause a crash to the browser if it tries to access external sites? I believe that is the case already, but I want to confirm.
Correct. We only have to make sure that the environment variable MOZ_DISABLE_NONLOCAL_CONNECTIONS is set in automation for all of the performance jobs. I assume that this has already been done a while ago, even for Talos and Raptor. But I can check that again.
Lastly, to turn off services via prefs is difficult- we add or edit services often and they need to be kept up between all the profiles in testing/profiles- could you file a bug so we have one place to turn off all services?
Do you mean an extra folder under testing/profiles something like local which would contain all those entries, which are currently spread across various other sub folders?
|
|
Assignee | |
Comment 10•5 years ago
|
||
(In reply to Henrik Skupin (:whimboo) [⌚️UTC+2] from comment #9)
Do you mean an extra folder under
testing/profilessomething likelocalwhich would contain all those entries, which are currently spread across various other sub folders?
Note that this would allow some test harnesses to include the common and base preferences, but opt-out from using the force local settings. This is something Marionette based harnesses like firefox-ui-functional remote tests could benefit from in the future.
|
|
Assignee | |
Comment 11•5 years ago
|
||
So both Talos and Raptor are using this environment variable. As such I just pushed a try build with the proxy settings removed for both Talos and Raptor but only Linux64. Not sure if there are tests only running on eg Windows.
https://treeherder.mozilla.org/#/jobs?repo=try&revision=a5df2a1cb6defde321b55483394d871ed44b5c21
Lets see how much failures we see.
|
|
||
Comment 12•5 years ago
|
||
I think having our common profile preferences well defined so we have a single place for all external blocking/spoofing server preferences to live- a good idea.
afaik, windows has xperf and that is the only test unique to windows; osx shouldn't have any unique tests- so linux is a good sample.
|
|
Assignee | |
Comment 13•5 years ago
|
||
As discussed with Joel on IRC the problem here is most likely the reference to the remove w3c logo. I replaced it with a local copy of it, and here is an updated try build for both the dromaeojs and g3 jobs:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=efb417eae51ec92beaceb46bf8eb6921670bffa6
Otherwise all other jobs are still green!
|
|
Assignee | |
Comment 14•5 years ago
|
||
|
|
Assignee | |
Comment 15•5 years ago
|
||
Depends on D32375
|
|
Assignee | |
Comment 17•5 years ago
|
||
|
|
Assignee | |
Comment 18•5 years ago
|
||
Just another try build with xperf job and raptor-youtube-playback, given that I was able to get rid of the horrible proxy workaround too.
https://treeherder.mozilla.org/#/jobs?repo=try&revision=b5148309c279a3236f28f339aef927fa7116acdf
|
|
Assignee | |
Comment 19•5 years ago
|
||
(In reply to Joel Maher ( :jmaher ) (UTC-4) from comment #12)
I think having our common profile preferences well defined so we have a single place for all external blocking/spoofing server preferences to live- a good idea.
I filed bug 1553949 for that.
|
||
Comment 20•5 years ago
|
||
Pushed by hskupin@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/2bdc4e2c82ae [talos] Replace remote w3c image with local copy. r=jmaher https://hg.mozilla.org/integration/autoland/rev/5b555225cd80 Remove usage of proxy server from testing/profiles/perf/user.js. r=jmaher https://hg.mozilla.org/integration/autoland/rev/77898b7a7ea2 [raptor] Remove proxy workaround from raptor-youtube-playback tests. r=perftest-reviewers,stephendonner
|
||
Comment 21•5 years ago
|
||
| bugherder | ||
https://hg.mozilla.org/mozilla-central/rev/2bdc4e2c82ae
https://hg.mozilla.org/mozilla-central/rev/5b555225cd80
https://hg.mozilla.org/mozilla-central/rev/77898b7a7ea2
|
||
Comment 22•5 years ago
•
|
||
Looks like this harness change updated our baseline:
== Change summary for alert #21343 (as of Thu, 06 Jun 2019 14:09:42 GMT) ==
Improvements:
6% dromaeo_css windows10-64-shippable opt e10s stylo 14,369.78 -> 15,177.96
For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=21343
Description
•