Update AWSY tp6 tests to mitmproxy 4
Categories
(Testing :: AWSY, enhancement)
Tracking
(firefox70 fixed)
| Tracking | Status | |
|---|---|---|
| firefox70 | --- | fixed |
People
(Reporter: davehunt, Assigned: Bebe)
References
Details
Attachments
(1 file)
The Raptor tests that AWSY tp6 is based on are now running against mitmproxy 4. There are also new recordings, and an addon that eliminates the upstream connection for the certificate. This bug covers updating to mitmproxy 4, which may also require us to update to the new recordings.
|
||
Comment 1•5 years ago
•
|
||
A few pointers on where to start:
- We keep a test manifest that will need to have it's hashes updated for the new recordings.
- We use a copy of the old alternate-server-replay script, I believe that needs to be replaced for mitmproxy 4
- We setup mitmproxy for our tests in the
setupTp6function, presumably various flags need to change
For a quick verification test you can use:
./mach awsy-test --quick --tp6 --preferences testing/awsy/conf/tp6-prefs.json
Remove the --quick for a full test run.
To test on try you can use ./mach try fuzzy and select opt-awsy-tp6 tests. For example to test on linux64:
./mach try fuzzy --artifact --rebuild 5 -q "'linux64/opt-awsy-tp6"
|
Reporter | |
Updated•5 years ago
|
|
|
Reporter | |
Updated•5 years ago
|
|
Assignee | |
Comment 2•5 years ago
|
||
|
|
Assignee | |
Comment 3•5 years ago
|
||
try build are failing when I try to build the path to the alternate server script.
:erahm can you offer some guidance to generate this path from awsy tests
|
|
||
Comment 4•5 years ago
|
||
(In reply to Florin Strugariu [:Bebe] (needinfo me) from comment #3)
try build are failing when I try to build the path to the alternate server script.
:erahm can you offer some guidance to generate this path from awsy tests
In automation it needs to be packaged in the test archive, raptor probably won't be present when running the awsy tests so this isn't going to work. I filed bug 1539983 several months ago as a proper fix for this. I'd suggest implementing that as a blocker or just copying it again if you don't have time to improve mozproxy.
|
||
Comment 6•5 years ago
|
||
| bugherder | ||
|
|
Assignee | |
Comment 7•5 years ago
|
||
== Change summary for alert #22877 (as of Wed, 28 Aug 2019 16:52:17 GMT) ==
Regressions:
4% Images linux64-shippable opt stylo tp6 7,323,405.67 -> 7,628,640.68
4% Images macosx1014-64-shippable opt stylo tp6 7,628,772.47 -> 7,917,352.93
3% Heap Unclassified macosx1014-64-shippable opt stylo tp6 129,589,822.54 -> 133,650,930.93
Improvements:
8% JS macosx1014-64-shippable opt stylo tp6 221,468,920.89 -> 202,851,354.16
7% JS windows10-64-shippable-qr opt stylo tp6 223,957,861.33 -> 208,711,376.40
7% JS windows7-32-shippable opt stylo tp6 166,452,167.28 -> 155,330,115.93
7% Explicit Memory windows10-64-shippable-qr opt stylo tp6 522,035,879.26 -> 487,240,643.49
6% JS linux64-shippable opt stylo tp6 219,058,170.34 -> 204,855,573.36
6% Resident Memory windows10-64-shippable opt stylo tp6 704,358,782.45 -> 661,592,972.28
6% Resident Memory linux64-shippable opt stylo tp6 790,873,364.16 -> 743,657,301.02
6% Resident Memory linux64-shippable-qr opt stylo tp6 938,792,698.28 -> 882,930,308.09
6% Explicit Memory macosx1014-64-shippable opt stylo tp6 558,541,461.16 -> 525,591,359.76
6% JS linux64-shippable-qr opt stylo tp6 219,079,557.58 -> 206,165,826.53
6% Explicit Memory windows10-64-shippable opt stylo tp6 497,079,661.73 -> 468,104,083.81
6% JS windows10-64-shippable opt stylo tp6 217,447,340.61 -> 204,946,106.05
6% Explicit Memory linux64-shippable-qr opt stylo tp6 634,690,748.28 -> 598,903,699.38
5% Explicit Memory windows7-32-shippable opt stylo tp6 400,947,285.43 -> 378,982,809.99
5% Explicit Memory linux64-shippable opt stylo tp6 533,389,636.81 -> 504,210,172.60
4% Resident Memory windows10-64-shippable-qr opt stylo tp6 755,287,666.09 -> 723,081,556.43
4% Resident Memory windows7-32-shippable opt stylo tp6 704,764,694.43 -> 678,161,512.68
For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=22877
Description
•