Closed Bug 1567137 Opened 5 months ago Closed 4 months ago

Update AWSY tp6 tests to mitmproxy 4

Categories

(Testing :: AWSY, enhancement)

Version 3
enhancement
Not set

Tracking

(firefox70 fixed)

RESOLVED FIXED
mozilla70
Tracking Status
firefox70 --- fixed

People

(Reporter: davehunt, Assigned: Bebe)

References

Details

Attachments

(1 file)

The Raptor tests that AWSY tp6 is based on are now running against mitmproxy 4. There are also new recordings, and an addon that eliminates the upstream connection for the certificate. This bug covers updating to mitmproxy 4, which may also require us to update to the new recordings.

Blocks: 1567138
No longer blocks: 1567122

A few pointers on where to start:

For a quick verification test you can use:

./mach awsy-test --quick --tp6 --preferences testing/awsy/conf/tp6-prefs.json

Remove the --quick for a full test run.

To test on try you can use ./mach try fuzzy and select opt-awsy-tp6 tests. For example to test on linux64:

./mach try fuzzy --artifact --rebuild 5 -q "'linux64/opt-awsy-tp6"
Assignee: nobody → krubin
Status: NEW → ASSIGNED
Assignee: krubin → fstrugariu

try build are failing when I try to build the path to the alternate server script.

https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=48aa1aeced858b211a03290a47485e97658f9d38

:erahm can you offer some guidance to generate this path from awsy tests

Flags: needinfo?(erahm)

(In reply to Florin Strugariu [:Bebe] (needinfo me) from comment #3)

try build are failing when I try to build the path to the alternate server script.

https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=48aa1aeced858b211a03290a47485e97658f9d38

:erahm can you offer some guidance to generate this path from awsy tests

In automation it needs to be packaged in the test archive, raptor probably won't be present when running the awsy tests so this isn't going to work. I filed bug 1539983 several months ago as a proper fix for this. I'd suggest implementing that as a blocker or just copying it again if you don't have time to improve mozproxy.

Flags: needinfo?(erahm)
Depends on: 1539983
Pushed by fstrugariu@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/cdd44c843e2a
Update AWSY tp6 tests to mitmproxy 4 r=perftest-reviewers,erahm,AlexandruIonescu
Status: ASSIGNED → RESOLVED
Closed: 4 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla70

== Change summary for alert #22877 (as of Wed, 28 Aug 2019 16:52:17 GMT) ==

Regressions:

4% Images linux64-shippable opt stylo tp6 7,323,405.67 -> 7,628,640.68
4% Images macosx1014-64-shippable opt stylo tp6 7,628,772.47 -> 7,917,352.93
3% Heap Unclassified macosx1014-64-shippable opt stylo tp6 129,589,822.54 -> 133,650,930.93

Improvements:

8% JS macosx1014-64-shippable opt stylo tp6 221,468,920.89 -> 202,851,354.16
7% JS windows10-64-shippable-qr opt stylo tp6 223,957,861.33 -> 208,711,376.40
7% JS windows7-32-shippable opt stylo tp6 166,452,167.28 -> 155,330,115.93
7% Explicit Memory windows10-64-shippable-qr opt stylo tp6 522,035,879.26 -> 487,240,643.49
6% JS linux64-shippable opt stylo tp6 219,058,170.34 -> 204,855,573.36
6% Resident Memory windows10-64-shippable opt stylo tp6 704,358,782.45 -> 661,592,972.28
6% Resident Memory linux64-shippable opt stylo tp6 790,873,364.16 -> 743,657,301.02
6% Resident Memory linux64-shippable-qr opt stylo tp6 938,792,698.28 -> 882,930,308.09
6% Explicit Memory macosx1014-64-shippable opt stylo tp6 558,541,461.16 -> 525,591,359.76
6% JS linux64-shippable-qr opt stylo tp6 219,079,557.58 -> 206,165,826.53
6% Explicit Memory windows10-64-shippable opt stylo tp6 497,079,661.73 -> 468,104,083.81
6% JS windows10-64-shippable opt stylo tp6 217,447,340.61 -> 204,946,106.05
6% Explicit Memory linux64-shippable-qr opt stylo tp6 634,690,748.28 -> 598,903,699.38
5% Explicit Memory windows7-32-shippable opt stylo tp6 400,947,285.43 -> 378,982,809.99
5% Explicit Memory linux64-shippable opt stylo tp6 533,389,636.81 -> 504,210,172.60
4% Resident Memory windows10-64-shippable-qr opt stylo tp6 755,287,666.09 -> 723,081,556.43
4% Resident Memory windows7-32-shippable opt stylo tp6 704,764,694.43 -> 678,161,512.68

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=22877

You need to log in before you can comment on or make changes to this bug.