1920821 - Enable mitm11 on Windows/MacOS

Status: RESOLVED FIXED

(Testing :: Performance, task, P1)

Description

[Kash Shampur [:kshampur] ⌚EST]

As stated in Bug 1883701, for desktop platforms we can't (yet) run mitm 10 on Linux, but we could still enable it for Win/MacOS in the mean time.
One thing is for the recording task, we can/should also add config options to be able to record on win/mac as well, since the current recording task uses linux.

Comment 1

[Kash Shampur [:kshampur] ⌚EST]

Attachment: Bug 1920821 - Enable mitmproxy 11.0.0 on Windows/MacOS r?#perftest

Comment 2

[Kash Shampur [:kshampur] ⌚EST]

Attachment: WIP: Bug 1920821 - Bump mitmproxy version for Android to 11.0.0

Comment 3

[Kash Shampur [:kshampur] ⌚EST]

Attachment: WIP: Bug 1920821 - Bump mitmproxy version for Android to 11.0.0

This patch updates the mitmproxy binary version from 10 -> 11 for
Android. 10.x.x was only recently landed but 11 was released shortly
afterwards and we might as well keep everything up to date on all
platforms where possible, as this is being done for Desktop

Comment 4

[Kash Shampur [:kshampur] ⌚EST]

Attachment: Bug 1920821 - Enable recording with mitmproxy 11 on Win/Mac. r?#perftest

This patch enables the ability to record pageload tests on platforms on
Win/Mac. Previously this was only available on Linux. Until the ubuntu
version on the linux platforms are updated, that is restricted to mitm8
recordings, meanwhile 11 can be used for win/mac.

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/48ee1eb36afa
Enable mitmproxy 11.0.0 on Windows/MacOS r=perftest-reviewers,sparky
https://hg.mozilla.org/integration/autoland/rev/0781de425698
Enable recording with mitmproxy 11 on Win/Mac. r=perftest-reviewers,afinder

Comment 6

[amarc]

https://hg.mozilla.org/mozilla-central/rev/48ee1eb36afa
https://hg.mozilla.org/mozilla-central/rev/0781de425698

Comment 7

[Mayank Bansal]

For whatever reason, this bug (or bug 1925145) lead to 10% improvement on some of the "image tab closed" AWSY tests on Linux only
Metric1, Metric2, Metric3

Comment 8

[Alex Finder]

(In reply to Pulsebot from comment #5)

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/48ee1eb36afa
Enable mitmproxy 11.0.0 on Windows/MacOS r=perftest-reviewers,sparky
https://hg.mozilla.org/integration/autoland/rev/0781de425698
Enable recording with mitmproxy 11 on Win/Mac. r=perftest-reviewers,afinder

As mentioned by Mayank in the previous comment, it seems this changeset caused an improvement on AWSY.

Perfherder has detected a awsy performance change from push 0781de425698668c4d22cb127a7022122b26bb36.

Improvements:

Ratio	Test	Platform	Options	Absolute values (old vs new)
9%	Images Tabs closed [+30s]	linux1804-64-shippable-qr	fission tp6	2,174,064.00 -> 1,988,156.00
8%	Images Tabs closed extra processes	linux1804-64-shippable-qr	fission tp6	2,643,578.67 -> 2,420,908.00
8%	Images Tabs closed	linux1804-64-shippable-qr	fission tp6	2,643,597.33 -> 2,420,908.00

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests.

If you need the profiling jobs you can trigger them yourself from treeherder job view or ask a sheriff to do that for you.

You can run these tests on try with ./mach try perf --alert 43242

For more information on performance sheriffing please see our FAQ.

Comment 9

[Kash Shampur [:kshampur] ⌚EST]

Thanks for looking into this Alex & Mayank!

So looks like this was an accidental change. Because our current linux machines don't support mitmproxy 11, I had implemented logic in this patch that defaults to 8.1.1 when running on the Linux platform e.g. https://searchfox.org/mozilla-central/rev/b6718bf263ae780289da471f0ea35797a09e2f05/testing/mozbase/mozproxy/mozproxy/server.py#18-24

But previously awsy has been using mitmproxy 5, so my changes here accidentally bumped that to 8.1.1

(worth noting, I doubt this would work on Mac/Windows if we bumped awsy to use mitmproxy 11, since the recordings are quite old, but probably would work for 8.1.1)

Good to see it is passing still (accidentally). This is somewhat related to Bug 1802602 where we want to update awsy's mitmproxy version, so it is reassuring to see it works.

I wouldn't pay much mind to the actual improvements numbers though - as this is a change in the tools version (mitmproxy) the baseline is likely expected to change. As long as the new data is stable we are good.
(And as I had already found in the original patch, the pageloads looked fine)