1520447 - For raptor tp6 on windows use mitmproxy executable instead of creating a separate python environment

Status: RESOLVED FIXED

(Testing :: Raptor, enhancement, P1)

Description

[Florin Strugariu [:Bebe]]

Mitmproxy provides win executable file now.

https://mitmproxy.org/downloads/#2.0.2/

Update the current setup to use the new executable and remove the python env

Comment 1

[Florin Strugariu [:Bebe]]

Attachment: Bug 1520447 For raptor tp6 on windows use mitmproxy executable instead of creating a separate python environment

Comment 2

[Florin Strugariu [:Bebe]]

to run this we need mitmproxy 2.0.2 to be available in tooltool
https://drive.google.com/open?id=10qyWF-bLGVCik3lwjBXpaa1nq7NZDRia

To test this when running the test on win please copy the above file to obj-****/testing/raptor

Comment 3

[Florin Strugariu [:Bebe]]

The mitmproxy file was downloaded from:
https://mitmproxy.org/downloads/#2.0.2/

Comment 4

[Florin Strugariu [:Bebe]]

:rwood can upload the bin to tool tool?
That would help with the review

Comment 5

[Robert Wood [:rwood]]

(In reply to Florin Strugariu [:Bebe] from comment #4)

:rwood can upload the bin to tool tool?
That would help with the review

Uploaded file (from link in Comment 2) to tooltool; emailed manifest to :bebe.

Comment 6

[Florin Strugariu [:Bebe]]

try push
https://treeherder.mozilla.org/#/jobs?repo=try&revision=35552b25c6b510227e4bd093e26a68dbf01edfc3&selectedJob=226504928

Looks like we have some issues when running the tests.
I will tests on a windows machine

Comment 7

[Florin Strugariu [:Bebe]]

Try run with windows kill command

https://treeherder.mozilla.org/#/jobs?repo=try&revision=a2580754ccdff010e01ed7fc4b0bb79204c1c6fc

Comment 8

[Florin Strugariu [:Bebe]]

did a little bit of research and mozprocess has a class design to handle processes.[1]
Using this we can kill/start the mitmdump process easily

[1]https://searchfox.org/mozilla-central/source/testing/mozharness/mozprocess/processhandler.py#1062

here is the try job
https://treeherder.mozilla.org/#/jobs?repo=try&revision=c599804cd986e79786705d73e2577c19828af3ff

:rwood can you review this change?

Comment 9

[Robert Wood [:rwood]]

(In reply to Florin Strugariu [:Bebe] from comment #8)

did a little bit of research and mozprocess has a class design to handle processes.[1]
Using this we can kill/start the mitmdump process easily

[1]https://searchfox.org/mozilla-central/source/testing/mozharness/mozprocess/processhandler.py#1062

here is the try job
https://treeherder.mozilla.org/#/jobs?repo=try&revision=c599804cd986e79786705d73e2577c19828af3ff

:rwood can you review this change?

I'll have a look at the phabricator review now; please add Raptor tp6 Chromium jobs to your try push ^ thanks!

Comment 10

[Robert Wood [:rwood]]

I'll have a look at the phabricator review now; please add Raptor tp6 Chromium jobs to your try push ^ thanks!

Also since this is a fairly big change, along with the Chromium tp6 please add to your try push:

• at least one Firefox benchmark and one Chromium benchmark job i.e. speedometer

• at least one Raptor tp6 job with --gecko-profile on

• run the raptor android power test

Thanks!

Comment 11

[Florin Strugariu [:Bebe]]

I ran all the tests just to make sure it works

We have 3 failures but I don't think they are related to this patch

I used the same try run:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=c599804cd986e79786705d73e2577c19828af3ff

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a5ad662d5bbb
For raptor tp6 on windows use mitmproxy executable instead of creating a separate python environment r=rwood,davehunt

Comment 13

[Cosmin Sabou [:CosminS]]

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=busted&revision=a5ad662d5bbb4f2f2a8964782e9cdab91385b8ca&searchStr=android&selectedJob=228185007

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=228185007&repo=autoland&lineNumber=34989

Backout link: https://hg.mozilla.org/integration/autoland/rev/23a24796db2bb56ec2172f747fcd1c6dbc23c358

Comment 14

[Cosmin Sabou [:CosminS]]

This looks to have triggered the nightlies to run on autoland: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=usercancel%2Csuccess%2Cpending%2Crunning%2Ctestfailed%2Cbusted%2Cexception&searchStr=nightly&revision=a5ad662d5bbb4f2f2a8964782e9cdab91385b8ca

Florin, please take a look. Thank you.

Comment 15

[Joel Maher ( :jmaher ) (UTC -8)]

I think the nightly builds were added as a result of an add-new job:
https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=usercancel%2Csuccess%2Cpending%2Crunning%2Ctestfailed%2Cbusted%2Cexception&searchStr=decision&revision=a5ad662d5bbb4f2f2a8964782e9cdab91385b8ca&selectedJob=228184946

Comment 16

[Florin Strugariu [:Bebe]]

Yes I triggered by mistake those jobs.

I was trying to run all raptor jobs to make sure this change works as expected

Comment 17

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/00df850ac3cf
For raptor tp6 on windows use mitmproxy executable instead of creating a separate python environment r=rwood,davehunt

Comment 18

[Stefan Hindli [:stefan_hindli]]

https://hg.mozilla.org/mozilla-central/rev/00df850ac3cf