1590137 - Raptor http proxy policies.json not reset if run interrupted (Windows)

Status: NEW

(Testing :: Raptor, defect, P3)

Description

[Andrew Creskey [:acreskey]]

I noticed that the live sites mode of the raptor tp6 pageload tests on Windows configures the https proxy within Firefox and ends up not loading any content.

Steps:
• Add the use_live_sites = true option to any test file, for instance
testing/raptor/raptor/tests/tp6/desktop/raptor-tp6-1-cold.ini
• Start the test
./mach raptor-test --test raptor-tp6-amazon-firefox-cold

Results:
Firefox will launch configured to use a local https proxy. mitmproxy will correctly not be started, and as a result no page will be loaded.

Note that the framework is detecting that it should be running in live mode because I saw this in the logs:
raptor-manifest Info: using live sites so turning playback off!

Comment 1

[Andrew Creskey [:acreskey]]

Note that the live sites features is working correctly on OSX (Firefox and Chrome) and also for Chrome on Windows.

Comment 2

[Andrew Creskey [:acreskey]]

Rob, if you can point me to where the proxy configuration is inserted I should be able to make sure that it doesn't do so for live sites on Windows.

Comment 3

[Robert Wood [:rwood]]

Thanks for spotting this Andrew! On desktop the proxy configuration is set in a Firefox policies file, inside of mozproxy:

https://searchfox.org/mozilla-central/rev/55aa17110091deef24b913d033ccaf58f9c6d337/testing/mozbase/mozproxy/mozproxy/backends/mitm/mitm.py#304

Comment 4

[Robert Wood [:rwood]]

(In reply to Robert Wood [:rwood] from comment #3)

Thanks for spotting this Andrew! On desktop the proxy configuration is set in a Firefox policies file, inside of mozproxy:

https://searchfox.org/mozilla-central/rev/55aa17110091deef24b913d033ccaf58f9c6d337/testing/mozbase/mozproxy/mozproxy/backends/mitm/mitm.py#304

(Maybe there is an old policies file still there, from a previous run with recorded sites?)

Comment 5

[Andrew Creskey [:acreskey]]

Thanks Rob - that is a good lead.

Comment 6

[Andrew Creskey [:acreskey]]

Yes, that's what's causing the problem:

If I cancel a run on Windows (Ctrl-C) the mitmproxy shutdown code isn't being executed and so the old proxy policies.json is left behind.

On OSX the Ctrl-C interrupt does end up running this shutdown code and restores the default policies.json:
https://searchfox.org/mozilla-central/rev/8a63fc190b39ed6951abb4aef4a56487a43962bc/testing/mozbase/mozproxy/mozproxy/backends/mitm/mitm.py#392

(I'll update the bug title)
So I think the question is, why doesn't the KeyboardInterrupt let raptor shutdown on Windows?

On OSX, these are logs after a KeyboardInterrupt:

^C20:38:30    ERROR - Process interrupted by the user, killing process with pid 43028
Process Process-1:
Traceback (most recent call last):
  File "/Users/acreskey/.pyenv/versions/2.7.11/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/Users/acreskey/.pyenv/versions/2.7.11/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/acreskey/dev/src/mozilla-central/testing/mozbase/mozsystemmonitor/mozsystemmonitor/resourcemonitor.py", line 121, in _collect
    while not _poll(pipe, poll_interval=sleep_interval):
  File "/Users/acreskey/dev/src/mozilla-central/testing/mozbase/mozsystemmonitor/mozsystemmonitor/resourcemonitor.py", line 87, in _poll
    return pipe.poll(poll_interval)
KeyboardInterrupt
20:38:30     INFO -  raptor-mitmproxy Info: Stopping mitmproxy playback, killing process 43033
20:38:31     INFO -  raptor-mitmproxy Info: Successfully killed the mitmproxy playback process
20:38:31     INFO -  raptor-mitmproxy Info: Turning off the browser proxy
20:38:31     INFO -  raptor-mitmproxy Info: writing: /Users/acreskey/dev/src/build/obj-release/dist/Nightly.app/Contents/Resources/distribution/policies.json

But on Windows:

20:33:56     INFO -  PID 16564 | console.log: "[raptor-runnerjs] post success"
Process Process-1:
2T0:33:59    ERROR - Process interrupted by the user, killing process with pid 9416
raceback (most recent call last):
  File "c:\mozilla-build\python\lib\multiprocessing\process.py", line 267, in _bootstrap
    self.run()
  File "c:\mozilla-build\python\lib\multiprocessing\process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "c:\mozilla-source\mozilla-central\testing/mozbase/mozsystemmonitor\mozsystemmonitor\resourcemonitor.py", line 121, in _collect
    while not _poll(pipe, poll_interval=sleep_interval):
  File "c:\mozilla-source\mozilla-central\testing/mozbase/mozsystemmonitor\mozsystemmonitor\resourcemonitor.py", line 87, in _poll
    return pipe.poll(poll_interval)
KeyboardInterrupt