If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

Marionette tests can no longer be run from |multiprocessing.Pool| tasks

RESOLVED WONTFIX

Status

Testing
Marionette
P3
normal
RESOLVED WONTFIX
9 months ago
12 days ago

People

(Reporter: erahm, Unassigned)

Tracking

Trunk
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

AWSY [1] uses a |multiprocessing.Pool| to manage concurrent memory tests [2]. After upgrading from marionette-client==3.1.0 -> marionette-harness==4.0.0 I started seeing the following exception:

> daemonic processes are not allowed to have children

It appears that we've changed how the Firefox process is launched such that it's upsetting |multiprocessing.Pool|. The common recommendation for fixing such an issue is to switch over to a |multiprocessing.pool.ThreadPool| [3], which does indeed fix the exception but leads to test failures (I think Marionette explicitly disallows running multiple tests at once).

I will follow up with a reduced test case.

[1] https://github.com/mozilla/areweslimyet
[2] https://github.com/mozilla/areweslimyet/blob/0b23e6a7a88a079780b3db961cd7d92f064cc9b0/benchtester/BatchTester.py#L260
[3] http://stackoverflow.com/a/34069003
:whimboo, Maja suggested this may have been introduced via bug 1283906.
Flags: needinfo?(hskupin)
Created attachment 8821396 [details]
pool_bug.tar.bz2

This is a reduced example (sorry for the multiple files, that's just how marionette works).

Usage:
  1. Edit pool_bug.py, set |BINARY| to your firefox binary
  2. |./run_test.sh|

Note that 4 browsers are opened, but the test harness bails with the aforementioned exception.
Eric, it would be great if you could add the exception you are seeing as comment. It might help us to better find out what's wrong.

I have to add that I didn't change anything how Firefox gets launched via any of the dependent bugs on the meta bug 1283906. I assume marionette-client==3.2.0 and marionette-client==3.3.0 also fail for you? For a change log of those releases see bug 1309899 and bug 1320073.
Flags: needinfo?(hskupin) → needinfo?(erahm)
(In reply to Henrik Skupin (:whimboo) from comment #3)
> Eric, it would be great if you could add the exception you are seeing as
> comment. It might help us to better find out what's wrong.
> 
> I have to add that I didn't change anything how Firefox gets launched via
> any of the dependent bugs on the meta bug 1283906. I assume
> marionette-client==3.2.0 and marionette-client==3.3.0 also fail for you? For
> a change log of those releases see bug 1309899 and bug 1320073.

You cant test with the reduced test case that is attached.
Flags: needinfo?(erahm)
Sorry, I don't have the time for that right now. If you can do it, it would be of big help. Once we know that a patch from myself caused it, I can make sure to get it fixed. Thanks.
> Traceback (most recent call last):
>   File "pool_bug.py", line 28, in run_test
>     runner.run_tests("test_pool_bug.py")
>   File "/home/erahm/dev/tmp/pool_bug/test_env/local/lib/python2.7/site-packages/marionette_harness/runner/base.py", line 815, in run_tests
>     self.fixture_servers = self.start_fixture_servers()
>   File "/home/erahm/dev/tmp/pool_bug/test_env/local/lib/python2.7/site-packages/marionette_harness/runner/base.py", line 908, in start_fixture_servers
>     return serve.start(root)
>   File "/home/erahm/dev/tmp/pool_bug/test_env/local/lib/python2.7/site-packages/marionette_harness/runner/serve.py", line 176, in start
>     servers = start_servers(doc_root, ssl_config, **kwargs)
>   File "/home/erahm/dev/tmp/pool_bug/test_env/local/lib/python2.7/site-packages/marionette_harness/runner/serve.py", line 156, in start_servers
>     proc.start(doc_root, ssl_config, **kwargs)
>   File "/home/erahm/dev/tmp/pool_bug/test_env/local/lib/python2.7/site-packages/marionette_harness/runner/serve.py", line 112, in start
>     self.proc.start()
>   File "/usr/lib/python2.7/multiprocessing/process.py", line 124, in start
>     'daemonic processes are not allowed to have children'
> AssertionError: daemonic processes are not allowed to have children
Ok, so that's a regression from bug 1103196 then. Andreas is the one you are looking for.
Flags: needinfo?(ato)
A wild guess is that it is because we’re setting multiprocessing.Process.daemon to True here: https://github.com/mozilla/gecko-dev/blob/master/testing/marionette/harness/marionette_harness/runner/serve.py#L111

You should be able to confirm this theory by removing that line and rerunning your existing tests.
Flags: needinfo?(ato)
(In reply to Andreas Tolfsen ‹:ato› from comment #8)
> A wild guess is that it is because we’re setting
> multiprocessing.Process.daemon to True here:
> https://github.com/mozilla/gecko-dev/blob/master/testing/marionette/harness/
> marionette_harness/runner/serve.py#L111
> 
> You should be able to confirm this theory by removing that line and
> rerunning your existing tests.

Testing with |daemon = False| still results in failure. I believe the assertion we're hitting is that the *parent* process cannot be daemonic [1]. Since I'm using a pool the parent is daemonic.

I've provided a reduced test case in attachment 8821396 [details] to make it easier for you to test locally.

[1] https://github.com/python/cpython/blob/2.7/Lib/multiprocessing/process.py#L123-L124
Priority: -- → P3
AWSY no longer uses multiprocessing, lets just close this.
Status: NEW → RESOLVED
Last Resolved: 12 days ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.