1532630 - mozprocess broken on python 3.6+

Status: RESOLVED FIXED

(Testing :: Mozbase, defect, P2)

Description

[Tarek Ziadé (:tarek)]

subprocess.Process._wait() can have a timeout option

ProcessHandlerMixin.Process._wait() should have one too

[task 2019-03-05T11:45:18.963Z]  0:52.01 =================================== FAILURES ===================================
[task 2019-03-05T11:45:18.963Z]  0:52.01 ______________ ProcTestPoll.test_poll_after_kill_no_process_group ______________
[task 2019-03-05T11:45:18.963Z]  0:52.01 
[task 2019-03-05T11:45:18.963Z]  0:52.01 self = <test_poll.ProcTestPoll testMethod=test_poll_after_kill_no_process_group>
[task 2019-03-05T11:45:18.963Z]  0:52.01 
[task 2019-03-05T11:45:18.963Z]  0:52.01     def test_poll_after_kill_no_process_group(self):
[task 2019-03-05T11:45:18.963Z]  0:52.01         """Process (no group) is killed, and poll() is called."""
[task 2019-03-05T11:45:18.963Z]  0:52.01         p = processhandler.ProcessHandler([self.python, self.proclaunch,
[task 2019-03-05T11:45:18.963Z]  0:52.01                                            "process_normal_finish_no_process_group.ini"],
[task 2019-03-05T11:45:18.963Z]  0:52.01                                           cwd=here,
[task 2019-03-05T11:45:18.963Z]  0:52.01                                           ignore_children=True
[task 2019-03-05T11:45:18.963Z]  0:52.01                                           )
[task 2019-03-05T11:45:18.963Z]  0:52.01         p.run()
[task 2019-03-05T11:45:18.964Z]  0:52.01 >       returncode = p.kill()
[task 2019-03-05T11:45:18.964Z]  0:52.01 
[task 2019-03-05T11:45:18.964Z]  0:52.01 testing/mozbase/mozprocess/tests/test_poll.py:70:
[task 2019-03-05T11:45:18.964Z]  0:52.01 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[task 2019-03-05T11:45:18.964Z]  0:52.01 testing/mozbase/mozprocess/mozprocess/processhandler.py:812: in kill
[task 2019-03-05T11:45:18.964Z]  0:52.01     self.proc.kill(sig=sig)
[task 2019-03-05T11:45:18.964Z]  0:52.01 testing/mozbase/mozprocess/mozprocess/processhandler.py:208: in kill
[task 2019-03-05T11:45:18.964Z]  0:52.01     self.returncode = self.wait()
[task 2019-03-05T11:45:18.964Z]  0:52.01 testing/mozbase/mozprocess/mozprocess/processhandler.py:229: in wait
[task 2019-03-05T11:45:18.964Z]  0:52.02     self.returncode = self._wait()
[task 2019-03-05T11:45:18.964Z]  0:52.02 testing/mozbase/mozprocess/mozprocess/processhandler.py:665: in _wait
[task 2019-03-05T11:45:18.964Z]  0:52.02     subprocess.Popen.wait(self)
[task 2019-03-05T11:45:18.964Z]  0:52.02 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[task 2019-03-05T11:45:18.964Z]  0:52.02 
[task 2019-03-05T11:45:18.964Z]  0:52.02 self = <mozprocess.processhandler.ProcessHandlerMixin.Process object at 0x10f97a3c8>
[task 2019-03-05T11:45:18.965Z]  0:52.02 timeout = None
[task 2019-03-05T11:45:18.965Z]  0:52.02 
[task 2019-03-05T11:45:18.965Z]  0:52.02     def wait(self, timeout=None):
[task 2019-03-05T11:45:18.965Z]  0:52.02         """Wait for child process to terminate; returns self.returncode."""
[task 2019-03-05T11:45:18.967Z]  0:52.02         if timeout is not None:
[task 2019-03-05T11:45:18.967Z]  0:52.02             endtime = _time() + timeout
[task 2019-03-05T11:45:18.967Z]  0:52.02         try:
[task 2019-03-05T11:45:18.967Z]  0:52.02 >           return self._wait(timeout=timeout)
[task 2019-03-05T11:45:18.967Z]  0:52.02 E           TypeError: _wait() got an unexpected keyword argument 'timeout'
[task 2019-03-05T11:45:18.967Z]  0:52.02 
[task 2019-03-05T11:45:18.967Z]  0:52.02 /tools/python37/lib/python3.7/subprocess.py:984: TypeError

Comment 1

[Tarek Ziadé (:tarek)]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=4386c91d02cd533f2c92d38db1eceb56410bf60e

Comment 2

[Tarek Ziadé (:tarek)]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=53cea0339e8a443fcdd0e4bf6fda4decf7bf7cfb

Comment 3

[Tarek Ziadé (:tarek)]

Attachment: Bug 1532630 - mozprocess broken on python 3.6+ - r?gbrown

Make sure ProcessHandlerMixin.Process works also under 3.6.

3.6 introduced a _wait() method in its Popen class, that
conflicts with the one in mozprocess. The patch renames it
and also adds timeout where possible.

Comment 4

[Geoff Brown [:gbrown]]

(In reply to Tarek Ziadé (:tarek) from comment #2)

https://treeherder.mozilla.org/#/
jobs?repo=try&revision=53cea0339e8a443fcdd0e4bf6fda4decf7bf7cfb

I couldn't see any test results here, nor in comment 1. Here's a new push:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=6cba2afeae02b335900498be6b6a31e709ba4686

Comment 5

[Tarek Ziadé (:tarek)]

try run

https://treeherder.mozilla.org/#/jobs?repo=try&revision=5a1b30c9672a2abf68f26ab8fddadf61fe92b6e3

not that the mozfile failure is unrelated to this change

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b52d5b669b43
mozprocess broken on python 3.6+ - r=gbrown

Comment 7

[Andrei Ciure[:aciure]]

https://hg.mozilla.org/mozilla-central/rev/b52d5b669b43

Comment 9

[Intermittent Failures Robot]

1 failures in 4658 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 1

Platform breakdown:

• macosx64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1532630&startday=2019-05-13&endday=2019-05-19&tree=all