[mozprocess] Add support for Python 3
Categories
(Testing :: Mozbase, defect, P3)
Tracking
(firefox67 fixed)
Tracking | Status | |
---|---|---|
firefox67 | --- | fixed |
People
(Reporter: whimboo, Assigned: ahutusoru, Mentored)
References
Details
(Keywords: good-first-bug)
Attachments
(4 files, 2 obsolete files)
Comment 1•7 years ago
|
||
Updated•7 years ago
|
Comment 2•6 years ago
|
||
Updated•6 years ago
|
Comment 3•6 years ago
|
||
Comment 4•6 years ago
|
||
Comment hidden (mozreview-request) |
Comment 6•6 years ago
|
||
Comment 7•6 years ago
|
||
mozreview-review |
Comment 8•6 years ago
|
||
mozreview-review |
Comment hidden (mozreview-request) |
Comment 10•6 years ago
|
||
Comment 11•6 years ago
|
||
Comment 12•6 years ago
|
||
Comment 13•6 years ago
|
||
Comment 14•6 years ago
|
||
Comment 15•6 years ago
|
||
Comment hidden (mozreview-request) |
Updated•6 years ago
|
Comment 17•6 years ago
|
||
Comment 18•6 years ago
|
||
mozreview-review |
Comment 19•6 years ago
|
||
Comment 20•6 years ago
|
||
Comment 21•6 years ago
|
||
Comment 22•6 years ago
|
||
Comment hidden (mozreview-request) |
Comment 25•6 years ago
|
||
mozreview-review |
Comment 26•6 years ago
|
||
Comment 27•6 years ago
|
||
bugherder |
Comment 28•6 years ago
|
||
Reporter | ||
Comment 29•6 years ago
|
||
Comment 30•6 years ago
|
||
Updated•6 years ago
|
Updated•6 years ago
|
Updated•6 years ago
|
Reporter | ||
Updated•6 years ago
|
Comment 32•6 years ago
|
||
Comment 33•6 years ago
|
||
Comment 34•6 years ago
|
||
Comment 35•6 years ago
|
||
Reporter | ||
Comment 36•6 years ago
|
||
Updated•6 years ago
|
Comment 37•6 years ago
|
||
Comment 38•6 years ago
|
||
Comment 39•6 years ago
|
||
Comment 40•6 years ago
|
||
Comment 41•6 years ago
|
||
Comment 42•6 years ago
|
||
Assignee | ||
Comment 43•6 years ago
|
||
Assignee | ||
Comment 44•6 years ago
|
||
Assignee | ||
Comment 45•6 years ago
|
||
Hello team,
I have updated Mozprocess to run with Python 3.5.3 also.
Because of Python BUG: https://bugs.python.org/issue32745 which affects all new versions after 3.5.3, the tests pass only for this version, as per comment https://bugzilla.mozilla.org/show_bug.cgi?id=1428713#c37.
Please review and let me know what you think so that we can land this. Thanks !
Comment 46•6 years ago
|
||
Thanks AndreiH, looks like there's a failure in the python 3 tests on Windows. Here's the log:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=223367462&repo=try&lineNumber=817
Here's my push:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=f317f2af5628b0b979cdef1be0a58c50079fbfc4
At least it looks like all the tests are failing with the same error. We could maybe schedule the Linux/OSX tasks in the meantime.
Assignee | ||
Comment 47•6 years ago
|
||
Hello Andrew !
Yes, this is because Python BUG: https://bugs.python.org/issue32745 which affects Python versions 3.6, 3.7 and 3.8 -> all new versions after 3.5.3. I tested locally with 3.5.3 and works fine, see previous comments.
Also discussed in above comments from: https://bugzilla.mozilla.org/show_bug.cgi?id=1428713#c38
Dave Hunt suggested that we should move forward with these modifications and tackle this specific bug in future modifications, when the bug is resolved. I also spoke with Dave Hunt in private.
Please see previous discussions for more details.
Thank you again for the review !
Comment 48•6 years ago
|
||
(In reply to AndreiH from comment #47)
Yes, this is because Python BUG: https://bugs.python.org/issue32745 which affects Python versions 3.6, 3.7 and 3.8 -> all new versions after 3.5.3. I tested locally with 3.5.3 and works fine, see previous comments.
Also discussed in above comments from: https://bugzilla.mozilla.org/show_bug.cgi?id=1428713#c38
Dave Hunt suggested that we should move forward with these modifications and tackle this specific bug in future modifications, when the bug is resolved. I also spoke with Dave Hunt in private.
Ah sorry, I missed that. However you'll need to modify your patch so that this failure doesn't show up otherwise your change will get backed out. Please add skip-if lines to the manifest so that the appropriate tests get skipped if we're on Windows with python 3.
Comment 49•6 years ago
|
||
Yes, let's skip the tests on Windows for Python 3. This will at least unblock other packages that depend on mozprocess from upgrading to Python 3 if they're not running on Windows. It looks like the Python bug affecting 3.5.4 and later hasn't had much activity in almost a year. I wonder if there's a workaround we could consider?
Assignee | ||
Comment 50•6 years ago
|
||
Thank you for the notification!
I have updated the code with the required skip-if = python == 3 && os == "win"
.
I don't know if there is a workaround for this without complicating things.
Let me know if there is something more to be done. Thanks!
Comment 51•6 years ago
|
||
It might be worth digging into why we created mozprocess in the first place (I believe circa python 2.4), and examine whether those reasons are still valid in 2019. That's a question for another bug and another time however.
Reporter | ||
Comment 52•6 years ago
|
||
As long as we have to support Python2.7 we still need it because of missing features especially on Windows. When we are on Python3 only, the world might look different, and that evaluation should clearly take place.
Comment 53•6 years ago
|
||
(In reply to Henrik Skupin (:whimboo) [⌚️UTC+1] from comment #52)
As long as we have to support Python2.7 we still need it because of missing features especially on Windows. When we are on Python3 only, the world might look different, and that evaluation should clearly take place.
We're already moving to Python 3, so I wouldn't want to wait until we're only Python 3 before assessing this. Perhaps if mozprocess is no longer needed with Python 3 we can remove the dependencies in the packages that use mozprocess? I suspect mozrunner would be a good place to start if we're considering this.
Reporter | ||
Comment 54•6 years ago
|
||
Again, I cannot say without further inspection. The IO completion port additions are way to complex, and maybe there is still some customization needed compared to the standard library modules. Best is to file a separate bug.
Assignee | ||
Comment 55•6 years ago
|
||
Updated•6 years ago
|
Updated•6 years ago
|
Assignee | ||
Comment 56•6 years ago
|
||
Updated•6 years ago
|
Updated•6 years ago
|
Comment 57•6 years ago
|
||
Comment 58•6 years ago
|
||
Backed out changeset 9c17fddb650f (bug 1428713) for mozprocess raptor failures
push that caused the backout: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception&classifiedState=unclassified&selectedJob=225471491&revision=9c17fddb650f2c2862bb8b27ce57b21267c89d81
backout: https://hg.mozilla.org/integration/autoland/rev/c82f1552a5f4b8fe67fe3a9b205f46fd19601d66
Comment 59•6 years ago
|
||
We need to update the version dependency for mozrunner to include mozprocess 1.0.0 in https://searchfox.org/mozilla-central/source/testing/mozbase/mozrunner/setup.py#19 we could just change this to mozprocess>=0.23,<2
Updated•6 years ago
|
Assignee | ||
Comment 60•6 years ago
•
|
||
(In reply to Dave Hunt [:davehunt] [he/him] ⌚️UTC from comment #59)
We need to update the version dependency for mozrunner to include mozprocess 1.0.0 in https://searchfox.org/mozilla-central/source/testing/mozbase/mozrunner/setup.py#19 we could just change this to mozprocess>=0.23,<2
So that's why it has been backed out.
Thank you :davehunt!
I have created a new revision, this time also updating the Dependency in mozrunner.
Please see it here: https://phabricator.services.mozilla.com/D18048
:ahal, can you please give it another look ? Thank you !
Comment 61•6 years ago
|
||
Comment 62•6 years ago
|
||
bugherder |
Comment 63•6 years ago
|
||
mozprocess 1.0.0 has been released to PyPI: https://pypi.org/project/mozprocess/1.0.0/
Updated•6 years ago
|
Comment 64•5 years ago
|
||
I filed a bug for the mozprocess on Windows error:
https://bugzilla.mozilla.org/show_bug.cgi?id=1585702
Reporter | ||
Comment 65•5 years ago
|
||
There is also bug 1597687 for: "TypeError: write() argument must be str, not bytes"
Description
•