1428711 - [moznetwork] Add support for Python 3

Status: RESOLVED FIXED

(Testing :: Mozbase, enhancement, P3)

Description

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Without dropping support for legacy Python, we need to add support for Python 3 to moznetwork.

Comment 1

[Dave Hunt [:davehunt] [he/him] ⌚BST]

To work on this bug you will need to install and configure Mercurial, which will enable you to download the Firefox source code. It will also be used to commit your changes locally and prepare a patch for review. See the installation guide at https://mozilla-version-control-tools.readthedocs.io/en/latest/hgmozilla/installing.html for help getting Mercurial on your system. Once installed, there are some extensions we recommend installing, details of these can be found here: https://mozilla-version-control-tools.readthedocs.io/en/latest/hgmozilla/extensions.html

To clone the Firefox source code we recommend using the unified repository. Details of this and how to create a bookmark for your work can be found here: https://mozilla-version-control-tools.readthedocs.io/en/latest/hgmozilla/unifiedrepo.html

As this bug relates to Python 3, you will need to have this installed on your system. Our continuous integration is currently using Python 3.5, so for best results we recommend using the same version locally. There are a number of ways to install Python, and they vary depending on your environment. We suggest reading over the guides at http://docs.python-guide.org/en/latest/starting/installation/#python-3-installation-guides to find the best method for you. Note that we also need to maintain support for Python 2, so you'll also need to have Python 2.7 installed.

Whilst we're moving towards adding support for Python 3, we've disabled any tests that fail against this version. This means that in order to work on this bug you will need to enable the tests. This can be done by removing "skip-if = python == 3" from the manifest file in `testing/mozbase/moznetwork/tests/manifest.ini`

To run the tests against Python 3, execute the following command:

```
mach python-test --python=3.5 testing/mozlog/moznetwork
```

Work through the test failures, and update the tests and source code for the package to simultaneously support Python 2.7 and Python 3.5. You can always check that your changes have not inadvertently broken support for Python 2 by using `--python=2.7` on the command line. 

If you're new to Python 3, the guide at https://docs.python.org/3/howto/pyporting.html may help you to get started with understanding the differences. To assist with supporting both Python 2 and 3, we have vendored the "six" package. You can read more about this and how to use it at https://pythonhosted.org/six/

Note that this package depends on others that may not yet support Python 3. If these dependencies are preventing tests from passing, then we can block this bug on getting support in those packages. If there are circular dependencies then we may need to coordinate landing a sequence of patches between bugs,

Once the package supports Python 3 and the tests pass, we will also need to prepare for a new release. Bump the version number in `testing/mozbase/moznetwork/setup.py` to "1.0.0", and update the classifiers so they reflect the versions of Python that we now support.

Finally, to prepare a universal distribution, create a `testing/mozbase/moznetwork/setup.cfg` file with the following contents:

```
[bdist_wheel]
universal=1
```

When your patch(es) are ready, you will need to push them for review. We are use MozReview for this, and instructions for how to prepare your machine and how to structure your commit messages can be found in our guide: https://mozilla-version-control-tools.readthedocs.io/en/latest/mozreview-user.html

There may be some review issues to address, but once your contribution has been approved and subsequently landed, we will release the new version of the package!

Comment 2

[tanya16108]

Hi I am a new conributor and want to work on this bug. Has it already been solved? If not please assign me to solve this bug.

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

tanya16108, you are welcome to work on this bug. No one else has chosen to work on it yet. Once you upload the first patch, the bug will be assigned to you. Are you still interested to work on it?

Comment 4

[hafsa]

Hello ! I'm an Outreachy applicant. Can I still claim this bug ?

Comment 5

[Dave Hunt [:davehunt] [he/him] ⌚BST]

(In reply to hafsa from comment #4)

Hello ! I'm an Outreachy applicant. Can I still claim this bug ?

I would suggest working on one bug at a time. As tanya16108 has also expressed interest in this bug, perhaps bug 1428716 would be a better one for yourself.

Comment 6

[Erick O.]

Hi,
I know people have expressed interest in fixing this bug in the past, but six months have passed since the last activity on this thread.
I am wondering if I could start working on a fix or should I try to claim another bug.
Thanks!

Comment 7

[Shivam Singhal [ :championshuttler ]]

I am on it. FYI

Thanks

Comment 8

[Edwin Takahashi (:egao | infrequent contributor)]

Attachment: Bug 1428711 - add python3 compatibility to moznetwork

Comment 9

[Edwin Takahashi (:egao | infrequent contributor)]

Try push:

https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&revision=a90594f48510441468980c65ed5f7433763c64a2

Logs

linux64/opt reftest using the updated moznetwork (versioned 0.3.0, will update to 1.0.0 for the patch)
https://taskcluster-artifacts.net/EJ8O_zYFRkmMzlT42HN1IA/0/public/logs/live_backing.log

Excerpt from the reftest log:

[task 2019-11-06T22:55:20.930Z] 22:55:20     INFO - Current package versions:
[task 2019-11-06T22:55:20.930Z] 22:55:20     INFO -   blessings == 1.7
[task 2019-11-06T22:55:20.931Z] 22:55:20     INFO -   browsermob-proxy == 0.8.0
[task 2019-11-06T22:55:20.931Z] 22:55:20     INFO -   certifi == 2018.4.16
[task 2019-11-06T22:55:20.931Z] 22:55:20     INFO -   chardet == 3.0.4
[task 2019-11-06T22:55:20.931Z] 22:55:20     INFO -   distro == 1.4.0
[task 2019-11-06T22:55:20.932Z] 22:55:20     INFO -   enum34 == 1.1.6
[task 2019-11-06T22:55:20.932Z] 22:55:20     INFO -   firefox-puppeteer == 52.1.0
[task 2019-11-06T22:55:20.932Z] 22:55:20     INFO -   functools32 == 3.2.3.post2
[task 2019-11-06T22:55:20.932Z] 22:55:20     INFO -   h2 == 3.0.1
[task 2019-11-06T22:55:20.935Z] 22:55:20     INFO -   hpack == 3.0.0
[task 2019-11-06T22:55:20.935Z] 22:55:20     INFO -   hyperframe == 5.1.0
[task 2019-11-06T22:55:20.935Z] 22:55:20     INFO -   idna == 2.8
[task 2019-11-06T22:55:20.935Z] 22:55:20     INFO -   jsonschema == 2.5.1
[task 2019-11-06T22:55:20.935Z] 22:55:20     INFO -   manifestparser == 1.2
[task 2019-11-06T22:55:20.935Z] 22:55:20     INFO -   marionette-driver == 3.0.0
[task 2019-11-06T22:55:20.936Z] 22:55:20     INFO -   marionette-harness == 5.0.0
[task 2019-11-06T22:55:20.936Z] 22:55:20     INFO -   mock == 1.0.1
[task 2019-11-06T22:55:20.936Z] 22:55:20     INFO -   mozInstall == 2.0.0
[task 2019-11-06T22:55:20.936Z] 22:55:20     INFO -   mozcrash == 1.1.0
[task 2019-11-06T22:55:20.936Z] 22:55:20     INFO -   mozdebug == 0.2.0
[task 2019-11-06T22:55:20.936Z] 22:55:20     INFO -   mozdevice == 3.0.5
[task 2019-11-06T22:55:20.936Z] 22:55:20     INFO -   mozfile == 2.1.0
[task 2019-11-06T22:55:20.936Z] 22:55:20     INFO -   mozhttpd == 0.7
[task 2019-11-06T22:55:20.936Z] 22:55:20     INFO -   mozinfo == 1.2.0
[task 2019-11-06T22:55:20.936Z] 22:55:20     INFO -   mozleak == 1.0.0
[task 2019-11-06T22:55:20.936Z] 22:55:20     INFO -   mozlog == 5.0
[task 2019-11-06T22:55:20.936Z] 22:55:20     INFO -   moznetwork == 0.3.0
[task 2019-11-06T22:55:20.936Z] 22:55:20     INFO -   mozpower == 1.0.0
[task 2019-11-06T22:55:20.936Z] 22:55:20     INFO -   mozprocess == 1.0.0
[task 2019-11-06T22:55:20.937Z] 22:55:20     INFO -   mozprofile == 2.4.0
[task 2019-11-06T22:55:20.937Z] 22:55:20     INFO -   mozproxy == 1.0
[task 2019-11-06T22:55:20.937Z] 22:55:20     INFO -   mozrunner == 7.7.0
[task 2019-11-06T22:55:20.937Z] 22:55:20     INFO -   mozscreenshot == 1.0.0
[task 2019-11-06T22:55:20.937Z] 22:55:20     INFO -   mozsystemmonitor == 0.4
[task 2019-11-06T22:55:20.937Z] 22:55:20     INFO -   mozterm == 1.0.0
[task 2019-11-06T22:55:20.937Z] 22:55:20     INFO -   moztest == 0.8
[task 2019-11-06T22:55:20.937Z] 22:55:20     INFO -   mozversion == 2.2.0
[task 2019-11-06T22:55:20.937Z] 22:55:20     INFO -   psutil == 5.4.3
[task 2019-11-06T22:55:20.937Z] 22:55:20     INFO -   requests == 2.22.0
[task 2019-11-06T22:55:20.937Z] 22:55:20     INFO -   simplejson == 3.3.0
[task 2019-11-06T22:55:20.937Z] 22:55:20     INFO -   six == 1.12.0
[task 2019-11-06T22:55:20.937Z] 22:55:20     INFO -   urllib3 == 1.25.3
[task 2019-11-06T22:55:20.937Z] 22:55:20     INFO -   wptserve == 2.0

Test chunk passed as expected.

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/520bfdde413e
add python3 compatibility to moznetwork r=ahal

Comment 11

[Stefan Hindli [:stefan_hindli]]

https://hg.mozilla.org/mozilla-central/rev/520bfdde413e

Comment 12

[Henrik Skupin [:whimboo][⌚️UTC+2]]

And the version got already pushed to PyPI: https://pypi.org/project/moznetwork/1.0.0/