1428707 - [mozdebug] Add support for Python 3

Status: RESOLVED FIXED

(Testing :: Mozbase, enhancement, P3)

Description

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Without dropping support for legacy Python, we need to add support for Python 3 to mozdebug.

Comment 1

[Dave Hunt [:davehunt] [he/him] ⌚BST]

To work on this bug you will need to install and configure Mercurial, which will enable you to download the Firefox source code. It will also be used to commit your changes locally and prepare a patch for review. See the installation guide at https://mozilla-version-control-tools.readthedocs.io/en/latest/hgmozilla/installing.html for help getting Mercurial on your system. Once installed, there are some extensions we recommend installing, details of these can be found here: https://mozilla-version-control-tools.readthedocs.io/en/latest/hgmozilla/extensions.html

To clone the Firefox source code we recommend using the unified repository. Details of this and how to create a bookmark for your work can be found here: https://mozilla-version-control-tools.readthedocs.io/en/latest/hgmozilla/unifiedrepo.html

As this bug relates to Python 3, you will need to have this installed on your system. Our continuous integration is currently using Python 3.5, so for best results we recommend using the same version locally. There are a number of ways to install Python, and they vary depending on your environment. We suggest reading over the guides at http://docs.python-guide.org/en/latest/starting/installation/#python-3-installation-guides to find the best method for you. Note that we also need to maintain support for Python 2, so you'll also need to have Python 2.7 installed.

Whilst we're moving towards adding support for Python 3, we've disabled any tests that fail against this version. This means that in order to work on this bug you will need to enable the tests. This can be done by removing "skip-if = python == 3" from the manifest file in `testing/mozbase/mozdebug/tests/manifest.ini`

To run the tests against Python 3, execute the following command:

```
mach python-test --python=3.5 testing/mozbase/mozdebug
```

Work through the test failures, and update the tests and source code for the package to simultaneuously support Python 2.7 and Python 3.5. You can always check that your changes have not inadvertantly broken support for Python 2 by using `--python=2.7` on the command line. 

If you're new to Python 3, the guide at https://docs.python.org/3/howto/pyporting.html may help you to get started with understanding the differences. To assist with supporting both Python 2 and 3, we have vendored the "six" package. You can read more about this and how to use it at https://pythonhosted.org/six/

Note that this package depends on others that may not yet support Python 3. If these dependencies are preventing tests from passing, then we can block this bug on getting support in those packages. If there are circular dependencies then we may need to coordinate landing a sequence of patches between bugs,

Once the package supports Python 3 and the tests pass, we will also need to prepare for a new release. Bump the version number in `testing/mozbase/mozdebug/setup.py` to "1.0.0", and update the classifiers so they reflect the versions of Python that we now support.

Finally, to prepare a universal distribution, create a `testing/mozbase/mozdebug/setup.cfg` file with the following contents:

```
[bdist_wheel]
universal=1
```

When your patch(es) are ready, you will need to push them for review. We are use MozReview for this, and instructions for how to prepare your machine and how to structure your commit messages can be found in our guide: https://mozilla-version-control-tools.readthedocs.io/en/latest/mozreview-user.html

There may be some review issues to address, but once your contribution has been approved and subsequently landed, we will release the new version of the package!

Comment 2

[Arshad Kazmi [:arshadkazmi42]]

I was giving it a try. 

I tried running this after removing "skip-if = python == 3"

```
mach python-test --python=2.7 testing/mozbase/mozdebug
mach python-test --python=3.7 testing/mozbase/mozdebug
```

All tests have passed for both the python versions. 
Also I found only one test file in mozdebug.
Is that all which needs to be validated? 
Or the whole mozabase directory needs to be validated for this?

Comment 3

[Dave Hunt [:davehunt] [he/him] ⌚BST]

For this bug we only need to run the mozdebug tests. It is a concern that there's only one test, but it's outside the scope of this bug to address that. Note that the versions we need to test against are 2.7 and 3.5 (rather than 3.7). If these are passing for you then please feel free to take care of the other changes mentioned in comment 0 and submit a patch for review.

Comment 4

[Arshad Kazmi [:arshadkazmi42]]

Done with the validations.

Tests are passed with both these commands

```
mach python-test --python=2.7 testing/mozbase/mozdebug
mach python-test --python=3.5 testing/mozbase/mozdebug
```

Should I push the code after removing this `skip-if = python == 3` from manifest.ini file?

Comment 5

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Yes, however please note there are other changes mentioned in comment 0, such as bumping the version number, updating the trove classifiers in setup.py, and marking the package as universal.

Comment 6

[Arshad Kazmi [:arshadkazmi42]]

Few things,

I have not made any code change, just tested the script by removing `skip-if = python == 3` from manifest.ini file and it passed.


For this 

> Once the package supports Python 3 and the tests pass, we will also need to prepare for a new release. Bump the version number > in `testing/mozbase/mozdebug/setup.py` to "1.0.0", and update the classifiers so they reflect the versions of Python that we 
> now support.
> 
> Finally, to prepare a universal distribution, create a `testing/mozbase/mozdebug/setup.cfg` file with the following contents:
> 
> ```
> [bdist_wheel]
> universal=1
> ```

Do I need to make these changes before pushing the code?

Should I push the code to Phabricator or MozReview? I have been using Phabricator for my code pushes for other components?

Comment 7

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Hmm.. the classifiers and setup.cfg changes will still be needed, but you make a good point that if you've made no code changes then it doesn't justify a major version bump. That said, in order to release a version with the necessary metadata to indicate that Python 3 is supported, we will need to bump the version. Let's make it a minor bump as we're just enabling tests and adding this metadata. As the current version is 0.1, let's make the new version 0.1.1.

Please use Phabricator, as MozReview has now been decomissioned.

Comment 8

[Arshad Kazmi [:arshadkazmi42]]

Attachment: Validated python3 support, updated package version

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/aa2736811468
Validated python3 support, updated package version r=davehunt

Comment 10

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Marking as leave-open so we push the version to pypi before the bug gets closed.

Comment 11

[Stefan Hindli [:stefan_hindli]]

https://hg.mozilla.org/mozilla-central/rev/aa2736811468

Comment 12

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Sorry for not spotting this during review, but the setup.cfg file needs to be located in testing/mozbase/mozdebug rather than testing/mozbase/mozdebug/mozdebug. Could you prepare a follow-up patch that moves this file? Meanwhile, I've discovered that I don't have permission to publish this package on PyPI, so I'll work on getting that.

Comment 13

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Okay, I got access and published 0.1.1 to PyPI. I'll leave this open for the setup.cfg follow-up as mentioned in comment 12. For the release I simple moved the file in my workspace.

https://pypi.org/project/mozdebug/0.1.1/

Comment 14

[Arshad Kazmi [:arshadkazmi42]]

Dave,
I have updated the patch.
Moved the file to testing/mozbase/mozdebug

Comment 15

[Arshad Kazmi [:arshadkazmi42]]

I have updated the same patch with the changes. Is that fine? Or should I create a new patch?

Comment 16

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Your patch already landed, so it would need to be a new patch. Thanks.

Comment 17

[Arshad Kazmi [:arshadkazmi42]]

Attachment: Bug 1428707 - Move testing/mozbase/mozdebug/mozdebug/setup.cfg to testing/mozbase/mozdebug/setup.cfg

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/6e0d21ef7ec6
Move testing/mozbase/mozdebug/mozdebug/setup.cfg to testing/mozbase/mozdebug/setup.cfg r=davehunt

Comment 19

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/6e0d21ef7ec6