1452420 - Fix Python 3 compatibility issues found by 'pylint --py3k'

Status: RESOLVED FIXED

(Tree Management :: Treeherder, enhancement, P2)

Description

[Ed Morley [:emorley]]

pylint has a `--py3k` mode that disables all of the standard checks, and runs some additional checks that are not normally run, that look for potential Python 3 incompatibilities.

There are a number of false positives and pylint is also very slow (3-5 minutes to run against the repository depending on what checks are enabled; and without parallelisation since it's pretty broken) - so I don't think it's worth running in CI - but we can at least fix the issues it finds.

Comment 1

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/treeherder/pull/3418

Comment 2

[Treeherder GitHub Bugbot]

Commits pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/5d6cb2371c7f9c4bfccc388b540bddef2fa0cbda
Bug 1452420 - Make .iter{items,keys}() usages Python 3 compatible

Fixes pylint `dict-iter-method` (and many more, since pylint missed
most of them, sigh).

https://github.com/mozilla/treeherder/commit/02d3c625c70577dd90ec5fcb01db320d207d00ee
Bug 1452420 - Cast Python 3 map/filter iterators where necessary

Python 3's `map()`, `filter()` and others now return iterators rather
than list/..., so must be cast back to a `list()` if used in contexts
where an iterator is not supported.

https://github.com/mozilla/treeherder/commit/163068c083a61e1235b370ad5e06869fcad53c27
Bug 1452420 - Replace types.StringTypes with six.string_types

Since the former doesn't exist in Python 3. See:
https://docs.python.org/2/library/types.html#types.StringTypes
https://pythonhosted.org/six/#six.string_types

Fixes pylint `deprecated-types-field`.

https://github.com/mozilla/treeherder/commit/79ce5dfd1a1cb0ceda8b401914ad0e34cfa6b7db
Bug 1452420 - Stop using Exception.message

Since it doesn't exist in Python 3.

Fixes pylint `exception-message-attribute`.

https://github.com/mozilla/treeherder/commit/4281bf2a3278791987c4fdc84882a8219de95f4c
Bug 1452420 - Make ABCMeta __metaclass__ Python 3 compatible

See:
https://pythonhosted.org/six/#six.add_metaclass
https://stackoverflow.com/a/35673504

Fixes pylint `metaclass-assignment`.

https://github.com/mozilla/treeherder/commit/42acc29e6cc5bd974ccff82df7e2a341447cd34b
Bug 1452420 - Add `from __future__ import division`

Adds the future import anywhere we use division, for consistency
between Python 2 and 3. Some instances of division required casting
to int, since the new division always returns float, even when
dividing by in int.

See:
https://eev.ee/blog/2016/07/31/python-faq-how-do-i-port-to-python-3/#division-always-produces-a-float

Fixes pylint `old-division`.

https://github.com/mozilla/treeherder/commit/164a1f8177da526f2df6d9b8af4fc4c7e30646a9
Bug 1452420 - Update pylintrc to reduce noise

We don't run pylint in CI since it's slow and prone to false
positives, however this updated config file makes it less noisy
when running on an adhoc basis locally.

https://github.com/mozilla/treeherder/commit/c8ca4b51d53c043e6153752608be53e5cd0be427
Bug 1452420 - Cast dict .{keys,values}() to list where necessary

Since in Python 3 they return iterators rather than lists, so if
used in contexts where iterators are not supported must first be
cast to `list`.

These cases weren't caught by pylint `dict-keys-not-iterating` and
`dict-values-not-iterating` (since it isn't able to infer the type
of anything but straightforward `dict` usages), but instead by manual
auditing.

Instead of casting `resp.json.keys()` in `test_performance_data_api.py`
the asserts have been removed, since they were duplicating the
coverage provided by the `assert` on the next line.