Make much of mozbuild/mozbuild and mozbuild/mozpack py3 compatible, ensuring mozpack.mozjar can run on py3

RESOLVED FIXED in Firefox 69

Status

enhancement
RESOLVED FIXED
4 months ago
12 days ago

People

(Reporter: Callek, Assigned: Callek)

Tracking

(Blocks 1 bug)

Trunk
mozilla69
Dependency tree / graph

Firefox Tracking Flags

(firefox69 fixed)

Details

Attachments

(28 attachments, 2 obsolete attachments)

47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
No description provided.
Type: defect → enhancement

This is necessary for any attempt at @import('six') or doing any similar py2+py3 work
in the configure sandbox.

shellutil.py needed special handling since it is invoked earlier than the
vendor directory adding we do, so it does not have six available.

(adds build_class to the builtin sandbox and allow CombinedDependsFunction to be hashed)

Attachment #9062099 - Attachment is obsolete: true
Attachment #9062098 - Attachment is obsolete: true
Attachment #9062093 - Attachment description: Bug 1547730 - Define __hash__ for SandboxDependsFunction so we can reference it as a dictionary key in py3 r=#build → Bug 1547730 - Define __hash__ for SandboxDependsFunction and CombinedDependsFunction so we can reference it as a dictionary key in py3 r=#build
Pushed by Callek@gmail.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/b4e024474194
Add six to path when running configure. r=#build
https://hg.mozilla.org/integration/mozilla-inbound/rev/afa5801534e4
Do dict.iteritems() via six to support py3. r=#build
https://hg.mozilla.org/integration/mozilla-inbound/rev/81bc9014907e
Do six.itervalues instead of dict.itervalues r=#build
https://hg.mozilla.org/integration/mozilla-inbound/rev/042cdcc4e103
use six.iterkeys() r=#build
https://hg.mozilla.org/integration/mozilla-inbound/rev/c846bf431b5c
Allow testing of mozpack on py3 r=#build
https://hg.mozilla.org/integration/mozilla-inbound/rev/1b766ee6bf23
Switch to six.string_types instead of types.StringTypes for py3 r=#build
https://hg.mozilla.org/integration/mozilla-inbound/rev/35e2dc6ad347
Do s/StandardError/Exception/ to support py3 r=#build
https://hg.mozilla.org/integration/mozilla-inbound/rev/fa39fb43fd12
do py3 for cStringIO r=#build
https://hg.mozilla.org/integration/mozilla-inbound/rev/aa2acfd554fc
do s/StringIO.StringIO/six.StringIO/ to support py3 r=#build
https://hg.mozilla.org/integration/mozilla-inbound/rev/b7b1062d0aad
don't use py2 builtin 'unicode' in mozpack and deps r=#build
https://hg.mozilla.org/integration/mozilla-inbound/rev/16b1c53aba9d
Don't try to py3-ize indented_repr yet. r=#build
https://hg.mozilla.org/integration/mozilla-inbound/rev/1891cfbb55d1
Compensate for __builtin__ use in files accessed by py3 r=#build
https://hg.mozilla.org/integration/mozilla-inbound/rev/3df74540bde3
Use six.moves.xrange() instead of xrange() r=#build
https://hg.mozilla.org/integration/mozilla-inbound/rev/3037f5bf1019
Fix b''.join(rand.choice(bytestring)..) to use bytes() constructor instead r=#build
https://hg.mozilla.org/integration/mozilla-inbound/rev/6c9f98dacbde
Use six.moves for urllib.urlparse r=#build
https://hg.mozilla.org/integration/mozilla-inbound/rev/03bdcaab1623
Use six.moves.reduce for reduce() r=#build
https://hg.mozilla.org/integration/mozilla-inbound/rev/75879e53727c
Recognize StreamWriter exceptions for fileno() in py3 r=#build
https://hg.mozilla.org/integration/mozilla-inbound/rev/2d6ba80390c8
Stop using some cases of types.NoneType. r=#build
https://hg.mozilla.org/integration/mozilla-inbound/rev/61b8fcc639e0
Define __hash__ for SandboxDependsFunction and CombinedDependsFunction so we can reference it as a dictionary key in py3 r=#build
https://hg.mozilla.org/integration/mozilla-inbound/rev/59a1393697e0
Use new inspection methods introduced in py3 but work in py2.7 for functions r=#build
https://hg.mozilla.org/integration/mozilla-inbound/rev/cef492de6a7f
Use six.moves.builtins in the sandbox for configure r=#build
https://hg.mozilla.org/integration/mozilla-inbound/rev/48f2c4980ad0
Make more of configure/__init__ work in py3  r=#build
https://hg.mozilla.org/integration/mozilla-inbound/rev/27ff9602d80a
Update a build moz.configure function, to support py3 r=#build
https://hg.mozilla.org/integration/mozilla-inbound/rev/3df19a96f8ac
Finish up some final py3-izing in getting mozjar.py to work in py3 r=#build
https://hg.mozilla.org/integration/mozilla-inbound/rev/d282ddabcb3d
Set python test annotations to correspond to currently-passing py3 tests r=#build
https://hg.mozilla.org/integration/mozilla-inbound/rev/9e9c2e06d0fb
don't fail on an unsupported .seek() needed to support adding from compressed `zipfile.ZipFile` entries. r=#build

This is due to a conflicting name in sys path when called from mozharness,
Which we fix by moving the codecoverage.py file in mozharness to codecov.py
avoiding pythons confusion.

Depends on D32407

Pushed by jwood@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/198001a673f6
Add six to path when running configure. r=glandium
https://hg.mozilla.org/integration/autoland/rev/abbfe68afc04
Do dict.iteritems() via six to support py3. r=glandium
https://hg.mozilla.org/integration/autoland/rev/66933913a2df
Do six.itervalues instead of dict.itervalues r=glandium
https://hg.mozilla.org/integration/autoland/rev/1e4407c3c9e3
use six.iterkeys() r=glandium
https://hg.mozilla.org/integration/autoland/rev/5588a24fd92c
Allow testing of mozpack on py3 r=glandium
https://hg.mozilla.org/integration/autoland/rev/01397bf93fc4
Switch to six.string_types instead of types.StringTypes for py3 r=glandium
https://hg.mozilla.org/integration/autoland/rev/6c6c8728e836
Do s/StandardError/Exception/ to support py3 r=glandium
https://hg.mozilla.org/integration/autoland/rev/228da077174e
do py3 for cStringIO r=glandium
https://hg.mozilla.org/integration/autoland/rev/af2c8864114c
do s/StringIO.StringIO/six.StringIO/ to support py3 r=glandium
https://hg.mozilla.org/integration/autoland/rev/f6509e01bf77
don't use py2 builtin 'unicode' in mozpack and deps r=glandium
https://hg.mozilla.org/integration/autoland/rev/47e737a70bea
Don't try to py3-ize indented_repr yet. r=glandium
https://hg.mozilla.org/integration/autoland/rev/d74097a6c6c5
Compensate for __builtin__ use in files accessed by py3 r=glandium
https://hg.mozilla.org/integration/autoland/rev/2dafc77a00b2
Use six.moves.xrange() instead of xrange() r=glandium
https://hg.mozilla.org/integration/autoland/rev/5839ef6f14c5
Fix b''.join(rand.choice(bytestring)..) to use bytes() constructor instead r=glandium
https://hg.mozilla.org/integration/autoland/rev/f234dadb03c2
Use six.moves for urllib.urlparse r=glandium
https://hg.mozilla.org/integration/autoland/rev/5eefa13f42a6
Use six.moves.reduce for reduce() r=glandium
https://hg.mozilla.org/integration/autoland/rev/7d416b0e2d4e
Recognize StreamWriter exceptions for fileno() in py3 r=glandium
https://hg.mozilla.org/integration/autoland/rev/577509d87235
Stop using some cases of types.NoneType. r=glandium
https://hg.mozilla.org/integration/autoland/rev/bd44eb6beabb
Define __hash__ for SandboxDependsFunction and CombinedDependsFunction so we can reference it as a dictionary key in py3 r=glandium
https://hg.mozilla.org/integration/autoland/rev/d89d2754488e
Use new inspection methods introduced in py3 but work in py2.7 for functions r=glandium
https://hg.mozilla.org/integration/autoland/rev/e7cfc61f2bc3
Use six.moves.builtins in the sandbox for configure r=glandium
https://hg.mozilla.org/integration/autoland/rev/690a6fec7137
Make more of configure/__init__ work in py3  r=glandium
https://hg.mozilla.org/integration/autoland/rev/29daeec3a135
Update a build moz.configure function, to support py3 r=glandium
https://hg.mozilla.org/integration/autoland/rev/a3d287588b12
Finish up some final py3-izing in getting mozjar.py to work in py3 r=glandium
https://hg.mozilla.org/integration/autoland/rev/ef2589543f46
Set python test annotations to correspond to currently-passing py3 tests r=glandium
https://hg.mozilla.org/integration/autoland/rev/7d0b13ac76f2
don't fail on an unsupported .seek() needed to support adding from compressed `zipfile.ZipFile` entries. r=glandium
https://hg.mozilla.org/integration/autoland/rev/12061a0decdb
Fix precomplete file generation when passing in unicode strings. r=glandium
https://hg.mozilla.org/integration/autoland/rev/dbadda4e3c1c
Fix use of absolute imports in codecoverage/*.py r=tomprince
Flags: needinfo?(bugspam.Callek)

https://hg.mozilla.org/mozilla-central/rev/198001a673f6
https://hg.mozilla.org/mozilla-central/rev/abbfe68afc04
https://hg.mozilla.org/mozilla-central/rev/66933913a2df
https://hg.mozilla.org/mozilla-central/rev/1e4407c3c9e3
https://hg.mozilla.org/mozilla-central/rev/5588a24fd92c
https://hg.mozilla.org/mozilla-central/rev/01397bf93fc4
https://hg.mozilla.org/mozilla-central/rev/6c6c8728e836
https://hg.mozilla.org/mozilla-central/rev/228da077174e
https://hg.mozilla.org/mozilla-central/rev/af2c8864114c
https://hg.mozilla.org/mozilla-central/rev/f6509e01bf77
https://hg.mozilla.org/mozilla-central/rev/47e737a70bea
https://hg.mozilla.org/mozilla-central/rev/d74097a6c6c5
https://hg.mozilla.org/mozilla-central/rev/2dafc77a00b2
https://hg.mozilla.org/mozilla-central/rev/5839ef6f14c5
https://hg.mozilla.org/mozilla-central/rev/f234dadb03c2
https://hg.mozilla.org/mozilla-central/rev/5eefa13f42a6
https://hg.mozilla.org/mozilla-central/rev/7d416b0e2d4e
https://hg.mozilla.org/mozilla-central/rev/577509d87235
https://hg.mozilla.org/mozilla-central/rev/bd44eb6beabb
https://hg.mozilla.org/mozilla-central/rev/d89d2754488e
https://hg.mozilla.org/mozilla-central/rev/e7cfc61f2bc3
https://hg.mozilla.org/mozilla-central/rev/690a6fec7137
https://hg.mozilla.org/mozilla-central/rev/29daeec3a135
https://hg.mozilla.org/mozilla-central/rev/a3d287588b12
https://hg.mozilla.org/mozilla-central/rev/ef2589543f46
https://hg.mozilla.org/mozilla-central/rev/7d0b13ac76f2
https://hg.mozilla.org/mozilla-central/rev/12061a0decdb
https://hg.mozilla.org/mozilla-central/rev/dbadda4e3c1c

Status: NEW → RESOLVED
Closed: 3 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla69
No longer regressions: 1555894
You need to log in before you can comment on or make changes to this bug.