Closed Bug 1210157 Opened 4 years ago Closed 5 months ago

bootstrap.py raises syntax error on Python 3

Categories

(Firefox Build System :: Bootstrap Configuration, defect)

defect
Not set

Tracking

(firefox70 fixed)

RESOLVED FIXED
mozilla70
Tracking Status
firefox70 --- fixed

People

(Reporter: altblitz, Assigned: mars)

References

(Blocks 2 open bugs)

Details

Attachments

(8 files, 1 obsolete file)

47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:44.0) Gecko/20100101 Firefox/44.0
Build ID: 20150921073455

Steps to reproduce:

http://areweeveryoneyet.org/onramp/desktop-linux.html

'If you don’t have one yet, create a "src" directory for yourself under your home directory: cd ~ && mkdir src' - done.
'Next download the bootstrap.py script and save it in your src/ directory.' - done
'And finally, in your terminal: cd ~/src and start the bootstrapper like this:
python bootstrap.py' - failed

[foo@ba ~/src ] uname  -a
Linux inode 4.2.1-1-ARCH #1 SMP PREEMPT Tue Sep 22 06:57:07 CEST 2015 x86_64 GNU/Linux
[foo@ba ~/src ] python --version
Python 3.4.3




Actual results:

[foo@bar ~/src ] python bootstrap.py  
Could not load the bootstrap Python environment.
This should never happen. Consider filing a bug.
Missing parentheses in call to 'print' (archlinux.py, line 67)



Expected results:

'... follow the prompts.'
In Arch python bootstrap.py should be python2 bootstrap.py

Your version of Mercurial (3.5.1) is sufficiently modern.
Your version of Python (2.7.10) is new enough.

Your system should be ready to build Firefox for Desktop! If you have not already,
obtain a copy of the source code by running:
    hg clone https://hg.mozilla.org/mozilla-central
Or, if you prefer Git:
    git clone https://git.mozilla.org/integration/gecko-dev.git

python2 bootstrap.py  6.71s user 1.21s system 12% cpu 1:03.48 total
Yeah, this is a Python 3 compatibility problem. We need to make the bootstrapper Python 3 compatible as a lot of distros are shipping Python 3 by default now.

We should at least detect Python 3 and raise an actionable error until we have full Python 3 support.
Component: Mercurial: hg.mozilla.org → Build Config
Product: Developer Services → Core
QA Contact: hwine
Summary: Firefox Onramp: Could not load the bootstrap Python environment → bootstrap.py raises syntax error on Python 3
Version: Development/Staging → unspecified
Product: Core → Firefox Build System
Component: General → Bootstrap Configuration
(In reply to Gregory Szorc [:gps] from comment #2)
> Yeah, this is a Python 3 compatibility problem. We need to make the
> bootstrapper Python 3 compatible as a lot of distros are shipping Python 3
> by default now.
> 
> We should at least detect Python 3 and raise an actionable error until we
> have full Python 3 support.

This was done in bug 1257095.
Assignee: nobody → mars
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true

mozboot calls out to 'mach artifact' to install dependencies on the host. Make
mozboot use the default interpreter for 'mach artifact' (Python 2 for now)
instead of using the interpreter that mozboot was called with.

This allows us to port the mozboot code to Python 3 without having to port
everything 'mach artifact' pulls in as well (which is a lot).

Make the mozboot base modules and Debian dev environment
bootstrap routines compatible with both Python 2.7 and Python 3.6+.

  • Remove Python 2.6 support from bootstrap.py and mozboot.
  • Mark minimum versions for bootstrap.py as Python 2.7 and Python 3.6+.
  • Make bootstrap.py support Python 3.
  • Make mozboot.base and mozboot.bootstrap support Python 3.
  • Make the Debian-based Linux distro bootstrap routines support
    Python 3.
  • Make "mach bootstrap" use the Python 3 interpreter.
  • Use unicode_literals in all mozboot modules.

Depends on D39080

Attachment #9080127 - Attachment description: Bug 1210157 - make mozboot call mach artifact with default python → Bug 1210157 - Make mozboot call mach artifact with default python
Attachment #9080127 - Attachment description: Bug 1210157 - Make mozboot call mach artifact with default python → Bug 1210157 - Make mozboot call mach artifact with default Python

Add unicode_literals to all mozboot module future statements to support
running the modules under Python 3. Remove comments about unicode_literals and
Python 2.6 support as Python 2.6 is no longer supported in tree.

Add support for both Python 3 and Python 2.7 to the mozboot.base and
mozboot.bootstrap modules. Remove legacy Python 2.6 code or mark it for later
removal.

Depends on D39359

Add support for Python 3 and Python 2.7 to the Debian-based linux distro
bootstrap routines.

Depends on D39360

Support installing the android development toolchain with Python 3 as well as
Python 2.7.

Depends on D39361

Add an option to run bootstrap.py with '--debug'. This will print full
tracebacks if the bootstrap process encounters an uncaught exception. It should
be useful for developers who are working on the bootstrap code as well as users
when writing bug reports.

Depends on D39362

Make bootstrap.py compatible with both Python 3.6+ and Python 2.7. Remove
Python 2.6 support as Python 2.6 is no longer supported in the Firefox source
tree.

Depends on D39363

Fix some library calls and syntax that are required to support Python 3.

Depends on D39364

Attachment #9080128 - Attachment is obsolete: true
Blocks: bootstrap-python3
No longer blocks: buildpython3
Blocks: 1569059
Attachment #9080644 - Attachment description: Bug 1210157 - Make bootstrap.py work with Python 3.6+ and Python 2.7 → Bug 1210157 - Make bootstrap.py work with Python 3.5+ and Python 2.7
Attachment #9080644 - Attachment description: Bug 1210157 - Make bootstrap.py work with Python 3.5+ and Python 2.7 → Bug 1210157 - Make bootstrap.py work with Python 3.6+ and Python 2.7
Attachment #9080644 - Attachment description: Bug 1210157 - Make bootstrap.py work with Python 3.6+ and Python 2.7 → Bug 1210157 - Make bootstrap.py work with Python 3.5+ and Python 2.7
Pushed by mfogels@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/f044d6116869
Use unicode_literals in all mozboot modules r=firefox-build-system-reviewers,mshal
https://hg.mozilla.org/integration/autoland/rev/5d26b3d7d4a0
Add Python 3 support to mozboot.base and mozboot.bootstrap r=firefox-build-system-reviewers,mshal
https://hg.mozilla.org/integration/autoland/rev/25aef37f3556
Add Python 3 support to the mozboot Debian bootstrapper r=firefox-build-system-reviewers,mshal
https://hg.mozilla.org/integration/autoland/rev/a65032b442bc
Add Python 3 support to mozboot Android toolchain bootstrap r=nalexander
https://hg.mozilla.org/integration/autoland/rev/0f5311bbd75b
Add a --debug option to bootstrap.py r=nalexander
https://hg.mozilla.org/integration/autoland/rev/06d452ac1d16
Make bootstrap.py work with Python 3.5+ and Python 2.7 r=firefox-build-system-reviewers,mshal
https://hg.mozilla.org/integration/autoland/rev/eec9458e553d
Fix Python 3 issues in mozboot.gentoo and mozboot.archlinux r=firefox-build-system-reviewers,mshal

Backed out 8 changesets (Bug 1569046, Bug 1210157) for flake8 failure on base.py

Push with failure: https://treeherder.mozilla.org/#/jobs?repo=autoland&searchStr=linting%2Copt%2Csource-test-mozlint-py-flake8%2C%28f8%29&fromchange=a59737bc6d819c1a9da579977c1afe6145964690&tochange=3c49b0c27b1558011f0927f6169fa522b2117c04

Backout link: https://hg.mozilla.org/integration/autoland/rev/3c49b0c27b1558011f0927f6169fa522b2117c04

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=262549430&repo=autoland&lineNumber=288
[task 2019-08-20T21:03:03.028Z] building 'psutil._psutil_posix' extension
[task 2019-08-20T21:03:03.028Z] x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=543 -DPSUTIL_LINUX=1 -I/usr/include/python2.7 -c psutil/_psutil_common.c -o build/temp.linux-x86_64-2.7/psutil/_psutil_common.o
[task 2019-08-20T21:03:03.028Z] x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=543 -DPSUTIL_LINUX=1 -I/usr/include/python2.7 -c psutil/_psutil_posix.c -o build/temp.linux-x86_64-2.7/psutil/_psutil_posix.o
[task 2019-08-20T21:03:03.028Z] x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wl,-Bsymbolic-functions -Wl,-z,relro -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/psutil/_psutil_common.o build/temp.linux-x86_64-2.7/psutil/_psutil_posix.o -o build/lib.linux-x86_64-2.7/psutil/_psutil_posix.so
[task 2019-08-20T21:03:03.028Z] copying build/lib.linux-x86_64-2.7/psutil/_psutil_linux.so -> psutil
[task 2019-08-20T21:03:03.028Z] copying build/lib.linux-x86_64-2.7/psutil/_psutil_posix.so -> psutil
[task 2019-08-20T21:03:03.029Z]
[task 2019-08-20T21:03:03.029Z] Error processing command. Ignoring because optional. (optional:packages.txt:comm/build/virtualenv_packages.txt)
[task 2019-08-20T21:06:15.592Z] TEST-UNEXPECTED-ERROR | /builds/worker/checkouts/gecko/python/mozboot/mozboot/base.py:27:1 | too many blank lines (3) (E303)
[taskcluster 2019-08-20 21:06:16.128Z] === Task Finished ===
[taskcluster 2019-08-20 21:06:17.135Z] Unsuccessful task run with exit code: 1 completed in 462.916 seconds

Flags: needinfo?(mars)
Flags: needinfo?(mars)
Pushed by mfogels@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/fd1ce26da310
Use unicode_literals in all mozboot modules r=firefox-build-system-reviewers,mshal
https://hg.mozilla.org/integration/autoland/rev/41bebeeebc27
Add Python 3 support to mozboot.base and mozboot.bootstrap r=firefox-build-system-reviewers,mshal
https://hg.mozilla.org/integration/autoland/rev/6a5f5594422f
Add Python 3 support to the mozboot Debian bootstrapper r=firefox-build-system-reviewers,mshal
https://hg.mozilla.org/integration/autoland/rev/d79183b963e7
Add Python 3 support to mozboot Android toolchain bootstrap r=nalexander
https://hg.mozilla.org/integration/autoland/rev/7c2b0c902854
Add a --debug option to bootstrap.py r=nalexander
https://hg.mozilla.org/integration/autoland/rev/f0f92aecee18
Make bootstrap.py work with Python 3.5+ and Python 2.7 r=firefox-build-system-reviewers,mshal
https://hg.mozilla.org/integration/autoland/rev/d00d447a0c29
Fix Python 3 issues in mozboot.gentoo and mozboot.archlinux r=firefox-build-system-reviewers,mshal
You need to log in before you can comment on or make changes to this bug.