$MACOSX_DEPLOYMENT_TARGET mismatch during build due to build/manifestparser.py

RESOLVED FIXED in mozilla8

Status

defect
RESOLVED FIXED
8 years ago
6 years ago

People

(Reporter: glandium, Assigned: k0scist)

Tracking

unspecified
mozilla8
x86
macOS
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments)

Reporter

Description

8 years ago
When building for x86, we default to and export MACOSX_DEPLOYMENT_TARGET=10.5. When reaching rules using xpccheck, we end up with a message about a $MACOSX_DEPLOYMENT_TARGET mismatch. This is sadly due to setuptools being broken:

# MACOSX_DEPLOYMENT_TARGET=10.5 python -c "from setuptools import setup"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/setuptools/__init__.py", line 2, in <module>
    from setuptools.extension import Extension, Library
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/setuptools/extension.py", line 2, in <module>
    from dist import _get_unpatched
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/setuptools/dist.py", line 5, in <module>
    from setuptools.command.install import install
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/setuptools/command/__init__.py", line 13, in <module>
    from distutils.command.bdist import bdist
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/command/bdist.py", line 29, in <module>
    class bdist (Command):
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/command/bdist.py", line 37, in bdist
    "(default: %s)" % get_platform()),
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/util.py", line 97, in get_platform
    cfgvars = get_config_vars()
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/sysconfig.py", line 525, in get_config_vars
    func()
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/sysconfig.py", line 408, in _init_posix
    raise DistutilsPlatformError(my_msg)
distutils.errors.DistutilsPlatformError: $MACOSX_DEPLOYMENT_TARGET mismatch: now "10.5" but "10.6" during configure

Unfortunately, this is not caught by the try/except clause around the setuptools import in manifestparser.py.
Josh hit this somewhere else and filed it upstream with Macports:
http://trac.macports.org/ticket/27916

which got moved upstream to Python:
http://bugs.python.org/issue9516

Comment 2

8 years ago
I believe this has been fixed in python and MacPorts has ported it to their port of python2.6 and up. Do "sudo port upgrade outdated" to get the new version (that'll upgrade everything else you have, like mercurial, as well).
Reporter

Comment 3

8 years ago
(In reply to comment #2)
> I believe this has been fixed in python and MacPorts has ported it to their
> port of python2.6 and up. Do "sudo port upgrade outdated" to get the new
> version (that'll upgrade everything else you have, like mercurial, as well).

This is apparently not the case. For the record, I figured I hadn't actually installed python from macports, and was using the system provided python 2.6. As a consequence, I installed python2.6 and py26-setuptools from macports (I also updated all ports I had installed) ... but the same happens.
I found what should be a workaround for bz: --enable-macos-target=10.6
Just so that people are aware:

This bug (which is a Python bug) only happens if you try to build the
current tree with MACOSX_DEPLOYMENT_TARGET set to something else than
what it was set to when your copy of Python was built.
This bug has not been fixed in the Python 2.7 included with Apple's
latest Lion+XCode combo -- Lion DP4 (build 11A480b) plus XCode 4.1 DP
6.
(Following up comment #6)

Though recent comments at http://bugs.python.org/issue9516 show the
Python developers are working on it.
(Following up comment #5)

> This bug (which is a Python bug) only happens if you try to build
> the current tree with MACOSX_DEPLOYMENT_TARGET set to something else
> than what it was set to when your copy of Python was built.

It'd be more accurate to say that this bug only happens if you try to
build the current Mozilla tree with MACOSX_DEPLOYMENT_TARGET set to *a
lower value than* what it was set to when your copy of Python was
built.  Building Python with a given value of MACOSX_DEPLOYMENT_TARGET
(and using the corresponding SDK) assures you can use that build back
to that version of OS X.

To do a universal Intel build (a 32/64 bit build) we need to do a
32-bit build using the 10.5 SDK (with MACOSX_DEPLOYMENT_TARGET set to
10.5) and a 64-bit build using the 10.6 SDK (with
MACOSX_DEPLOYMENT_TARGET set to 10.6) -- which we afterwards combine
into a single package.  For Python to work properly when doing such a
build, it needs to have been built with the 10.5 SDK and with
MACOSX_DEPLOYMENT_TARGET set to 10.5.

But the Python that comes with OS X 10.6 was built with the 10.6 SDK
and with MACOSX_DEPLOYMENT_TARGET set to 10.6.  And the Python that
comes with OS X 10.7 was built with the 10.7 SDK and with
MACOSX_DEPLOYMENT_TARGET set to 10.7.  So neither one can be used to
do a universal Intel build of the Mozilla tree.

I still	haven't figured out which SDK the MacPorts python26 (for OS X
10.6) and python27 (for OS X 10.7) were built with.  But since we use
MacPorts' python to successfully do universal Intel builds on OS X
10.6.X, I assume MacPorts python26 was built with the 10.5 SDK.

In any case I find I can build Python 2.6.7 on OS X 10.6.8 with the
following configure parameters, then use it successfully to build the
current mozilla-central tree:

--enable-framework --enable-universalsdk=/Developer/SDKs/MacOSX10.5.sdk/
  --with-universal-archs=intel --enable-ipv6 MACOSX_DEPLOYMENT_TARGET=10.5
FWIW, the build slaves running 10.6 have quite an old copy of python and macports and presumably avoid the problem because of that.
It looks like we don't use setuptools in a regular build, so we can just avoid the import.

This is a hack, but at least it gets the common case working while the bug is fixed for real in python.
Assignee: nobody → respindola
Status: NEW → ASSIGNED
Attachment #547889 - Flags: review?(ted.mielczarek)
Attachment #547889 - Flags: review?(ted.mielczarek) → review?(jhammel)
Assignee

Comment 11

8 years ago
Comment on attachment 547889 [details] [diff] [review]
patch for avoiding the problem

This makes setup uncallable from the command line.  I'll upload a recommendation shortly.
Attachment #547889 - Flags: review?(jhammel) → review-
Assignee

Comment 12

8 years ago
If the import of setuptools fails for any reason, not just an importerror, ignore and move on.
Attachment #548201 - Flags: review?(ted.mielczarek)
Assignee

Comment 13

8 years ago
Note that this is a patch vs the canonical source, http://hg.mozilla.org/automation/ManifestDestiny, not mozilla-central
Comment on attachment 548201 [details] [diff] [review]
if import messes up, just don't use it

Review of attachment 548201 [details] [diff] [review]:
-----------------------------------------------------------------
Attachment #548201 - Flags: review?(ted.mielczarek) → review+
Assignee: respindola → jhammel
Assignee

Comment 15

8 years ago
pushed: http://hg.mozilla.org/automation/ManifestDestiny/rev/76eae0af3786

m-c, etc still needs to be updated
Assignee: jhammel → nobody
Assignee

Comment 17

8 years ago
(In reply to comment #15)
> pushed: http://hg.mozilla.org/automation/ManifestDestiny/rev/76eae0af3786
> 
> m-c, etc still needs to be updated

Sorry, linked to the wrong changeset: http://hg.mozilla.org/automation/ManifestDestiny/rev/4175f310b7d4
http://hg.mozilla.org/mozilla-central/rev/6953340878aa
Status: ASSIGNED → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla8
Blocks: 783727
This is still happening using Mac OS 10.7.5 and trying to build different changesets: 

 5:28.01   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/sysconfig.py", line 407, in _init_posix
 5:28.01     raise DistutilsPlatformError(my_msg)
 5:28.01 distutils.errors.DistutilsPlatformError: $MACOSX_DEPLOYMENT_TARGET mismatch: now "10.6" but "10.7" during configure
 5:28.02 make[7]: *** [libs-xpcshell-tests] Error 1
 5:28.02 make[6]: *** [libs] Error 2
 5:28.02 make[5]: *** [libs] Error 2
 5:28.02 make[4]: *** [libs_tier_platform] Error 2
 5:28.02 make[3]: *** [tier_platform] Error 2
 5:28.02 make[2]: *** [default] Error 2
 5:28.04 make[1]: *** [realbuild] Error 2
 5:28.04 make: *** [build] Error 2
 5:28.08 34 compiler warnings present.
Any idea why this might be happening, Jeff?
Flags: needinfo?(jhammel)
Note that this is still (as far as I know) ultimately a Python bug, and can be worked around by rebuilding/reinstalling Python as per comment #8.
(In reply to Steven Michaud from comment #21)
> Note that this is still (as far as I know) ultimately a Python bug, and can
> be worked around by rebuilding/reinstalling Python as per comment #8.

Not working if I rebuild/reinstall Python. I have reinstall the entire OS and still get this issue.

This is my .mozconfig file:

ac_add_options --with-ccache=/usr/local/bin/ccache
mk_add_options MOZ_MAKE_FLAGS="-j4"
> I have reinstall the entire OS and still get this issue.

Reinstalling the OS doesn't do what I described in comment #8.  The Python bug is triggered by how Apple built Python in the first place.
Assignee

Comment 24

6 years ago
I don't really know about the Mac python version mismatch bug.  AFAIK it is a vendor bug, but I know little about Mac OS :(
Flags: needinfo?(jhammel)
Don't use the Python provided by Apple/OS X. Instead, install Python from Homebrew or MacPorts (or install yourself if you are feeling adventurous). Use |mach bootstrap| to guide you through the system bootstrap process.

Perhaps we should ban Apple's system Python in the build system...
Whiteboard: fixed-in-bs
Assignee

Comment 26

6 years ago
> Perhaps we should ban Apple's system Python in the build system...

Or at least warn...maybe point to a URL resource for this error if/when it occurs
(In reply to Steven Michaud from comment #23)
> > I have reinstall the entire OS and still get this issue.
> 
> Reinstalling the OS doesn't do what I described in comment #8.  The Python
> bug is triggered by how Apple built Python in the first place.

I am using Homebrew for all program's instalation.
> I am using Homebrew for all program's instalation.

What version of Python do you have installed from Homebrew?

If (as I assume) it's 2.7.2, try installing Python 2.7.3 (or 2.7.4 or 2.7.5) from Homebrew, and see if this helps.

(I suspect Homebrew makes the same mistake with Python 2.7.2 as Apple does, and builds it MACOSX_DEPLOYMENT_TARGET set to 10.7, instead of 10.6.)
You need to log in before you can comment on or make changes to this bug.