659881 - $MACOSX_DEPLOYMENT_TARGET mismatch during build due to build/manifestparser.py

Status: RESOLVED FIXED

(Testing :: XPCShell Harness, defect)

Description

[Mike Hommey [:glandium]]

When building for x86, we default to and export MACOSX_DEPLOYMENT_TARGET=10.5. When reaching rules using xpccheck, we end up with a message about a $MACOSX_DEPLOYMENT_TARGET mismatch. This is sadly due to setuptools being broken:

# MACOSX_DEPLOYMENT_TARGET=10.5 python -c "from setuptools import setup"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/setuptools/__init__.py", line 2, in <module>
    from setuptools.extension import Extension, Library
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/setuptools/extension.py", line 2, in <module>
    from dist import _get_unpatched
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/setuptools/dist.py", line 5, in <module>
    from setuptools.command.install import install
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/setuptools/command/__init__.py", line 13, in <module>
    from distutils.command.bdist import bdist
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/command/bdist.py", line 29, in <module>
    class bdist (Command):
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/command/bdist.py", line 37, in bdist
    "(default: %s)" % get_platform()),
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/util.py", line 97, in get_platform
    cfgvars = get_config_vars()
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/sysconfig.py", line 525, in get_config_vars
    func()
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/sysconfig.py", line 408, in _init_posix
    raise DistutilsPlatformError(my_msg)
distutils.errors.DistutilsPlatformError: $MACOSX_DEPLOYMENT_TARGET mismatch: now "10.5" but "10.6" during configure

Unfortunately, this is not caught by the try/except clause around the setuptools import in manifestparser.py.

Comment 1

[(not currently active) Ted Mielczarek]

Josh hit this somewhere else and filed it upstream with Macports:
http://trac.macports.org/ticket/27916

which got moved upstream to Python:
http://bugs.python.org/issue9516

Comment 2

[Josh Aas]

I believe this has been fixed in python and MacPorts has ported it to their port of python2.6 and up. Do "sudo port upgrade outdated" to get the new version (that'll upgrade everything else you have, like mercurial, as well).

Comment 3

[Mike Hommey [:glandium]]

(In reply to comment #2)
> I believe this has been fixed in python and MacPorts has ported it to their
> port of python2.6 and up. Do "sudo port upgrade outdated" to get the new
> version (that'll upgrade everything else you have, like mercurial, as well).

This is apparently not the case. For the record, I figured I hadn't actually installed python from macports, and was using the system provided python 2.6. As a consequence, I installed python2.6 and py26-setuptools from macports (I also updated all ports I had installed) ... but the same happens.

Comment 4

[(not currently active) Ted Mielczarek]

I found what should be a workaround for bz: --enable-macos-target=10.6

Comment 5

[Steven Michaud [:smichaud] (Retired)]

Just so that people are aware:

This bug (which is a Python bug) only happens if you try to build the
current tree with MACOSX_DEPLOYMENT_TARGET set to something else than
what it was set to when your copy of Python was built.

Comment 6

[Steven Michaud [:smichaud] (Retired)]

This bug has not been fixed in the Python 2.7 included with Apple's
latest Lion+XCode combo -- Lion DP4 (build 11A480b) plus XCode 4.1 DP
6.

Comment 7

[Steven Michaud [:smichaud] (Retired)]

(Following up comment #6)

Though recent comments at http://bugs.python.org/issue9516 show the
Python developers are working on it.

Comment 8

[Steven Michaud [:smichaud] (Retired)]

(Following up comment #5)

> This bug (which is a Python bug) only happens if you try to build
> the current tree with MACOSX_DEPLOYMENT_TARGET set to something else
> than what it was set to when your copy of Python was built.

It'd be more accurate to say that this bug only happens if you try to
build the current Mozilla tree with MACOSX_DEPLOYMENT_TARGET set to *a
lower value than* what it was set to when your copy of Python was
built.  Building Python with a given value of MACOSX_DEPLOYMENT_TARGET
(and using the corresponding SDK) assures you can use that build back
to that version of OS X.

To do a universal Intel build (a 32/64 bit build) we need to do a
32-bit build using the 10.5 SDK (with MACOSX_DEPLOYMENT_TARGET set to
10.5) and a 64-bit build using the 10.6 SDK (with
MACOSX_DEPLOYMENT_TARGET set to 10.6) -- which we afterwards combine
into a single package.  For Python to work properly when doing such a
build, it needs to have been built with the 10.5 SDK and with
MACOSX_DEPLOYMENT_TARGET set to 10.5.

But the Python that comes with OS X 10.6 was built with the 10.6 SDK
and with MACOSX_DEPLOYMENT_TARGET set to 10.6.  And the Python that
comes with OS X 10.7 was built with the 10.7 SDK and with
MACOSX_DEPLOYMENT_TARGET set to 10.7.  So neither one can be used to
do a universal Intel build of the Mozilla tree.

I still	haven't figured out which SDK the MacPorts python26 (for OS X
10.6) and python27 (for OS X 10.7) were built with.  But since we use
MacPorts' python to successfully do universal Intel builds on OS X
10.6.X, I assume MacPorts python26 was built with the 10.5 SDK.

In any case I find I can build Python 2.6.7 on OS X 10.6.8 with the
following configure parameters, then use it successfully to build the
current mozilla-central tree:

--enable-framework --enable-universalsdk=/Developer/SDKs/MacOSX10.5.sdk/
  --with-universal-archs=intel --enable-ipv6 MACOSX_DEPLOYMENT_TARGET=10.5

Comment 9

[Nick Thomas [:nthomas] (UTC+12)]

FWIW, the build slaves running 10.6 have quite an old copy of python and macports and presumably avoid the problem because of that.

Comment 10

[Rafael Ávila de Espíndola (:espindola) (not reading bugmail)]

Attachment: patch for avoiding the problem

It looks like we don't use setuptools in a regular build, so we can just avoid the import.

This is a hack, but at least it gets the common case working while the bug is fixed for real in python.

Comment 11

[Jeff Hammel]

Comment on attachment 547889 [details] [diff] [review]
patch for avoiding the problem

This makes setup uncallable from the command line.  I'll upload a recommendation shortly.

Comment 12

[Jeff Hammel]

Attachment: if import messes up, just don't use it

If the import of setuptools fails for any reason, not just an importerror, ignore and move on.

Comment 13

[Jeff Hammel]

Note that this is a patch vs the canonical source, http://hg.mozilla.org/automation/ManifestDestiny, not mozilla-central

Comment 14

[(not currently active) Ted Mielczarek]

Comment on attachment 548201 [details] [diff] [review]
if import messes up, just don't use it

Review of attachment 548201 [details] [diff] [review]:
-----------------------------------------------------------------

Comment 15

[Jeff Hammel]

pushed: http://hg.mozilla.org/automation/ManifestDestiny/rev/76eae0af3786

m-c, etc still needs to be updated

Comment 16

[(not currently active) Ted Mielczarek]

http://hg.mozilla.org/projects/build-system/rev/6953340878aa

Comment 17

[Jeff Hammel]

(In reply to comment #15)
> pushed: http://hg.mozilla.org/automation/ManifestDestiny/rev/76eae0af3786
> 
> m-c, etc still needs to be updated

Sorry, linked to the wrong changeset: http://hg.mozilla.org/automation/ManifestDestiny/rev/4175f310b7d4

Comment 18

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

http://hg.mozilla.org/mozilla-central/rev/6953340878aa

Comment 19

[Mihai Morar, (:MihaiMorar)]

This is still happening using Mac OS 10.7.5 and trying to build different changesets: 

 5:28.01   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/sysconfig.py", line 407, in _init_posix
 5:28.01     raise DistutilsPlatformError(my_msg)
 5:28.01 distutils.errors.DistutilsPlatformError: $MACOSX_DEPLOYMENT_TARGET mismatch: now "10.6" but "10.7" during configure
 5:28.02 make[7]: *** [libs-xpcshell-tests] Error 1
 5:28.02 make[6]: *** [libs] Error 2
 5:28.02 make[5]: *** [libs] Error 2
 5:28.02 make[4]: *** [libs_tier_platform] Error 2
 5:28.02 make[3]: *** [tier_platform] Error 2
 5:28.02 make[2]: *** [default] Error 2
 5:28.04 make[1]: *** [realbuild] Error 2
 5:28.04 make: *** [build] Error 2
 5:28.08 34 compiler warnings present.

Comment 20

[Steven Michaud [:smichaud] (Retired)]

Any idea why this might be happening, Jeff?

Comment 21

[Steven Michaud [:smichaud] (Retired)]

Note that this is still (as far as I know) ultimately a Python bug, and can be worked around by rebuilding/reinstalling Python as per comment #8.

Comment 22

[Mihai Morar, (:MihaiMorar)]

(In reply to Steven Michaud from comment #21)
> Note that this is still (as far as I know) ultimately a Python bug, and can
> be worked around by rebuilding/reinstalling Python as per comment #8.

Not working if I rebuild/reinstall Python. I have reinstall the entire OS and still get this issue.

This is my .mozconfig file:

ac_add_options --with-ccache=/usr/local/bin/ccache
mk_add_options MOZ_MAKE_FLAGS="-j4"

Comment 23

[Steven Michaud [:smichaud] (Retired)]

> I have reinstall the entire OS and still get this issue.

Reinstalling the OS doesn't do what I described in comment #8.  The Python bug is triggered by how Apple built Python in the first place.

Comment 24

[Jeff Hammel]

I don't really know about the Mac python version mismatch bug.  AFAIK it is a vendor bug, but I know little about Mac OS :(

Comment 25

[Gregory Szorc [:gps]]

Don't use the Python provided by Apple/OS X. Instead, install Python from Homebrew or MacPorts (or install yourself if you are feeling adventurous). Use |mach bootstrap| to guide you through the system bootstrap process.

Perhaps we should ban Apple's system Python in the build system...

Comment 26

[Jeff Hammel]

> Perhaps we should ban Apple's system Python in the build system...

Or at least warn...maybe point to a URL resource for this error if/when it occurs

Comment 27

[Gregory Szorc [:gps]]

Filed bug 913117.

Comment 28

[Mihai Morar, (:MihaiMorar)]

(In reply to Steven Michaud from comment #23)
> > I have reinstall the entire OS and still get this issue.
> 
> Reinstalling the OS doesn't do what I described in comment #8.  The Python
> bug is triggered by how Apple built Python in the first place.

I am using Homebrew for all program's instalation.

Comment 29

[Steven Michaud [:smichaud] (Retired)]

> I am using Homebrew for all program's instalation.

What version of Python do you have installed from Homebrew?

If (as I assume) it's 2.7.2, try installing Python 2.7.3 (or 2.7.4 or 2.7.5) from Homebrew, and see if this helps.

(I suspect Homebrew makes the same mistake with Python 2.7.2 as Apple does, and builds it MACOSX_DEPLOYMENT_TARGET set to 10.7, instead of 10.6.)