Closed Bug 1081313 Opened 11 years ago Closed 11 years ago

Build Python27, Mercurial, and wget for Yosemite

Categories

(Infrastructure & Operations :: RelOps: Puppet, task)

Product:
Infrastructure & Operations
Component:
RelOps: Puppet
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: dustin, Assigned: dustin)

References

Details

Attachments

(1 file)

bug1081313.patch
10.18 KB, patch
sbruno
: review+
dustin
: checked-in+
Details | Diff | Splinter Review
This seems to have been built per-release in the past, so we should continue to do so.
Summary: Build Python27 for Yosemite → Build Python27 and Mercurial for Yosemite
I've copied these packages from 10.9 to 10.10 for now, which seems to work
Summary: Build Python27 and Mercurial for Yosemite → Build Python27, Mercurial, and wget for Yosemite
Did we really built those packages on our own? Why was that necessary? Have we done any changes to the default configuration, or even patches for special needs on our side? In case of our QA nodes we always used the default packages without any problems.
Python and MySQL are the same versions as on other systems -- and much newer than those available with the OS. They're in /tools. The version of wget shipped with OS X was also too old for reasons I don't recall.
So why not using the general DMG folder for applications which are compatible across all versions of OS X, and only using subfolders for specific os versions, where we need special releases of those tools? That would give us way lesser work to do in the future for new OS X releases.
I don't recall *why* these were built per-version, but it's likely we had a good reason -- maybe Python links to different frameworks on different versions?
We never experienced those things on our systems, and we install the same version of Python and Mercurial. So maybe checking back history in the puppet repo to find this out? ;)
I might not be understanding what you're asking. What are the "default packages"?
Ups, sorry. Should have been better phrases as "default distributed packages", means packages e.g for python which are downloadable from python.org. It's not about the default python package in OS X.
Ah, gotcha. We want Python to be in the same location across operating systems, so DMGs from python.org wouldn't do the trick. I don't know of an upstream wget DMG, but if such a thing existed it would probably work.
I managed to install XCode-6.1 seed 2 command line tools, but the result is full of fail: lots of /Users/cltbld/build/Python-2.7.3/Modules/_ctypes/libffi_osx/ffi.c:107:1: warning: unused function 'struct_on_stack' [-Wunused-function] struct_on_stack( ^ 1 warning generated. ld: warning: could not create compact unwind for _ffi_call_unix64: does not use RBP or RSP based frame Undefined symbols for architecture x86_64: ... ld: symbol(s) not found for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) followed by Python build finished, but the necessary bits to build these modules were not found: _bsddb dl gdbm imageop linuxaudiodev ossaudiodev readline spwd sunaudiodev To find the necessary bits, look in setup.py in detect_modules() for the module's name. Failed to build these modules: _AE _AH _App _bisect _CarbonEvt _CF _CG _Cm _codecs_cn _codecs_hk _codecs_iso2022 _codecs_jp _codecs_kr _codecs_tw _collections _csv _Ctl _ctypes _ctypes_test _curses _curses_panel _Dlg _Drag _elementtree _Evt _File _Fm _Folder _functools _hashlib _heapq _Help _hotshot _IBCarbon _Icn _io _json _Launch _List _locale _lsprof _Menu _Mlte _multibytecodec _multiprocessing _OSA _Qd _Qdoffs _Qt _random _Res _scproxy _Scrap _Snd _socket _sqlite3 _ssl _struct _TE _testcapi _tkinter _Win array audioop autoGIL binascii bsddb185 bz2 cmath ColorPicker cPickle crypt cStringIO datetime dbm fcntl future_builtins gestalt grp icglue itertools MacOS math mmap Nav nis operator OSATerminology parser pyexpat resource select strop syslog termios time unicodedata zlib which is basically every extension module.
Same thing with or without --without-system-ffi. http://bugs.python.org/issue22093 suggests that the warning itself is harmless. As for the "Undefined symbols for architecture x86_64" .. it turns out that's due to --enable-shared. So, I set up a conditional for OS X 10.10 to remove --without-system-ffi and --enable-shared. The result seems to work.
wget doesn't make it easy, either: ./texi2pod.pl ./wget.texi wget.pod Option VERSION not defined /usr/bin/pod2man --center="GNU Wget" --release="GNU Wget 1.12" wget.pod > wget.1 wget.pod around line 1972: Expected text after =item, not a number wget.pod around line 1977: Expected text after =item, not a number wget.pod around line 1983: Expected text after =item, not a number wget.pod around line 1988: Expected text after =item, not a number wget.pod around line 1993: Expected text after =item, not a number wget.pod around line 1998: Expected text after =item, not a number wget.pod around line 2003: Expected text after =item, not a number wget.pod around line 2008: Expected text after =item, not a number POD document had syntax errors at /usr/bin/pod2man5.18 line 72. make[1]: *** [wget.1] Error 255 make: *** [all-recursive] Error 1
This works perfectly fine on a Lion builder! I think pod2man is broken on Yosemite or XCode-6.1-seed-2: [cltbld@t-yosemite-r5-0001.test.releng.scl3.mozilla.com ~]$ pod2man --help The contents of this script should normally never run! The perl wrapper should pick the correct script in /usr/bin by appending the appropriate version. You can try appending the appropriate perl version number. See perlmacosx.pod for more information about multiple version support in Mac OS X.
Attachment #8505000 - Flags: review?(hskupin)
Comment on attachment 8505000 [details] [diff] [review] bug1081313.patch Review of attachment 8505000 [details] [diff] [review]: ----------------------------------------------------------------- Sorry Dustin but I don't think that I'm the right person to get this reviewed. I have no real build experience on OS X.
Attachment #8505000 - Flags: review?(hskupin)
Attachment #8505000 - Flags: review?(sbruno)
Comment on attachment 8505000 [details] [diff] [review] bug1081313.patch Review of attachment 8505000 [details] [diff] [review]: ----------------------------------------------------------------- It looks ok to me. This review was probably more useful to me than to you, since it reminds me at least two things related to Bug 961279: 1. when running bsdtar -x on the rpm I created for mercurial, it fails - so maybe there's something wrong with the package! I'd better check. 2. When I used pkgbuild, I omitted the --install-location option. Not sure this affects how the package is installed, but - again - better to check!
Attachment #8505000 - Flags: review?(sbruno) → review+
So, thanks for asking me to review it :-)
Sure, thanks for the review. I *think* --install-location defaults to /usr, which might have your packages installing to /usr/usr or /usr/tools. Note that the bsdtar -x is on the *srpm*, too.
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: