882869 - Build OS X packages per version

Status: RESOLVED FIXED

(Infrastructure & Operations :: RelOps: Puppet, task)

Description

[Dustin J. Mitchell [:dustin] (he/him)]

This means:
 - adjusting the packages::pkgdmg define to add a version prefix to each package URL
 - rebuilding all of the packages for lion, that weren't already rebuilt as part of bug 760093, and putting them in /data/repos/DMGs/10.7/
 - moving the mountainlion packages to the /data/repos/DMGs/10.8/
 - updating docs

Comment 1

[Amy Rich [:arr] [:arich]]

Should we build on Snow Leopard and only recompile on newer versions if we find incompatibilities?  I would suspect that stuff will generally be compatible on newer OSes.

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

I think it's easier to eliminate the possibility of ambiguity, rather than hope any particular compatibility patterns hold.

Comment 3

[Dustin J. Mitchell [:dustin] (he/him)]

OK, by my reckoning, the situation is

packages::                  10.7        10.8
----------                  ----        -----
autoconf                    OK          NOT USED
ccache                      OLD         NOT USED
java                        NOT USED    OK
mozilla::git                NEEDED      NOT USED
mozilla::py27_mercurial     NEEDED      OK
mozilla::python27           NEEDED      OK
nrpe                        OK          NOT USED
p7zip                       OLD         NOT USED
screenresolution            1.5         1.6
upx                         OK          NOT USED
wget                        NEEDED      OK
xcode                       OK          NOT USED
xcode_cmdline_tools         NOT USED    OK
yasm                        OLD         NOT USED

I think packages::xcode is generic, so I'll configure it that way.  If we find
out it doesn't work on Maverick, we can move things around.

After building a NRPE with the right config arguments, I didn't adjust the
manifests to install it, which explains the NRPE problems.

So, the packages to build are
 mozilla::git
 mozilla::python27
 mozilla::py27_mercurial
 wget

the rest is just shuffling files around.

Comment 4

[Dustin J. Mitchell [:dustin] (he/him)]

Attachment: bug882869.patch

Note that this uses the new version of hg on 10.7.

Comment 5

[Justin Wood (:Callek)]

Comment on attachment 763644 [details] [diff] [review]
bug882869.patch

Review of attachment 763644 [details] [diff] [review]:
-----------------------------------------------------------------

It shocks me that xcode would _not_ be version specific... If you haven't tested that it installs fine and works on 10.8 I'd be much happier if we stuck it in the 10.7 dir and left the 10.8 dir blank, so that trying to install it on 10.8 would fail until we do test.

Comment 6

[Dustin J. Mitchell [:dustin] (he/him)]

Nothing about what I did to download or build it was specific to Lion.  I don't know if *newer* versions of xcode are independent, but that's a separate issue.

Comment 7

[Justin Wood (:Callek)]

(In reply to Dustin J. Mitchell [:dustin] from comment #6)
> Nothing about what I did to download or build it was specific to Lion.  I
> don't know if *newer* versions of xcode are independent, but that's a
> separate issue.

I know that XCode on 10.6 and XCode on 10.7 was different, and installed/acted different. The app store itself was different md5sums when downloaded from a 10.7 and a 10.6 machine, fwiw. (exampled when I was installing for SeaMonkey)...

I won't be too surprised if it did work on 10.8, but if you did not test that it installs and worked there, which your above reads "I didn't test it, but looks fine" -- please fix this.

Comment 8

[Dustin J. Mitchell [:dustin] (he/him)]

This isn't a download from the app store - this version of xcode is from before the app-store days.  The checksums from the app store are different because, I think, it tags the download with your apple ID, time of download, etc.

Comment 9

[Dustin J. Mitchell [:dustin] (he/him)]

talos-mtnlion-r5-001:~ root# gcc
i686-apple-darwin11-llvm-gcc-4.2: no input files
talos-mtnlion-r5-001:~ root# cat <<EOF >hello.c
> #include "stdio.h"
> main() { printf("Hello, world.\n"); }
> EOF
talos-mtnlion-r5-001:~ root# gcc -o hello hello.c
talos-mtnlion-r5-001:~ root# ./hello
Hello, world.
talos-mtnlion-r5-001:~ root#

Comment 10

[Dustin J. Mitchell [:dustin] (he/him)]

This failed for Lion - see bug 760093 comment 78

Comment 11

[Dustin J. Mitchell [:dustin] (he/him)]

Temporary hack, since the 10.8 build of Python works great on 10.7, but the 10.7 build fails:

recv repos/DMGs/10.7/python27-2.7.2-1-BROKEN.dmg
recv repos/DMGs/10.7/python27-2.7.2-1.dmg  -> ../10.8/python27-2.7.2-1.dmg

Comment 12

[Dustin J. Mitchell [:dustin] (he/him)]

We should probably fix this while knocking out the upgrade to 2.7.3 for bug 602908.  But after everything's on puppet-3.2.2.

Comment 13

[Dustin J. Mitchell [:dustin] (he/him)]

The bad python segfaults with
 /path/to/python -m ctypes
so no need to fire up twistd to see the fireworks.

Comment 14

[Dustin J. Mitchell [:dustin] (he/him)]

I rebuilt 2.7.3 on a spare lion builder, and it still segfaults.

Comment 15

[Dustin J. Mitchell [:dustin] (he/him)]

I'm suspecting http://bugs.python.org/issue13370

Comment 16

[Dustin J. Mitchell [:dustin] (he/him)]

But the fix to that doesn't stop the segfaulting

Comment 17

[Dustin J. Mitchell [:dustin] (he/him)]

I stand corrected - that patch, plus --without-system-ffi, fixes the error.

The mountain-lion build is failing now, though.  I'll revisit tomorrow.

Comment 18

[Dustin J. Mitchell [:dustin] (he/him)]

I get the same failure using clang.  I wonder if we need a newer version of XCode on 10.8 (I'm using 4.1).

Comment 19

[Dustin J. Mitchell [:dustin] (he/him)]

I'm going to delete the symlinks that I put in place to transition while landing attachment 763644 [details] [diff] [review]:

[[e-mail] DMGs]# ls -al | grep ^l
lrwxrwxrwx 1 puppetsync puppetsync      22 Jun 17 14:29 git-1.7.9.4-1.dmg -> 10.8/git-1.7.9.4-1.dmg
lrwxrwxrwx 1 puppetsync puppetsync      25 Jun 17 14:29 python27-2.7.2-1.dmg -> 10.8/python27-2.7.2-1.dmg
lrwxrwxrwx 1 puppetsync puppetsync      35 Jun 17 14:29 python27-mercurial-2.5.4-1.dmg -> 10.8/python27-mercurial-2.5.4-1.dmg
lrwxrwxrwx 1 puppetsync puppetsync      29 Jun 17 14:29 screenresolution-1.6.dmg -> 10.8/screenresolution-1.6.dmg
lrwxrwxrwx 1 puppetsync puppetsync      20 Jun 17 14:29 wget-1.12-1.dmg -> 10.8/wget-1.12-1.dmg

Comment 20

[Dustin J. Mitchell [:dustin] (he/him)]

Kim suspects she used 4.4 to build the original 10.8 DMGs, although she didn't build Python.

Comment 21

[Dustin J. Mitchell [:dustin] (he/him)]

Ugh, I rebuilt on 10.8 with a copy of xcode-4.6.3, and ctypes still segfaults.

Comment 22

[Dustin J. Mitchell [:dustin] (he/him)]

Attachment: bug882869-p2.patch

This upgrades the python27-dmg.sh script to build Python-2.7.3.  It doesn't yet install Python-2.7.3 -- we can do that in another bug. 

I've tested this both on Lion (with Xcode 4.1) and Mountain Lion (with Xcode-4.6.3).

Comment 23

[Justin Wood (:Callek)]

Comment on attachment 774128 [details] [diff] [review]
bug882869-p2.patch

Review of attachment 774128 [details] [diff] [review]:
-----------------------------------------------------------------

I can't easily view linked bugs for the dog build changes cited.  but at a glance it looks and sounds good

Comment 24

[Dustin J. Mitchell [:dustin] (he/him)]

I'm not sure what those comments mean.  I assume "dog" is "dmg", and I assume "changes cited" means the changes in the patch, and I assume "linked bugs" means the bugs named in comments in the patch.  I don't know why you'd have trouble viewing them.

But I do know what r+ means.

Comment 25

[Justin Wood (:Callek)]

(In reply to Dustin J. Mitchell [:dustin] from comment #24)
> I'm not sure what those comments mean.  I assume "dog" is "dmg", and I
> assume "changes cited" means the changes in the patch, and I assume "linked
> bugs" means the bugs named in comments in the patch.  I don't know why you'd
> have trouble viewing them.
> 
> But I do know what r+ means.

autocorrect on a cell phone while in a waiting room. it hurts, sorry