919126 - Deploy Python 2.7.3 on SeaMonkey build/test machines

Status: RESOLVED FIXED

(SeaMonkey :: Build Config, defect)

Description

[Frank Wein [:mcsmurf]]

Bug 870420 changed the minimum Python build requirement to Python 2.7.3. Therefor SeaMonkey machines need the new Python version installed as well, at least Linux and OS X, Windows seems to be fine. At the same time Bug 908090 could be fixed as well by removing references to old Python versions in the system PATH (if python 2.5 is set there, not sure).

Comment 1

[Justin Wood (:Callek)]

Creating .i686 as we speak ala Bug 817504...

Comment 2

[Justin Wood (:Callek)]

Attachment: log of rpmbuild for py2.7 i686

Forgive the raw-log dump, and the word-wrap that happened on my terminal:

[cltbld@linux-ix-slave15 rpmbuild]$ ls -l
total 20
drwxrwxr-x 2 cltbld cltbld 4096 Feb  4 00:45 BUILD
drwxrwxr-x 2 cltbld cltbld 4096 Feb  4 00:45 RPMS
drwxrwxr-x 2 cltbld cltbld 4096 Feb  4 00:45 SOURCES
drwxrwxr-x 2 cltbld cltbld 4096 Feb  4 00:45 SPECS
drwxrwxr-x 2 cltbld cltbld 4096 Feb  4 00:45 SRPMS
[cltbld@linux-ix-slave15 rpmbuild]$ cd SPECS/
[cltbld@linux-ix-slave15 SPECS]$ wget -O python27.spec http://hg.mozilla.org/build/rpm-sources/raw-file/963b152d7
186/python26/centos5-i686/python26.spec
--00:47:45--  http://hg.mozilla.org/build/rpm-sources/raw-file/963b152d7186/python26/centos5-i686/python26.spec
Resolving hg.mozilla.org... 63.245.215.25
Connecting to hg.mozilla.org|63.245.215.25|:80... connected.
HTTP request sent, awaiting response... 200 Script output follows
Length: 1131 (1.1K) [text/plain]
Saving to: `python27.spec'

100%[========================================================================>] 1,131       --.-K/s   in 0s

00:47:45 (51.8 MB/s) - `python27.spec' saved [1131/1131]

[cltbld@linux-ix-slave15 SPECS]$ wget -O ../SOURCES/Python-2.7.2.tar.bz2 http://www.python.org/ftp/python/2.7.2/P
ython-2.7.2.tar.bz2
--00:53:54--  http://www.python.org/ftp/python/2.7.2/Python-2.7.2.tar.bz2
Resolving www.python.org... 82.94.164.162, 2001:888:2000:d::a2
Connecting to www.python.org|82.94.164.162|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 11754834 (11M) [application/x-bzip2]
Saving to: `../SOURCES/Python-2.7.2.tar.bz2'
100%[=================================================>] 1,131       --.-K/s   in 0s

09:44:22 (184 MB/s) - `python27.spec' saved [1131/1131]

[cltbld@bld-centos5-32-vmw-001 SPECS]$ wget -O ../SOURCES/Python-2.7.3.tar.bz2 http://www
.python.org/ftp/python/2.7.3/Python-2.7.2.tar.bz2
--09:46:13--  http://www.python.org/ftp/python/2.7.3/Python-2.7.2.tar.bz2
Resolving www.python.org... 82.94.164.162, 2001:888:2000:d::a2
Connecting to www.python.org|82.94.164.162|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
09:46:14 ERROR 404: Not Found.

[cltbld@bld-centos5-32-vmw-001 SPECS]$ wget -O ../SOURCES/Python-2.7.3.tar.bz2 http://www
.python.org/ftp/python/2.7.3/Python-2.7.3.tar.bz2
--09:47:13--  http://www.python.org/ftp/python/2.7.3/Python-2.7.3.tar.bz2
Resolving www.python.org... 82.94.164.162, 2001:888:2000:d::a2
Connecting to www.python.org|82.94.164.162|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 11793433 (11M) [application/x-bzip2]
Saving to: `../SOURCES/Python-2.7.3.tar.bz2'

100%[=================================================>] 11,793,433   613K/s   in 30s

09:47:43 (386 KB/s) - `../SOURCES/Python-2.7.3.tar.bz2' saved [11793433/11793433]

[cltbld@bld-centos5-32-vmw-001 SPECS]$ ls
python27.spec
[cltbld@bld-centos5-32-vmw-001 SPECS]$ nano python27.spec
[cltbld@bld-centos5-32-vmw-001 SPECS]$ wget -O python26.spec http://hg.mozilla.org/build/
rpm-sources/raw-file/tip/python26/centos5-i686/python26.spec
--09:50:52--  http://hg.mozilla.org/build/rpm-sources/raw-file/tip/python26/centos5-i686/p
ython26.spec
Resolving hg.mozilla.org... 63.245.215.25
Connecting to hg.mozilla.org|63.245.215.25|:80... connected.
HTTP request sent, awaiting response... 200 Script output follows
Length: 1131 (1.1K) [text/plain]
Saving to: `python26.spec'

100%[=================================================>] 1,131       --.-K/s   in 0s

09:50:53 (184 MB/s) - `python26.spec' saved [1131/1131]

[cltbld@bld-centos5-32-vmw-001 SPECS]$ diff -U3 python26.spec python27.spec
--- python26.spec       2013-12-23 09:50:53.000000000 -0800
+++ python27.spec       2013-12-23 09:50:14.000000000 -0800
@@ -1,8 +1,8 @@
-%define pybasever 2.6
+%define pybasever 2.7

 Name: python26
 Summary: An interpreted, interactive, object-oriented programming language.
-Version: %{pybasever}.5
+Version: %{pybasever}.3
 Release: 0moz1%{?dist}
 License: PSF - see LICENSE
 Group: Development/Languages

[cltbld@bld-centos5-32-vmw-001 SPECS]$ echo '%_topdir /builds/rpmbuild' > ~/.rpmmacros
[cltbld@bld-centos5-32-vmw-001 SPECS]$ rpmbuild -ba --target=i686 python27.spec | tee ../log.txt

That log.txt is attached

[cltbld@bld-centos5-32-vmw-001 SPECS]$ ls ../RPMS/i686/
python26-2.7.3-0moz1.i686.rpm  python26-debuginfo-2.7.3-0moz1.i686.rpm
[cltbld@bld-centos5-32-vmw-001 SPECS]$ ls ../SRPMS/
python26-2.7.3-0moz1.src.rpm

THOSE files are now uploaded to: http://people.mozilla.org/~jwood/rpms/

Lastly cleaned up the generated files on the slave, and returned slave to the MoCo pool

Comment 3

[Justin Wood (:Callek)]

(In reply to Justin Wood (:Callek) from comment #2)
> Created attachment 8351048 [details]
> log of rpmbuild for py2.7 i686
> 
> Forgive the raw-log dump, and the word-wrap that happened on my terminal:
> 
> [cltbld@linux-ix-slave15 rpmbuild]$ ls -l
> total 20
> drwxrwxr-x 2 cltbld cltbld 4096 Feb  4 00:45 BUILD
> drwxrwxr-x 2 cltbld cltbld 4096 Feb  4 00:45 RPMS
> drwxrwxr-x 2 cltbld cltbld 4096 Feb  4 00:45 SOURCES
> drwxrwxr-x 2 cltbld cltbld 4096 Feb  4 00:45 SPECS
> drwxrwxr-x 2 cltbld cltbld 4096 Feb  4 00:45 SRPMS
> [cltbld@linux-ix-slave15 rpmbuild]$ cd SPECS/
> [cltbld@linux-ix-slave15 SPECS]$ wget -O python27.spec
> http://hg.mozilla.org/build/rpm-sources/raw-file/963b152d7
> 186/python26/centos5-i686/python26.spec
> --00:47:45-- 
> http://hg.mozilla.org/build/rpm-sources/raw-file/963b152d7186/python26/
> centos5-i686/python26.spec
> Resolving hg.mozilla.org... 63.245.215.25
> Connecting to hg.mozilla.org|63.245.215.25|:80... connected.
> HTTP request sent, awaiting response... 200 Script output follows
> Length: 1131 (1.1K) [text/plain]
> Saving to: `python27.spec'
> 
> 100%[========================================================================
> >] 1,131       --.-K/s   in 0s
> 
> 00:47:45 (51.8 MB/s) - `python27.spec' saved [1131/1131]
> 
> [cltbld@linux-ix-slave15 SPECS]$ wget -O ../SOURCES/Python-2.7.2.tar.bz2
> http://www.python.org/ftp/python/2.7.2/P
> ython-2.7.2.tar.bz2
> --00:53:54--  http://www.python.org/ftp/python/2.7.2/Python-2.7.2.tar.bz2
> Resolving www.python.org... 82.94.164.162, 2001:888:2000:d::a2
> Connecting to www.python.org|82.94.164.162|:80... connected.
> HTTP request sent, awaiting response... 200 OK
> Length: 11754834 (11M) [application/x-bzip2]
> Saving to: `../SOURCES/Python-2.7.2.tar.bz2'
> 100%[=================================================>] 1,131       --.-K/s
> in 0s
> 
> 09:44:22 (184 MB/s) - `python27.spec' saved [1131/1131]
> 
> [cltbld@bld-centos5-32-vmw-001 SPECS]$ wget -O
> ../SOURCES/Python-2.7.3.tar.bz2 http://www
> .python.org/ftp/python/2.7.3/Python-2.7.2.tar.bz2
> --09:46:13--  http://www.python.org/ftp/python/2.7.3/Python-2.7.2.tar.bz2
> Resolving www.python.org... 82.94.164.162, 2001:888:2000:d::a2
> Connecting to www.python.org|82.94.164.162|:80... connected.
> HTTP request sent, awaiting response... 404 Not Found
> 09:46:14 ERROR 404: Not Found.
> 
> [cltbld@bld-centos5-32-vmw-001 SPECS]$ wget -O
> ../SOURCES/Python-2.7.3.tar.bz2 http://www
> .python.org/ftp/python/2.7.3/Python-2.7.3.tar.bz2
> --09:47:13--  http://www.python.org/ftp/python/2.7.3/Python-2.7.3.tar.bz2
> Resolving www.python.org... 82.94.164.162, 2001:888:2000:d::a2
> Connecting to www.python.org|82.94.164.162|:80... connected.
> HTTP request sent, awaiting response... 200 OK
> Length: 11793433 (11M) [application/x-bzip2]
> Saving to: `../SOURCES/Python-2.7.3.tar.bz2'
> 
> 100%[=================================================>] 11,793,433   613K/s
> in 30s
> 
> 09:47:43 (386 KB/s) - `../SOURCES/Python-2.7.3.tar.bz2' saved
> [11793433/11793433]
> 
> [cltbld@bld-centos5-32-vmw-001 SPECS]$ ls
> python27.spec
> [cltbld@bld-centos5-32-vmw-001 SPECS]$ nano python27.spec
> [cltbld@bld-centos5-32-vmw-001 SPECS]$ wget -O python26.spec
> http://hg.mozilla.org/build/
> rpm-sources/raw-file/tip/python26/centos5-i686/python26.spec
> --09:50:52-- 
> http://hg.mozilla.org/build/rpm-sources/raw-file/tip/python26/centos5-i686/p
> ython26.spec
> Resolving hg.mozilla.org... 63.245.215.25
> Connecting to hg.mozilla.org|63.245.215.25|:80... connected.
> HTTP request sent, awaiting response... 200 Script output follows
> Length: 1131 (1.1K) [text/plain]
> Saving to: `python26.spec'
> 
> 100%[=================================================>] 1,131       --.-K/s
> in 0s
> 
> 09:50:53 (184 MB/s) - `python26.spec' saved [1131/1131]
> 
> [cltbld@bld-centos5-32-vmw-001 SPECS]$ diff -U3 python26.spec python27.spec
> --- python26.spec       2013-12-23 09:50:53.000000000 -0800
> +++ python27.spec       2013-12-23 09:50:14.000000000 -0800
> @@ -1,8 +1,8 @@
> -%define pybasever 2.6
> +%define pybasever 2.7
> 
>  Name: python26
>  Summary: An interpreted, interactive, object-oriented programming language.
> -Version: %{pybasever}.5
> +Version: %{pybasever}.3
>  Release: 0moz1%{?dist}
>  License: PSF - see LICENSE
>  Group: Development/Languages
> 
> [cltbld@bld-centos5-32-vmw-001 SPECS]$ echo '%_topdir /builds/rpmbuild' >
> ~/.rpmmacros
> [cltbld@bld-centos5-32-vmw-001 SPECS]$ rpmbuild -ba --target=i686
> python27.spec | tee ../log.txt
> 
> That log.txt is attached
> 
> [cltbld@bld-centos5-32-vmw-001 SPECS]$ ls ../RPMS/i686/
> python26-2.7.3-0moz1.i686.rpm  python26-debuginfo-2.7.3-0moz1.i686.rpm
> [cltbld@bld-centos5-32-vmw-001 SPECS]$ ls ../SRPMS/
> python26-2.7.3-0moz1.src.rpm
> 
> THOSE files are now uploaded to: http://people.mozilla.org/~jwood/rpms/
> 
> Lastly cleaned up the generated files on the slave, and returned slave to
> the MoCo pool


....and this one is bad (note: python26-2.7.3) I'll fix in a moment

Comment 4

[Justin Wood (:Callek)]

Attachment: log of rpmbuild for py2.7 x86_64

[root@sea-vm-linux64-1 builds]# mkdir rpmbuild
[root@sea-vm-linux64-1 builds]# cd rpmbuild/
[root@sea-vm-linux64-1 rpmbuild]# mkdir {BUILD,RPMS,SOURCES,SPECS,SRPMS}
[root@sea-vm-linux64-1 rpmbuild]# cd SPECS/
[root@sea-vm-linux64-1 SPECS]# wget -O python27.spec http://hg.mozilla.org/build/rpm-sour
ces/raw-file/tip/python26/centos5-i686/python26.spec
--14:53:54--  http://hg.mozilla.org/build/rpm-sources/raw-file/tip/python26/centos5-i686/p
ython26.spec
Resolving hg.mozilla.org... 63.245.215.25
Connecting to hg.mozilla.org|63.245.215.25|:80... connected.
HTTP request sent, awaiting response... 200 Script output follows
Length: 1131 (1.1K) [text/plain]
Saving to: `python27.spec'

100%[=================================================>] 1,131       --.-K/s   in 0s

14:53:54 (216 MB/s) - `python27.spec' saved [1131/1131]

[root@sea-vm-linux64-1 SPECS]# wget -O ../SOURCES/Python-2.7.3.tar.bz2 http://www.python.
org/ftp/python/2.7.3/Python-2.7.3.tar.bz2
--14:54:55--  http://www.python.org/ftp/python/2.7.3/Python-2.7.3.tar.bz2
Resolving www.python.org... 82.94.164.162, 2001:888:2000:d::a2
Connecting to www.python.org|82.94.164.162|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 11793433 (11M) [application/x-bzip2]
Saving to: `../SOURCES/Python-2.7.3.tar.bz2'

100%[=================================================>] 11,793,433  1.54M/s   in 11s

14:55:06 (1.02 MB/s) - `../SOURCES/Python-2.7.3.tar.bz2' saved [11793433/11793433]

[root@sea-vm-linux64-1 SPECS]# nano python27.spec
[root@sea-vm-linux64-1 SPECS]# diff -U3 python26.spec python27.spec
--- python26.spec       2013-12-22 14:56:26.000000000 -0800
+++ python27.spec       2013-12-22 14:58:56.000000000 -0800
@@ -1,8 +1,8 @@
-%define pybasever 2.6
+%define pybasever 2.7

-Name: python26
+Name: python27
 Summary: An interpreted, interactive, object-oriented programming language.
-Version: %{pybasever}.5
+Version: %{pybasever}.3
 Release: 0moz1%{?dist}
 License: PSF - see LICENSE
 Group: Development/Languages
[root@sea-vm-linux64-1 SPECS]# echo '%_topdir /builds/rpmbuild' > ~/.rpmmacros
[root@sea-vm-linux64-1 SPECS]# rpmbuild -ba --target=x86_64 python27.spec | tee ../log.txt

This log is attached

[root@sea-vm-linux64-1 SPECS]# ls ../RPMS/x86_64/
python27-2.7.3-0moz1.x86_64.rpm  python27-debuginfo-2.7.3-0moz1.x86_64.rpm
[root@sea-vm-linux64-1 SPECS]# ls ../SRPMS/
python27-2.7.3-0moz1.src.rpm

THOSE files are now uploaded to: http://people.mozilla.org/~jwood/rpms/

Lastly cleaned up the generated files on the slave, and returned slave to our pool

Comment 5

[Justin Wood (:Callek)]

Disabled gpgcheck and installed on our linux64 -- and triggered a [dep] beta build to see if it works on m-beta

Comment 6

[Justin Wood (:Callek)]

Attachment: log of rpmbuild for py2.7 i686 (real)

[cltbld@bld-centos5-32-vmw-001 builds]# mkdir rpmbuild
[cltbld@bld-centos5-32-vmw-001 builds]# cd rpmbuild/
[cltbld@bld-centos5-32-vmw-001 rpmbuild]# mkdir {BUILD,RPMS,SOURCES,SPECS,SRPMS}
[cltbld@bld-centos5-32-vmw-001 rpmbuild]$ cd SPECS/
[cltbld@bld-centos5-32-vmw-001 SPECS]$ wget -O python27.spec http://hg.mozilla.org/build/
rpm-sources/raw-file/tip/python26/centos5-i686/python26.spec
--11:59:29--  http://hg.mozilla.org/build/rpm-sources/raw-file/tip/python26/centos5-i686/p
ython26.spec
Resolving hg.mozilla.org... 63.245.215.25
Connecting to hg.mozilla.org|63.245.215.25|:80... connected.
HTTP request sent, awaiting response... 200 Script output follows
Length: 1131 (1.1K) [text/plain]
Saving to: `python27.spec'

100%[=================================================>] 1,131       --.-K/s   in 0s

11:59:29 (2.16 MB/s) - `python27.spec' saved [1131/1131]

[cltbld@bld-centos5-32-vmw-001 SPECS]$ wget -O ../SOURCES/Python-2.7.3.tar.bz2 http://www
.python.org/ftp/python/2.7.3/Python-2.7.3.tar.bz2
--11:59:45--  http://www.python.org/ftp/python/2.7.3/Python-2.7.3.tar.bz2
Resolving www.python.org... 82.94.164.162, 2001:888:2000:d::a2
Connecting to www.python.org|82.94.164.162|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 11793433 (11M) [application/x-bzip2]
Saving to: `../SOURCES/Python-2.7.3.tar.bz2'

100%[=================================================>] 11,793,433   867K/s   in 14s

11:59:59 (810 KB/s) - `../SOURCES/Python-2.7.3.tar.bz2' saved [11793433/11793433]

[cltbld@bld-centos5-32-vmw-001 SPECS]$ nano python27.spec
[cltbld@bld-centos5-32-vmw-001 SPECS]$ echo '%_topdir /builds/rpmbuild' > ~/.rpmmacros
[cltbld@bld-centos5-32-vmw-001 SPECS]$ rpmbuild -ba --target=i686 python27.spec | tee ../
log.txt


....and thats at http://people.mozilla.org/~jwood/rpms/python27-2.7.3-0moz1.i686.rpm

Comment 7

[Justin Wood (:Callek)]

Installed on sea-vm-linux32-{1..6} with:

sed -i 's/gpgcheck=1/gpgcheck=0/' /etc/yum.conf
yum install -y ./python27-2.7.3-0moz1.i686.rpm
sed -i 's/gpgcheck=0/gpgcheck=1/' /etc/yum.conf
rm -f ./python27-2.7.3-0moz1.i686.rpm

Comment 8

[Edmund Wong (:ewong)]

Installed on sea-mini-osx64-{1..4}:

*from python27-2.7.3-1.dmg

  - /usr/bin/hdiutil mount -plist -nobrowse -readonly -noidme -mountrandom /tmp ./python27-2.7.3-1.dmg
  - sudo /usr/sbin/installer -verbose -pkg /tmp/dmg.<randomchars>/python27-2.7.3-1.pkg -target /
  - /usr/bin/hdiutil eject /tmp/dmg.<randomchars>

*then
  - sudo su -
  -  ls -la /tools
  -  rm -rf /tools/python /tools/buildbot-0.8.4-pre-moz2
  -  ln -s /tools/python27 /tools/python
  -  ln -s /tools/python27 /tools/python2
  -  wget -O /tools/misc-python/virtualenv.py http://puppetagain.pub.build.mozilla.org/data/python/packages/virtualenv.py; rm -f /tools/misc-python/*.pyc
  -  wget -O /tools/misc-python/distribute-0.6.24.tar.gz http://puppetagain.pub.build.mozilla.org/data/python/packages/distribute-0.6.24.tar.gz
  -  mkdir /tools/buildbot-0.8.4-pre-moz2
  -  /tools/python27/bin/python2.7 -BE /tools/misc-python/virtualenv.py --python=/tools/python27/bin/python2.7 --distribute --never-download /tools/buildbot-0.8.4-pre-moz2
  -  /tools/buildbot-0.8.4-pre-moz2/bin/pip install --no-deps --no-index --find-links=http://puppetagain.pub.build.mozilla.org/data/python/packages/ zope.interface==3.6.1
  -  /tools/buildbot-0.8.4-pre-moz2/bin/pip install --no-deps --no-index --find-links=http://puppetagain.pub.build.mozilla.org/data/python/packages/ buildbot-slave==0.8.4-pre-moz2
  -  /tools/buildbot-0.8.4-pre-moz2/bin/pip install --no-deps --no-index --find-links=http://puppetagain.pub.build.mozilla.org/data/python/packages/ buildbot==0.8.4-pre-moz2
  -  /tools/buildbot-0.8.4-pre-moz2/bin/pip install --no-deps --no-index --find-links=http://puppetagain.pub.build.mozilla.org/data/python/packages/ Twisted==10.2.0
  -  /tools/buildbot-0.8.4-pre-moz2/bin/pip install --no-deps --no-index --find-links=http://puppetagain.pub.build.mozilla.org/data/python/packages/ simplejson==2.1.3
  -  vi /tools/tooltool.py
      - changed shebang |python| to |python2.7|

  -  sudo reboot

Comment 9

[Edmund Wong (:ewong)]

(In reply to Edmund Wong (:ewong) from comment #8)

>   -  vi /tools/tooltool.py
>       - changed shebang |python| to |python2.7|

To be specific:

#! /tools/python/bin/python2.7

Comment 10

[Edmund Wong (:ewong)]

Attachment: linux/linux-debug/linux64 config changes for Python 2.7.3 [checked-in]

Comment 11

[Tony Mechelynck [:tonymec]]

Linux trunk builds started being built on both L32 and L64 starting 3 January, but after the 7 January nightly there were no more L64 builds. L32 is still being built. See bug 958656. The failure logs say: Python 2.7.3 required, 2.6.5 found.

2.5a2 Aurora is still not being built on Linux.

Comment 12

[Edmund Wong (:ewong)]

OTR:

Currently things are working because the configs were manually patched.
I've just reverted the linux/linux-debug/linux64 changes.

I'll be pushing the linux/linux-debug/linux64 patch for bustagefix (still
for a post push review).

This should 'allow me' to apply the wip patch for bug 908090 to fix
the unittests.

Comment 13

[Edmund Wong (:ewong)]

Pushed to buildbot-configs:

http://hg.mozilla.org/build/buildbot-configs/rev/324edd42ff53

Comment 14

[Frank Wein [:mcsmurf]]

Is this bug still relevant in any way? As a reminder: It was about installing Python 2.7.3 and changing the buildbot config.

Comment 15

[Edmund Wong (:ewong)]

patch was pushed.  Callek has approved me dropping reviews for already-pushed patches.