734466 - Run talos on python 2.7 in production

Status: RESOLVED FIXED

(Testing :: Talos, defect)

Description

[William Lachance (:wlach)]

(this deprecates an earlier bug to make talos run on python 2.5 in production, which we decided not to do as it's not worth the effort if we're going to do this anyway)

We would like to run talos on python 2.7 in production, which matches what most people use for development these days. 

There's a few dependencies here, in addition to just installing python 2.7 on the slaves:

1. We need to either install a copy of PyYAML for python 2.7 or eliminate the need for it
2. We need to either install a copy of pywin32 for python 2.7 or eliminate the need for it

Comment 1

[William Lachance (:wlach)]

So, to move from requirements to implementation, here's my plan:

(1) Land bug

Comment 2

[William Lachance (:wlach)]

So, to move from requirements to implementation, here's my plan (as approved by armenzg and ctalbert):

(1) Land bug 734136
(2) Put python 2.7 on all staging machines. Apply pywin32 patch (bug 726700) on staging. Do test runs to make sure things still work.
(3) Coordinate with releng to get python 2.7 installed on all test machines (bug 711299)
(4) Land bug 726700, move to python 2.7 in production.

Comment 3

[William Lachance (:wlach)]

(2) is half done. We've verified that things still work in staging with python 2.7 (+ my pywin32 patch on win32), but someone (maybe :jmaher) told me still need to make sure that changing the python version doesn't affect the talos numbers. I don't really see why they would, but you never know I guess.

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: change buildbot to run windows talos and unittests to use python 2.7 (1.0)

Comment 5

[Armen [:armenzg]]

Comment on attachment 675119 [details] [diff] [review]
change buildbot to run windows talos and unittests to use python 2.7 (1.0)

This is correct but we will need to run it through staging.
We have to also remember that this could affect esr10.

Comment 6

[Armen [:armenzg]]

You will see a win7 slave and a winxp slave take opt, debug and talos jobs in here:
http://dev-master01.build.scl1.mozilla.com:8041/one_line_per_build

The build triggered is for http://hg.mozilla.org/mozilla-central/rev/d3de9d8e2b5b.

Can I have a talos.zip that would not need pywin32?

I got this:
talos-r3-w7-010: 
		Started Thu, 25 Oct 2012 10:46:42
Running test tscrollr: 
		Started Thu, 25 Oct 2012 10:46:42
Traceback (most recent call last):
  File "run_tests.py", line 303, in <module>
    main()
  File "run_tests.py", line 300, in main
    run_tests(parser)
  File "run_tests.py", line 248, in run_tests
    mytest = TTest(browser_config['remote'])
  File "c:\talos-slave\talos-data\talos\ttest.py", line 84, in __init__
    cmanager, platformtype, ffprocess = self.getPlatformType(remote)
  File "c:\talos-slave\talos-data\talos\ttest.py", line 112, in getPlatformType
    import cmanager_win32
  File "c:\talos-slave\talos-data\talos\cmanager_win32.py", line 5, in <module>
    import win32pdh
ImportError: No module named win32pdh

Comment 7

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

There's the patch in bug 726700 for pywin32 removal, I think.

Comment 8

[William Lachance (:wlach)]

(In reply to Gary Kwong [:gkw, :nth10sd] from comment #7)
> There's the patch in bug 726700 for pywin32 removal, I think.

It's bitrotted a bit (just eyeballing, the create_talos_zip parts are obsolete), but probably wouldn't be too difficult to get working again.

Comment 9

[Armen [:armenzg]]

I'm also adding this to the mix:
diff -r 0cebf7d7070b process/factory.py
--- a/process/factory.py        Wed Oct 24 22:02:26 2012 -0400
+++ b/process/factory.py        Thu Oct 25 14:05:49 2012 -0700
@@ -5323,7 +5323,7 @@
         if (platform in ("fedora", "fedora64", "leopard", "snowleopard", "lion", "mountainlion")):
             return "/tools/buildbot/bin/python"
         elif (platform in ('w764', 'win7', 'xp')):
-            return "C:\\mozilla-build\\python25\\python.exe"
+            return "C:\\mozilla-build\\python27\\python.exe"
         elif (platform.find("android") > -1):
             # path in the foopies
             return "/usr/local/bin/python2.6"

Comment 10

[Aki Sasaki (not active)]

Attachment: determine talos python version in mozharness

Comment 11

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 701247 [details] [diff] [review]
determine talos python version in mozharness

great, lets leave this open or file a follow up bug to remove this once we have the data.

Comment 12

[Aki Sasaki (not active)]

Comment on attachment 701247 [details] [diff] [review]
determine talos python version in mozharness

http://hg.mozilla.org/build/mozharness/rev/8b709d339c2b

Comment 13

[Joel Maher ( :jmaher ) (UTC -8)]

this appears to be fixed now that we are running talos in mozharness on firefox25 based trees.

Comment 14

[Armen [:armenzg]]

(In reply to Joel Maher (:jmaher) from comment #13)
> this appears to be fixed now that we are running talos in mozharness on
> firefox25 based trees.

Agreed.

Comment 15

[Pete Moore [:pmoore][:pete]]

ted
12:05:34 are we running b2g emulator tests on machines with python 2.6?
12:06:16 https://tbpl.mozilla.org/php/getParsedLog.php?id=36063865&tree=Try&full=1
12:06:25 05:58:34     INFO -      with gzip.GzipFile(filename, "wb", fileobj=tf) as gz:
12:06:25 05:58:34     INFO -  AttributeError: GzipFile instance has no attribute '__exit__'
12:06:32 gerard-majax__ [[e-mail]] entered the room.
ted
12:06:45  Changed in version 2.7: Support for the with statement was added.
12:13:35 kmoir is now known as kmoir-afk
12:15:21 kmoir-afk left the room (quit: Ping timeout).

pmoore
12:18:56 ted: i'm checking
12:24:16 RudyL left the room (quit: Quit: RudyL).
12:30:32 dminor [[e-mail]] entered the room.

pmoore
12:34:13 ted: sorry for delay. yes it appears /home/cltbld/talos-slave/test/build/venv/bin/python is version 2.6.2

ted
12:35:05 ick

pmoore
12:36:07 i know there is a lot of history to which particular python version we are using and where - but i don't know the details personally - i think the guys will know who come online in the next few hours
12:36:25 is this something new - like previously it was 2.7 but for some reason it has regressed?
12:36:42 or is it just a surprise that it wasn't 2.7, that you hadn't expected, but maybe it was always like this?

ted
12:36:47 more likely the latter
12:36:55 we ought to be running 2.7 everywhere
12:37:07 i would bet that that machine has 2.7 hiding somewhere

pmoore
12:37:14 let me see if i can mine the bugs to find out why

ted
12:37:16 it's just not being used

pmoore
12:37:46 i can run a search to see
12:38:23 but my guess is there was a reason we didn't upgrade - i'll find out...

ted
12:40:45 i doubt it's intentional
12:40:50 more likely to have been overlooked
12:41:57 https://bugzilla.mozilla.org/show_bug.cgi?id=711299

pmoore
12:42:05 https://bugzilla.mozilla.org/show_bug.cgi?id=734466
12:42:36 hmmm yes, i tend to agree with you
12:42:55 several bugs for upgrading to python 2.7.3, as far as i can see
12:44:29 ted: i'm going to quickly grab lunch, and get back to you on this

Comment 16

[Pete Moore [:pmoore][:pete]]

host where test ran is talos-r3-fed-098

python called from:
/home/cltbld/talos-slave/test/build/venv/bin/python

Comment 17

[Pete Moore [:pmoore][:pete]]

OK, this python version was created by:
Running command: ['/tools/buildbot/bin/python', '/tools/misc-python/virtualenv.py', '--no-site-packages', '--distribute', '/home/cltbld/talos-slave/test/build/venv'] in /home/cltbld/talos-slave/test/build

so the virtualenv python binary (/home/cltbld/talos-slave/test/build/venv/bin/python) originated from /tools/buildbot/bin/python:

[cltbld@talos-r3-fed-098 ~]$ /tools/buildbot/bin/python --version
Python 2.6.2

Comment 18

[Armen [:armenzg]]

FTR, we're moving the b2g reftests from the r3 machines to EC2 probably before the end of April (hopefully before).

Not worth fixing.