786885 - jetperf script doesn't work on windows

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P3)

Description

[Jeff Hammel]

(mozharness) C:\Users\jhammel\mozharness\src\mozharness>python scripts\jetperf.p
y --config-file configs\jetperf\test_config.py
16:55:37     INFO - MultiFileLogger online at 20120829 16:55:37 in C:\Users\jham
mel\mozharness\src\mozharness
16:55:37     INFO - Run as scripts\jetperf.py --config-file configs\jetperf\test
_config.py
16:55:37     INFO - Dumping config to C:\Users\jhammel\mozharness\src\mozharness
\logs\localconfig.json.
16:55:37     INFO - {'addon-directories': [],
16:55:37     INFO -  'append_to_log': False,
16:55:37     INFO -  'base_work_dir': 'C:\\Users\\jhammel\\mozharness\\src\\mozh
arness',
16:55:37     INFO -  'config_file': 'configs\\jetperf\\test_config.py',
16:55:37     INFO -  'datazilla_urls': ['file://C:\\Users\\jhammel\\mozharness\\
src\\mozharness\\20120829_1655.json'],
16:55:37     INFO -  'default_actions': ['clobber',
16:55:37     INFO -                      'pull',
16:55:37     INFO -                      'build',
16:55:37     INFO -                      'create-virtualenv',
16:55:37     INFO -                      'test',
16:55:37     INFO -                      'baseline'],
16:55:37     INFO -  'log_level': 'info',
16:55:37     INFO -  'log_to_console': True,
16:55:37     INFO -  'repos': [{'dest': 'addon-sdk',
16:55:37     INFO -             'repo': 'http://hg.mozilla.org/projects/addon-sd
k',
16:55:37     INFO -             'revision': 'default'},
16:55:37     INFO -            {'dest': 'addons_clone',
16:55:37     INFO -             'repo': 'http://hg.mozilla.org/projects/addon-sd
k-jetperf-tests/',
16:55:37     INFO -             'revision': 'default'}],
16:55:37     INFO -  'talos_url': 'http://hg.mozilla.org/build/talos/archive/tip
.tar.gz',
16:55:37     INFO -  'tests': ['ts'],
16:55:37     INFO -  'virtualenv_modules': ['talos', 'mozinstall'],
16:55:37     INFO -  'virtualenv_path': 'venv',
16:55:37     INFO -  'volatile_config': {'actions': None, 'add_actions': None, '
no_actions': None},
16:55:37     INFO -  'work_dir': 'build'}
16:55:37     INFO - #####
16:55:37     INFO - ##### Running clobber step.
16:55:37     INFO - #####
16:55:37     INFO - rmtree: C:\Users\jhammel\mozharness\src\mozharness\build
16:55:37     INFO - #####
16:55:37     INFO - ##### Running pull step.
16:55:37     INFO - #####
16:55:37     INFO - mkdir: C:\Users\jhammel\mozharness\src\mozharness\build
16:55:37     INFO - Changing directory to C:\Users\jhammel\mozharness\src\mozhar
ness\build.
16:55:37     INFO - Setting C:\Users\jhammel\mozharness\src\mozharness\build\add
on-sdk to http://hg.mozilla.org/projects/addon-sdk revision default.
16:55:37     INFO - Cloning http://hg.mozilla.org/projects/addon-sdk to C:\Users
\jhammel\mozharness\src\mozharness\build\addon-sdk.
16:55:37     INFO - Running command: ['hg', '--config', 'ui.merge=internal:merge
', 'clone', 'http://hg.mozilla.org/projects/addon-sdk', 'C:\\Users\\jhammel\\moz
harness\\src\\mozharness\\build\\addon-sdk']
16:55:37     INFO - Copy/paste: hg --config ui.merge=internal:merge clone http:/
/hg.mozilla.org/projects/addon-sdk C:\Users\jhammel\mozharness\src\mozharness\bu
ild\addon-sdk
16:55:56     INFO -  requesting all changes
16:55:56     INFO -  adding changesets
16:55:56     INFO -  adding manifests
16:55:56     INFO -  adding file changes
16:55:56     INFO -  added 3511 changesets with 8082 changes to 1648 files
16:55:56     INFO -  updating to branch default
16:55:56     INFO -  714 files updated, 0 files merged, 0 files removed, 0 files
 unresolved
16:55:56     INFO - Return code: 0
16:55:56     INFO - Updating C:\Users\jhammel\mozharness\src\mozharness\build\ad
don-sdk.
16:55:56     INFO - Getting output from command: ['hg', '--config', 'ui.merge=in
ternal:merge', 'branch'] in C:\Users\jhammel\mozharness\src\mozharness\build\add
on-sdk
16:55:56     INFO - Copy/paste: hg --config ui.merge=internal:merge branch
16:55:56     INFO - Reading from file tmpfile_stdout
16:55:56     INFO - Output received:
16:55:56     INFO -  default
16:55:56     INFO - Running command: ['hg', '--config', 'ui.merge=internal:merge
', 'update', '-C'] in C:\Users\jhammel\mozharness\src\mozharness\build\addon-sdk

16:55:56     INFO - Copy/paste: hg --config ui.merge=internal:merge update -C
16:55:56     INFO -  0 files updated, 0 files merged, 0 files removed, 0 files u
nresolved
16:55:56     INFO - Return code: 0
16:55:56     INFO - Getting output from command: ['hg', '--config', 'ui.merge=in
ternal:merge', 'parent', '--template', '{node|short}'] in C:\Users\jhammel\mozha
rness\src\mozharness\build\addon-sdk
16:55:56     INFO - Copy/paste: hg --config ui.merge=internal:merge parent --tem
plate {node|short}
16:55:57     INFO - Reading from file tmpfile_stdout
16:55:57     INFO - Output received:
16:55:57     INFO -  cedfa109ee69
16:55:57     INFO - Updating C:\Users\jhammel\mozharness\src\mozharness\build\ad
don-sdk revision default.
16:55:57     INFO - Running command: ['hg', '--config', 'ui.merge=internal:merge
', 'update', '-C', '-r', 'default'] in C:\Users\jhammel\mozharness\src\mozharnes
s\build\addon-sdk
16:55:57     INFO - Copy/paste: hg --config ui.merge=internal:merge update -C -r
 default
16:55:57     INFO -  0 files updated, 0 files merged, 0 files removed, 0 files u
nresolved
16:55:57     INFO - Return code: 0
16:55:57     INFO - Getting output from command: ['hg', '--config', 'ui.merge=in
ternal:merge', 'parent', '--template', '{node|short}'] in C:\Users\jhammel\mozha
rness\src\mozharness\build\addon-sdk
16:55:57     INFO - Copy/paste: hg --config ui.merge=internal:merge parent --tem
plate {node|short}
16:55:57     INFO - Reading from file tmpfile_stdout
16:55:57     INFO - Output received:
16:55:57     INFO -  cedfa109ee69
16:55:57     INFO - Setting C:\Users\jhammel\mozharness\src\mozharness\build\add
ons_clone to http://hg.mozilla.org/projects/addon-sdk-jetperf-tests/ revision de
fault.
16:55:57     INFO - Cloning http://hg.mozilla.org/projects/addon-sdk-jetperf-tes
ts/ to C:\Users\jhammel\mozharness\src\mozharness\build\addons_clone.
16:55:57     INFO - Running command: ['hg', '--config', 'ui.merge=internal:merge
', 'clone', 'http://hg.mozilla.org/projects/addon-sdk-jetperf-tests/', 'C:\\User
s\\jhammel\\mozharness\\src\\mozharness\\build\\addons_clone']
16:55:57     INFO - Copy/paste: hg --config ui.merge=internal:merge clone http:/
/hg.mozilla.org/projects/addon-sdk-jetperf-tests/ C:\Users\jhammel\mozharness\sr
c\mozharness\build\addons_clone
16:55:59     INFO -  requesting all changes
16:55:59     INFO -  adding changesets
16:55:59     INFO -  adding manifests
16:55:59     INFO -  adding file changes
16:55:59     INFO -  added 2 changesets with 11 changes to 10 files
16:55:59     INFO -  updating to branch default
16:55:59     INFO -  10 files updated, 0 files merged, 0 files removed, 0 files
unresolved
16:55:59     INFO - Return code: 0
16:55:59     INFO - Updating C:\Users\jhammel\mozharness\src\mozharness\build\ad
dons_clone.
16:55:59     INFO - Getting output from command: ['hg', '--config', 'ui.merge=in
ternal:merge', 'branch'] in C:\Users\jhammel\mozharness\src\mozharness\build\add
ons_clone
16:55:59     INFO - Copy/paste: hg --config ui.merge=internal:merge branch
16:55:59     INFO - Reading from file tmpfile_stdout
16:55:59     INFO - Output received:
16:55:59     INFO -  default
16:55:59     INFO - Running command: ['hg', '--config', 'ui.merge=internal:merge
', 'update', '-C'] in C:\Users\jhammel\mozharness\src\mozharness\build\addons_cl
one
16:55:59     INFO - Copy/paste: hg --config ui.merge=internal:merge update -C
16:56:00     INFO -  0 files updated, 0 files merged, 0 files removed, 0 files u
nresolved
16:56:00     INFO - Return code: 0
16:56:00     INFO - Getting output from command: ['hg', '--config', 'ui.merge=in
ternal:merge', 'parent', '--template', '{node|short}'] in C:\Users\jhammel\mozha
rness\src\mozharness\build\addons_clone
16:56:00     INFO - Copy/paste: hg --config ui.merge=internal:merge parent --tem
plate {node|short}
16:56:00     INFO - Reading from file tmpfile_stdout
16:56:00     INFO - Output received:
16:56:00     INFO -  979a2de7e5b7
16:56:00     INFO - Updating C:\Users\jhammel\mozharness\src\mozharness\build\ad
dons_clone revision default.
16:56:00     INFO - Running command: ['hg', '--config', 'ui.merge=internal:merge
', 'update', '-C', '-r', 'default'] in C:\Users\jhammel\mozharness\src\mozharnes
s\build\addons_clone
16:56:00     INFO - Copy/paste: hg --config ui.merge=internal:merge update -C -r
 default
16:56:00     INFO -  0 files updated, 0 files merged, 0 files removed, 0 files u
nresolved
16:56:00     INFO - Return code: 0
16:56:00     INFO - Getting output from command: ['hg', '--config', 'ui.merge=in
ternal:merge', 'parent', '--template', '{node|short}'] in C:\Users\jhammel\mozha
rness\src\mozharness\build\addons_clone
16:56:00     INFO - Copy/paste: hg --config ui.merge=internal:merge parent --tem
plate {node|short}
16:56:00     INFO - Reading from file tmpfile_stdout
16:56:00     INFO - Output received:
16:56:00     INFO -  979a2de7e5b7
16:56:00     INFO - Changing directory to C:\Users\jhammel\mozharness\src\mozhar
ness.
16:56:00     INFO - #####
16:56:00     INFO - ##### Running build step.
16:56:00     INFO - #####
16:56:00     INFO - Scanning C:\Users\jhammel\mozharness\src\mozharness\build\ad
dons_clone for addon directories
16:56:00     INFO - Found ['C:\\Users\\jhammel\\mozharness\\src\\mozharness\\bui
ld\\addons_clone\\empty-addon', 'C:\\Users\\jhammel\\mozharness\\src\\mozharness
\\build\\addons_clone\\pagemod-addon', 'C:\\Users\\jhammel\\mozharness\\src\\moz
harness\\build\\addons_clone\\widget-addon'] addon directories
16:56:00     INFO - Building addons: ['C:\\Users\\jhammel\\mozharness\\src\\mozh
arness\\build\\addons_clone\\empty-addon', 'C:\\Users\\jhammel\\mozharness\\src\
\mozharness\\build\\addons_clone\\pagemod-addon', 'C:\\Users\\jhammel\\mozharnes
s\\src\\mozharness\\build\\addons_clone\\widget-addon']
16:56:00     INFO - rmtree: C:\Users\jhammel\mozharness\src\mozharness\build\add
ons
16:56:00     INFO - mkdir: C:\Users\jhammel\mozharness\src\mozharness\build\addo
ns
16:56:00     INFO - Running command: ['C:\\Users\\jhammel\\mozharness\\src\\mozh
arness\\build\\addon-sdk\\bin\\cfx', 'xpi'] in C:\Users\jhammel\mozharness\src\m
ozharness\build\addons\empty-addon
16:56:00     INFO - Copy/paste: C:\Users\jhammel\mozharness\src\mozharness\build
\addon-sdk\bin\cfx xpi
Traceback (most recent call last):
  File "scripts\jetperf.py", line 254, in <module>
    main()
  File "scripts\jetperf.py", line 251, in main
    jetperf.run()
  File "C:\Users\jhammel\mozharness\src\mozharness\mozharness\base\script.py", l
ine 620, in run
    self._possibly_run_method(method_name, error_if_missing=True)
  File "C:\Users\jhammel\mozharness\src\mozharness\mozharness\base\script.py", l
ine 591, in _possibly_run_method
    return getattr(self, method_name)()
  File "scripts\jetperf.py", line 160, in build
    code = self.run_command([cfx, 'xpi'], cwd=path)
  File "C:\Users\jhammel\mozharness\src\mozharness\mozharness\base\script.py", l
ine 399, in run_command
    cwd=cwd, stderr=subprocess.STDOUT, env=env)
  File "C:\Python27\Lib\subprocess.py", line 679, in __init__
    errread, errwrite)
  File "C:\Python27\Lib\subprocess.py", line 896, in _execute_child
    startupinfo)
WindowsError: [Error 193] %1 is not a valid Win32 application

This wouldn't have succeeded anyway since i didn't pass a binary path, but it shouldn't have failed here

Comment 1

[Aki Sasaki (not active)]

Should cfx work on windows?

Comment 2

[Jeff Hammel]

(In reply to Aki Sasaki [:aki] from comment #1)
> Should cfx work on windows?

Should?  Probably.  Does? Maybe, but I can't verify:

C:\Users\jhammel\mozharness\src\mozharness\build\addons\empty-addon>c:\Users\jh
mmel\mozharness\src\mozharness\build\addon-sdk\bin\cfx
python: can't open file '\bin\cfx': [Errno 2] No such file or directory

C:\Users\jhammel\mozharness\src\mozharness\build\addons\empty-addon>c:\Users\jh
mmel\mozharness\src\mozharness\build\addon-sdk\bin\cfx.bat
python: can't open file '\bin\cfx': [Errno 2] No such file or directory

:ochameau, what am I doing wrong here?

Comment 3

[Jeff Hammel]

This actually turns out to be not too bad. We just have to call cfx with sys.executable as the first argument and things seem to be fine (until the next error anyway).  I can't seem to install pywin32.  Sad sad sad...

Comment 4

[Jeff Hammel]

Attachment: call this with sys.executable

Comment 5

[Jeff Hammel]

And the next fun:

................................................................................
.............................................................................don
e.
15:46:56     INFO -  Installing pip..................done.
15:46:56     INFO - Return code: 0
15:46:56     INFO - Installing talos into virtualenv C:\Users\jhammel\mozharness
\src\mozharness\build\venv
15:46:56  WARNING - VIRTUAL_ENV C:\Users\jhammel\mozharness set; this may break
mozharness virtualenv calls!
15:46:56     INFO - Running command: ['C:\\Users\\jhammel\\mozharness\\src\\mozh
arness\\build\\venv\\Scripts\\pip', 'install', 'http://hg.mozilla.org/build/talo
s/archive/tip.tar.gz'] in C:\Users\jhammel\mozharness\src\mozharness\build
15:46:56     INFO - Copy/paste: C:\Users\jhammel\mozharness\src\mozharness\build
\venv\Scripts\pip install http://hg.mozilla.org/build/talos/archive/tip.tar.gz
15:47:47     INFO -  Downloading/unpacking http://hg.mozilla.org/build/talos/arc
hive/tip.tar.gz
15:47:47     INFO -    Running setup.py egg_info for package from http://hg.mozi
lla.org/build/talos/archive/tip.tar.gz
15:47:47     INFO -  Downloading/unpacking PyYAML (from talos==0.0)
15:47:47     INFO -    Running setup.py egg_info for package PyYAML
15:47:47     INFO -  Downloading/unpacking mozdevice>=0.2 (from talos==0.0)
15:47:47     INFO -    Downloading mozdevice-0.5.tar.gz
15:47:47     INFO -    Running setup.py egg_info for package mozdevice
15:47:47     INFO -  Downloading/unpacking mozhttpd>=0.3 (from talos==0.0)
15:47:47     INFO -    Downloading mozhttpd-0.3.tar.gz
15:47:47     INFO -    Running setup.py egg_info for package mozhttpd
15:47:47     INFO -  Downloading/unpacking mozinfo (from talos==0.0)
15:47:47     INFO -    Downloading mozinfo-0.3.3.tar.gz
15:47:47     INFO -    Running setup.py egg_info for package mozinfo
15:47:47     INFO -  Downloading/unpacking httplib2 (from talos==0.0)
15:47:47     INFO -    Running setup.py egg_info for package httplib2
15:47:47     INFO -  Downloading/unpacking oauth2 (from talos==0.0)
15:47:47     INFO -    Downloading oauth2-1.5.211.tar.gz
15:47:47     INFO -    Running setup.py egg_info for package oauth2
15:47:47     INFO -  Downloading/unpacking datazilla (from talos==0.0)
15:47:47     INFO -    Downloading datazilla-1.0.1.tar.gz
15:47:47     INFO -    Running setup.py egg_info for package datazilla
15:47:47     INFO -  Downloading/unpacking pywin32 (from talos==0.0)
15:47:47     INFO -    Could not find any downloads that satisfy the requirement
 pywin32 (from talos==0.0)
15:47:47     INFO -  No distributions at all found for pywin32 (from talos==0.0)

15:47:47     INFO -  Storing complete log in C:\Users\jhammel\AppData\Roaming\pi
p\pip.log
15:47:47    ERROR - Return code: 1
15:47:47    FATAL - Unable to install http://hg.mozilla.org/build/talos/archive/
tip.tar.gz!
15:47:47    FATAL - Exiting -1

Comment 6

[Aki Sasaki (not active)]

Comment on attachment 657070 [details] [diff] [review]
call this with sys.executable

Sure.  We prefer sys.executable over self.query_python_path() ?

Comment 7

[Aki Sasaki (not active)]

Specify a pywin32_url in your config file that points to the appropriate version; that should fix that issue.

Comment 8

[Jeff Hammel]

(In reply to Aki Sasaki [:aki] from comment #6)
> Comment on attachment 657070 [details] [diff] [review]
> call this with sys.executable
> 
> Sure.  We prefer sys.executable over self.query_python_path() ?

Yes; cfx does its own virtualenv-like thing.  I don't *think* it matters which python we use, but currently it was (de facto) using the sys.executable and I don't see any reason to change it to use the virtualenv executable (and, in fact the normal steps run 'build' and invoke cfx before the virtualenv is created).

Comment 9

[Jeff Hammel]

(In reply to Jeff Hammel [:jhammel] from comment #4)
> Created attachment 657070 [details] [diff] [review]
> call this with sys.executable

pushed: http://hg.mozilla.org/build/mozharness/rev/07613127a183

Comment 10

[Jeff Hammel]

(In reply to Aki Sasaki [:aki] from comment #7)
> Specify a pywin32_url in your config file that points to the appropriate
> version; that should fix that issue.

So I'm tempted to mirror the logic from http://hg.mozilla.org/build/talos/file/1c5976f92643/setup.py#l28 to the appropriate mozharness configs.  Should I add this to http://hg.mozilla.org/build/mozharness/file/ca3b37a3b8ff/configs/jetperf/windows_config.py and/or http://hg.mozilla.org/build/mozharness/file/ca3b37a3b8ff/configs/jetperf/prod_config.py and/or http://hg.mozilla.org/build/mozharness/file/ca3b37a3b8ff/configs/jetperf/test_config.py?

Comment 11

[Aki Sasaki (not active)]

You don't need to figure out the os, since we're only using windows_config.py on windows.

We only need to set pywin32_url in windows_config.py, as I mentioned above.

Comment 12

[Jeff Hammel]

Attachment: WIP

This allows me to get a little further, sorta.  Now i'm getting an error about not having privileges to install

Comment 13

[Jeff Hammel]

10:19:30     INFO - Creating virtualenv C:\Users\jhammel\mozharness\src\mozharne
ss\build\venv
10:19:30     INFO - Running command: ['virtualenv', '--no-site-packages', '--dis
tribute', 'C:\\Users\\jhammel\\mozharness\\src\\mozharness\\build\\venv'] in C:\
Users\jhammel\mozharness\src\mozharness\build
10:19:30     INFO - Copy/paste: virtualenv --no-site-packages --distribute C:\Us
ers\jhammel\mozharness\src\mozharness\build\venv
10:19:34     INFO -  The --no-site-packages flag is deprecated; it is now the de
fault behavior.
10:19:34     INFO -  Using real prefix 'C:\\Python27'
10:19:34     INFO -  New python executable in C:\Users\jhammel\mozharness\src\mo
zharness\build\venv\Scripts\python.exe
10:19:34     INFO -  Overwriting C:\Users\jhammel\mozharness\src\mozharness\buil
d\venv\Lib\distutils\__init__.py with new content
10:19:34     INFO -  Installing distribute......................................
................................................................................
.............................................................................don
e.
10:19:34     INFO -  Installing pip..................done.
10:19:34     INFO - Return code: 0
10:19:34     INFO - Installing pywin32 into virtualenv C:\Users\jhammel\mozharne
ss\src\mozharness\build\venv
10:19:34  WARNING - VIRTUAL_ENV C:\Users\jhammel\mozharness set; this may break
mozharness virtualenv calls!
10:19:34     INFO - Running command: ['C:\\Users\\jhammel\\mozharness\\src\\mozh
arness\\build\\venv\\Scripts\\easy_install', 'http://superb-sea2.dl.sourceforge.
net/project/pywin32/pywin32/Build216/pywin32-216.win32-py2.7.exe'] in C:\Users\j
hammel\mozharness\src\mozharness\build
10:19:34     INFO - Copy/paste: C:\Users\jhammel\mozharness\src\mozharness\build
\venv\Scripts\easy_install http://superb-sea2.dl.sourceforge.net/project/pywin32
/pywin32/Build216/pywin32-216.win32-py2.7.exe
10:19:34    ERROR - caught OS error 22: The requested operation requires elevati
on while running ['C:\\Users\\jhammel\\mozharness\\src\\mozharness\\build\\venv\
\Scripts\\easy_install', 'http://superb-sea2.dl.sourceforge.net/project/pywin32/
pywin32/Build216/pywin32-216.win32-py2.7.exe']
10:19:34    FATAL - Unable to install http://superb-sea2.dl.sourceforge.net/proj
ect/pywin32/pywin32/Build216/pywin32-216.win32-py2.7.exe!
10:19:34    FATAL - Exiting -1

Comment 14

[Jeff Hammel]

URL should be:

http://superb-sea2.dl.sourceforge.net/project/pywin32/pywin32/Build216/pywin32-216.win32-py2.7.exe

Comment 15

[Aki Sasaki (not active)]

I think I hit this in talos.
My guess is that the OS sees 'install' in the running exe and kills it.
My workaround involved query_exe and setting easy_install to the equivalent of 'python easy_install'.
So my windows config file includes this:

    "exes": {
        'python': PYTHON,
        'virtualenv': [PYTHON, 'c:/mozilla-build/buildbotve/virtualenv.py'],
        'hg': 'c:/mozilla-build/hg/hg',
        'easy_install': ['%s/scripts/python' % VENV_PATH,
                         '%s/scripts/easy_install-2.7-script.py' % VENV_PATH],
        'mozinstall': ['%s/scripts/python' % VENV_PATH,
                       '%s/scripts/mozinstall-script.py' % VENV_PATH],
    },

Since the running exe here is python, the OS lets it through.

Comment 16

[Jeff Hammel]

So I "fixed" this by turning off notifications and what not for installing software on windows (forgive me, I know very little about windows, how most people have their windows configured, or what not).  I'm not sure if this is an appropriate fix or not.

Comment 17

[Aki Sasaki (not active)]

That won't solve it for the production windows test slaves, but we have a workaround there, and it's good that you're getting further.

Comment 18

[Jeff Hammel]

(In reply to Aki Sasaki [:aki] from comment #15)
> I think I hit this in talos.
> My guess is that the OS sees 'install' in the running exe and kills it.
> My workaround involved query_exe and setting easy_install to the equivalent
> of 'python easy_install'.

You seem to be correct, sir!

http://answers.microsoft.com/en-us/windows/forum/windows_7-security/uac-message-do-you-want-to-allow-the-following/bea30ad8-9ef8-4897-aab4-841a65f7af71

Albeit, this is one of the stupidest "security" measures I have ever seen.

> So my windows config file includes this:
> 
>     "exes": {
>         'python': PYTHON,
>         'virtualenv': [PYTHON, 'c:/mozilla-build/buildbotve/virtualenv.py'],
>         'hg': 'c:/mozilla-build/hg/hg',
>         'easy_install': ['%s/scripts/python' % VENV_PATH,
>                          '%s/scripts/easy_install-2.7-script.py' %
> VENV_PATH],
>         'mozinstall': ['%s/scripts/python' % VENV_PATH,
>                        '%s/scripts/mozinstall-script.py' % VENV_PATH],
>     },
> 
> Since the running exe here is python, the OS lets it through.

Maybe its a good idea to patch this in mozharness.base.python ?

Comment 19

[Aki Sasaki (not active)]

mozharness.base.python has no idea where the paths for your python and the like will live, or what version they will be; that's why it belongs in config files that match your system.

Comment 20

[Jeff Hammel]

I'll need to know what paths to put here for the production config or a config I can base this on or other way I can get this information.

Comment 21

[Jeff Hammel]

Attachment: install pywin32 for windows

This manually specifies the pywin32 dependency such that it might succeed

Comment 22

[Aki Sasaki (not active)]

Comment on attachment 659375 [details] [diff] [review]
install pywin32 for windows

This is fine for your test config, but on production we'll have a separate config file for windows.

This is what I'm using in staging for windows talos:
http://hg.mozilla.org/users/asasaki_mozilla.com/mozharness/file/4e12a601baf4/configs/talos/windows_config.py

Comment 23

[Jeff Hammel]

pushed: http://hg.mozilla.org/build/mozharness/rev/4e2cd8ba1dcf

Comment 24

[Jeff Hammel]

(In reply to Aki Sasaki [:aki] from comment #19)
> mozharness.base.python has no idea where the paths for your python and the
> like will live, or what version they will be; that's why it belongs in
> config files that match your system.

I consistently have easy_install-script.py in ${VIRTUAL_ENV}\Scripts .

Comment 25

[Aki Sasaki (not active)]

(In reply to Jeff Hammel [:jhammel] from comment #24)
> (In reply to Aki Sasaki [:aki] from comment #19)
> > mozharness.base.python has no idea where the paths for your python and the
> > like will live, or what version they will be; that's why it belongs in
> > config files that match your system.
> 
> I consistently have easy_install-script.py in ${VIRTUAL_ENV}\Scripts .

Looks like that's true for my xp staging slave too.
Cool, that may be a good default.

Comment 26

[Jeff Hammel]

(In reply to Aki Sasaki [:aki] from comment #25)
> (In reply to Jeff Hammel [:jhammel] from comment #24)
> > (In reply to Aki Sasaki [:aki] from comment #19)
> > > mozharness.base.python has no idea where the paths for your python and the
> > > like will live, or what version they will be; that's why it belongs in
> > > config files that match your system.
> > 
> > I consistently have easy_install-script.py in ${VIRTUAL_ENV}\Scripts .
> 
> Looks like that's true for my xp staging slave too.
> Cool, that may be a good default.

Filed: bug 791840

Comment 27

[Aki Sasaki (not active)]

Are you unblocked here?

Comment 28

[Jeff Hammel]

I haven't been able to test.  We will see

Comment 29

[Jeff Hammel]

Yep, this now works