1092089 - Default virtualenv_modules configuration is failing

Status: RESOLVED INVALID

(Release Engineering :: Applications: MozharnessCore, defect)

Description

[Sylvestre Ledru [:Sylvestre]]

As suggested in bug 1045531, I am using a virtualenv to install a dependency.

        # Default configuration
        default_config = {
            'debug_build': False,
            # this will pip install it automajically when we call the create-virtualenv action
            'virtualenv_modules': ['google-api-python-client'],
            "find_links": [   # so mozharness knows where to look for the package
                "http://pypi.pvt.build.mozilla.org/pub",
                "http://pypi.pub.build.mozilla.org/pub",
            ],
            # the path inside the work_dir ('build') of where we will install the env.
            # pretty sure it's the default and not needed.
            'virtual_env': 'venv',
        }


However, this configuration is failing with:
10:08:46     INFO - Creating virtualenv None
10:08:46     INFO - Running command: ['virtualenv', '--no-site-packages', '--distribute', None] in /home/sylvestre/dev/mozilla/push-apk/mozharness/scripts/build
10:08:46    FATAL - Uncaught exception: Traceback (most recent call last):
10:08:46    FATAL -   File "/home/sylvestre/dev/mozilla/push-apk/mozharness/mozharness/base/script.py", line 1268, in run
10:08:46    FATAL -     self.run_action(action)
10:08:46    FATAL -   File "/home/sylvestre/dev/mozilla/push-apk/mozharness/mozharness/base/script.py", line 1210, in run_action
10:08:46    FATAL -     self._possibly_run_method(method_name, error_if_missing=True)
10:08:46    FATAL -   File "/home/sylvestre/dev/mozilla/push-apk/mozharness/mozharness/base/script.py", line 1151, in _possibly_run_method
10:08:46    FATAL -     return getattr(self, method_name)()
10:08:46    FATAL -   File "/home/sylvestre/dev/mozilla/push-apk/mozharness/mozharness/base/python.py", line 365, in create_virtualenv
10:08:46    FATAL -     halt_on_failure=True)
10:08:46    FATAL -   File "/home/sylvestre/dev/mozilla/push-apk/mozharness/mozharness/base/script.py", line 683, in run_command
10:08:46    FATAL -     self.info("Copy/paste: %s" % subprocess.list2cmdline(command))
10:08:46    FATAL -   File "/usr/lib/python2.7/subprocess.py", line 616, in list2cmdline
10:08:46    FATAL -     needquote = (" " in arg) or ("\t" in arg) or not arg
10:08:46    FATAL - TypeError: argument of type 'NoneType' is not iterable

I think we should add a check when the value is None (and probably gives an hint)

Comment 1

[Sylvestre Ledru [:Sylvestre]]

Jordan, can you help here? it is probably a PBKAC ;) Thanks

Comment 2

[Jordan Lund (:jlund)]

> 10:08:46     INFO - Creating virtualenv None

this seems wrong. grepping for 'Creating virtualenv ' in mozharness yields: http://mxr.mozilla.org/build/source/mozharness/mozharness/base/python.py#329

so it seems http://mxr.mozilla.org/build/source/mozharness/mozharness/base/python.py#93 is never getting reassigned from None.

>             # pretty sure it's the default and not needed.
>             'virtual_env': 'venv',
>         }
> 

this was wrong. query_virtaulenv_path expects c['virtualenv_path']. so s/virtual_env/virtualenv_path

it would be nice if mozharness had an api and config validation[1] so if you inherit a mixin (e.g. VirtualenvMixin), you were told what config items were needed and or missing at runtime

[1] https://bugzil.la/699343

Comment 3

[Sylvestre Ledru [:Sylvestre]]

OK. Thanks.
FYI, I used your proposition in https://bugzilla.mozilla.org/show_bug.cgi?id=1045531#c15

Seems to work fine now :)

Comment 4

[Jordan Lund (:jlund)]

(In reply to Sylvestre Ledru [:sylvestre] from comment #3)
> OK. Thanks.
> FYI, I used your proposition in
> https://bugzilla.mozilla.org/show_bug.cgi?id=1045531#c15

right, sorry I meant *I* was wrong.

> Seems to work fine now :)

\o/ great news.