1278196 - Permanent message 'ImportError: No module named functools32' in logs of Windows jobs after bug 1272176 landed

Status: RESOLVED FIXED

(Release Engineering :: Applications: MozharnessCore, defect)

Description

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Regression range:
https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&autoclassify&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&filter-resultStatus=retry&filter-resultStatus=usercancel&filter-resultStatus=runnable&filter-resultStatus=success&fromchange=4353662915021fe5da16a9df0bf4ab4efee426c5&tochange=291d7bedba4f501b2cf3880b7e43b9a9ec0d9f1f&filter-searchStr=windows%20bc7&selectedJob=29370336

After bug 1272176 landed, 'ImportError: No module named functools32' gets shown in the logs of Windows jobs including the aggregated log view.

E.g. from http://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-inbound-win32-debug/1464906397/mozilla-inbound_xp_ix-debug_test-mochitest-browser-chrome-7-bm109-tests1-windows-build197.txt.gz

17:25:25     INFO - Running post-run listener: _resource_record_post_run
17:25:25  WARNING - Exception when reporting resource usage: Traceback (most recent call last):
17:25:25  WARNING -   File "C:\slave\test\scripts\mozharness\base\python.py", line 521, in _resource_record_post_run
17:25:25  WARNING -     self._log_resource_usage()
17:25:25  WARNING -   File "C:\slave\test\scripts\mozharness\base\python.py", line 539, in _log_resource_usage
17:25:25  WARNING -     import jsonschema
17:25:25  WARNING -   File "C:\slave\test\build\venv\Lib\site-packages\jsonschema\__init__.py", line 12, in <module>
17:25:25  WARNING -     from jsonschema.exceptions import (
17:25:25  WARNING -   File "C:\slave\test\build\venv\Lib\site-packages\jsonschema\exceptions.py", line 6, in <module>
17:25:25  WARNING -     from jsonschema import _utils
17:25:25  WARNING -   File "C:\slave\test\build\venv\Lib\site-packages\jsonschema\_utils.py", line 6, in <module>
17:25:25  WARNING -     from jsonschema.compat import str_types, MutableMapping, urlsplit
17:25:25  WARNING -   File "C:\slave\test\build\venv\Lib\site-packages\jsonschema\compat.py", line 39, in <module>
17:25:25  WARNING -     from functools32 import lru_cache
17:25:25  WARNING - ImportError: No module named functools32

Comment 1

[Gregory Szorc [:gps]]

https://pypi.python.org/packages/c5/60/6ac26ad05857c601308d8fb9e87fa36d0ebf889423f47c3502ef034365db/functools32-3.2.3-2.tar.gz should get uploaded to our internal PyPI mirror. It is a dependency for jsonschema.

jlund: can you please upload and close this out?

Comment 2

[Jordan Lund (:jlund)]

(In reply to Gregory Szorc [:gps] from comment #1)

> jlund: can you please upload and close this out?

seems like it is already there:
http://pypi.pub.build.mozilla.org/pub/functools32-3.2.3-2.tar.gz

sha: f6253dfbe0538ad2e387bd8fdfd9293c925d63553f5813c4e587745416501e6d  /Users/jlund/Downloads//functools32-3.2.3-2.tar.gz

Comment 3

[Gregory Szorc [:gps]]

Attachment: Bug 1278196 - Install functools32;

functools32 is a dependency of jsonschema. I guess dependencies don't get
installed automatically with whatever pip invocation we're using in mozharness.

Review commit: https://reviewboard.mozilla.org/r/58262/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/58262/

Comment 4

[Jordan Lund (:jlund)]

Comment on attachment 8760845 [details]
Bug 1278196 - Install functools32;

https://reviewboard.mozilla.org/r/58262/#review55314

r- on the basis I think this is a unittest (mochitest) issue not a talos one.

::: testing/mozharness/mozharness/mozilla/testing/talos.py:336
(Diff revision 1)
>              requirements=[os.path.join(self.talos_path,
>                                         'requirements.txt')]
>          )
> -        # install jsonschema for perfherder validation
> +        # install jsonschema for perfherder validation (functools32 is a
> +        # dependency)
> +        self.install_module(module="functools32")

is this just mochitest browser chrome?

if so, I don't think we need this installed for talos jobs.

maybe: https://dxr.mozilla.org/mozilla-central/source/testing/mozharness/scripts/desktop_unittest.py#284,298

where we can probably use line 298 mochitest condition block and install via `register_virtualenv_module` or an in-tree requirements file and append to `requirements_files`

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

gentle nudge, oh where are you functools32

Comment 6

[William Lachance (:wlach)]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=777ddbb84707

Comment 7

[William Lachance (:wlach)]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=7b3e91ebbcc1

Comment 8

[William Lachance (:wlach)]

Attachment: Bug 1278196 - Explicitly install functools32 w/ jsonschema

Review commit: https://reviewboard.mozilla.org/r/65268/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/65268/

Comment 9

[William Lachance (:wlach)]

The above push should fix the functools32 warnings we're seeing in automation (see try results: https://treeherder.mozilla.org/#/jobs?repo=try&revision=a7223e726213). It should only be necessary until we're using a newer version of pip.

Comment 10

[Gregory Szorc [:gps]]

Comment on attachment 8772473 [details]
Bug 1278196 - Explicitly install functools32 w/ jsonschema

https://reviewboard.mozilla.org/r/65268/#review63212

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/9299e5e388d4
Explicitly install functools32 w/ jsonschema r=gps

Comment 12

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/9299e5e388d4