1439694 - ImportError jsonschema when running talos locally

Status: RESOLVED FIXED

(Testing :: Talos, defect)

Description

[Alex Gaynor [:Alex_Gaynor]]

Steps to reproduce:

1) rm -rf obj-*/testing/talos-venv/ (or $OBJDIR) is for you
2) env MOZCONFIG=(pwd)/mozconfig-opt MOZ_UPLOAD_DIR=(pwd)/sync-profile ./mach talos-test --activeTests sessionrestore --geckoProfile --cycles=1 --tpcycles=1 - this will work cleanly, the virtualenv is created, jsonschema is installed, and works fine
3) env MOZCONFIG=(pwd)/mozconfig-opt MOZ_UPLOAD_DIR=(pwd)/sync-profile ./mach talos-test --activeTests sessionrestore --geckoProfile --cycles=1 --tpcycles=1 - (the same command) this fails like so:

[...]
13:55:43     INFO -  Thanks for running Talos locally. Results are in ['/Users/agaynor/projects/mozilla-central/testing/mozharness/build/local.json']
13:55:43     INFO - Return code: 0
13:55:43     INFO - Running post-action listener: _package_coverage_data
13:55:43     INFO - Running post-action listener: _resource_record_post_action
13:55:43     INFO - [mozharness: 2018-02-20 18:55:43.729577Z] Finished run-tests step (failed)
13:55:43    FATAL - Uncaught exception: Traceback (most recent call last):
13:55:43    FATAL -   File "/Users/agaynor/projects/mozilla-central/testing/mozharness/mozharness/base/script.py", line 2059, in run
13:55:43    FATAL -     self.run_action(action)
13:55:43    FATAL -   File "/Users/agaynor/projects/mozilla-central/testing/mozharness/mozharness/base/script.py", line 1998, in run_action
13:55:43    FATAL -     self._possibly_run_method(method_name, error_if_missing=True)
13:55:43    FATAL -   File "/Users/agaynor/projects/mozilla-central/testing/mozharness/mozharness/base/script.py", line 1938, in _possibly_run_method
13:55:43    FATAL -     return getattr(self, method_name)()
13:55:43    FATAL -   File "/Users/agaynor/projects/mozilla-central/testing/mozharness/mozharness/mozilla/testing/talos.py", line 730, in run_tests
13:55:43    FATAL -     self._validate_treeherder_data(parser)
13:55:43    FATAL -   File "/Users/agaynor/projects/mozilla-central/testing/mozharness/mozharness/mozilla/testing/talos.py", line 599, in _validate_treeherder_data
13:55:43    FATAL -     import jsonschema
13:55:43    FATAL -   File "/Users/agaynor/projects/mozilla-central/build/mach_bootstrap.py", line 363, in __call__
13:55:43    FATAL -     module = self._original_import(name, globals, locals, fromlist, level)
13:55:43    FATAL - ImportError: No module named jsonschema
13:55:43    FATAL - Running post_fatal callback...
13:55:43    FATAL - Exiting -1
13:55:43     INFO - Running post-run listener: _resource_record_post_run
13:55:43     INFO - Running post-run listener: _upload_blobber_files
13:55:43  WARNING - Blob upload gear skipped. Missing cmdline options.
13:55:43     INFO - Running post-run listener: copy_logs_to_upload_dir
13:55:43     INFO - Copying logs to upload dir...

Comment 1

[Robert Wood [:rwood]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=90e0b9ca460aa0ea98f4a603c82e46f26a9f53f5

Comment 2

[Robert Wood [:rwood]]

Attachment: Bug 1439694 - Fix jsonschema import error when running talos locally;

Review commit: https://reviewboard.mozilla.org/r/223342/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/223342/

Comment 3

[Robert Wood [:rwood]]

Comment on attachment 8954186 [details]
Bug 1439694 - Fix jsonschema import error when running talos locally;

Cancelling the review, this solution doesn't work for tp6 since tp6 uses a different py3 venv

Comment 4

[Robert Wood [:rwood]]

Comment on attachment 8954186 [details]
Bug 1439694 - Fix jsonschema import error when running talos locally;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/223342/diff/1-2/

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8954186 [details]
Bug 1439694 - Fix jsonschema import error when running talos locally;

https://reviewboard.mozilla.org/r/223342/#review229992

Comment 6

[Robert Wood [:rwood]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=c44fba1b4bb2edd14a22d4f0b4bf39029572cf43

Comment 7

[Robert Wood [:rwood]]

Comment on attachment 8954186 [details]
Bug 1439694 - Fix jsonschema import error when running talos locally;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/223342/diff/2-3/

Comment 8

[Robert Wood [:rwood]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=2fe6dbf0fe3ed5fa0467c5c352e773424bb76e58

Comment 9

[Robert Wood [:rwood]]

Note: Don't land this until it's tested on the new moonshot hardware (which will be re-landing in Bug 1431161)

Comment 10

[Robert Wood [:rwood]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=eb265a50a39da99792ead28e655683a267910bf2

Comment 11

[Robert Wood [:rwood]]

Comment 10 is a try push of this jsonschema patch but running on the new moonshot win10 h/w

Comment 12

[Robert Wood [:rwood]]

Comment on attachment 8954186 [details]
Bug 1439694 - Fix jsonschema import error when running talos locally;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/223342/diff/3-4/

Comment 13

[Robert Wood [:rwood]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=3dcbea375313c2115f41dbcfb7d921e65e1ff7f2

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f249c97b9d35
Fix jsonschema import error when running talos locally; r=jmaher

Comment 15

[Andrei Ciure[:aciure]]

https://hg.mozilla.org/mozilla-central/rev/f249c97b9d35

Comment 16

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/fb0f35c060ef