1477870 - Talos fails to find module 'util' when run locally in moziila-build environment

Status: RESOLVED FIXED

(Testing :: Talos, defect)

Description

[(No longer employed by Mozilla) Aaron Klotz]

I am able to follow all of the setup steps until I reach https://wiki.mozilla.org/Buildbot/Talos/Running#Testing_your_Talos_Patch_Locally

From testing/talos, running:

> talos --help

results in:

Traceback (most recent call last):
  File "c:\Users\dblohm7\src\mc2\testing\talos\Scripts\talos-script.py", line 11, in <module>
    load_entry_point('talos', 'console_scripts', 'talos')()
  File "c:\Users\dblohm7\mozilla-build\python\lib\site-packages\pkg_resources\__init__.py", line 572, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "c:\Users\dblohm7\mozilla-build\python\lib\site-packages\pkg_resources\__init__.py", line 2752, in load_entry_point
    return ep.load()
  File "c:\Users\dblohm7\mozilla-build\python\lib\site-packages\pkg_resources\__init__.py", line 2405, in load
    return self.resolve()
  File "c:\Users\dblohm7\mozilla-build\python\lib\site-packages\pkg_resources\__init__.py", line 2411, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "c:\Users\dblohm7\src\mc2\testing\talos\talos\run_tests.py", line 18, in <module>
    import utils
ImportError: No module named utils

Comment 1

[(No longer employed by Mozilla) Aaron Klotz]

(For context, I am making changes to the xperf test and I'd like to test them locally)

Comment 2

[Robert Wood [:rwood]]

Hi Aaron, I believe this is a legacy issue where we updated the talos requirements awhile back, but we don't force re-building the talos-venv each time you run locally.

Please try deleting the obj../testing/talos-venv folder and try running talos again. That will force it to update the virtualenv with the newest requirements. Thanks.

Comment 3

[(No longer employed by Mozilla) Aaron Klotz]

No change.

Comment 4

[Robert Wood [:rwood]]

Hey Aaron sorry just noticed the wiki page you referenced is our old one, please see:

https://wiki.mozilla.org/Performance_sheriffing/Talos/Running

Try running the test using mach, i.e.:

./mach talos-test --suite xperf-e10s

:igoldan, I know you created the new Talos page, are you planning on removing the old one (or maybe make the one buildbot/talos automatically link to the new perf sheriff / talos one)?

Comment 5

[(No longer employed by Mozilla) Aaron Klotz]

Note that those same steps are still present in the "Hacking on Talos Itself" section on that wiki page.

Comment 6

[Ionuț Goldan [:igoldan]]

(In reply to Robert Wood [:rwood] from comment #4)
> :igoldan, I know you created the new Talos page, are you planning on
> removing the old one (or maybe make the one buildbot/talos automatically
> link to the new perf sheriff / talos one)?

I've just proposed the old page for deletion.

Comment 7

[(No longer employed by Mozilla) Aaron Klotz]

Given that this is looking like more of a documentation issue than anything, I'm removing this bug as a dependency.

Comment 8

[Florin Strugariu [:Bebe]]

Attachment: Bug 1477870.diff

I had the same issue with running talos from the command line

From what i found it's a import issue that can be easily fixed by updating the imports.

Also it looks like mozrunner was missing from the requirements file

With some help :igoldan from I ran the test on the try server and the changed don't affect integration with mach and treeherder
https://treeherder.mozilla.org/#/jobs?repo=try&revision=6165b00b1abb57297aa2233b63c6480df00b76e1

Please review and send feedback.

Comment 9

[Robert Wood [:rwood]]

Comment on attachment 9013991 [details] [diff] [review]
Bug 1477870.diff

Review of attachment 9013991 [details] [diff] [review]:
-----------------------------------------------------------------

This patch changes the cycles for tresize to 1 instead of 20 (guessing that was meant to be temporary for testing?).

Comment 10

[Florin Strugariu [:Bebe]]

Attachment: Bug 1477870.diff

Updated the Patch

Comment 11

[Florin Strugariu [:Bebe]]

:rwood Sorry forgot to remove that from the test code

Comment 12

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 9014031 [details] [diff] [review]
Bug 1477870.diff

Review of attachment 9014031 [details] [diff] [review]:
-----------------------------------------------------------------

this change sounds like a good idea and I have little concern.  as :rwood is on r? I will trust him to ensure it doesn't break anything new or nitpick any syntax or ordering.

Comment 13

[Robert Wood [:rwood]]

(In reply to Florin Strugariu [:Bebe] from comment #10)
> Created attachment 9014031 [details] [diff] [review]
> Bug 1477870.diff
> 
> Updated the Patch

Hi Florin,

I still highly recommend using ./mach talos-test to run talos instead, as I'm not sure how support is for all the new features etc. when running directly via the 'talos' command. However thank you for fixing these import errors anyway.

I applied your patch locally on a clean build env, created the talos-venv and verified 'talos help' now works and doesn't error out like it did in the Description.

However when I tried to actually run a talos test directly like this:

(talos) Roberts-MacBook-Pro-1927:talos rwood$ talos --executablePath /Users/rwood/mozilla-unified/obj-x86_64-apple-darwin17.7.0/dist/NightlyDebug.app/Contents/MacOS/firefox --activeTests tart

It errors out further down:

"Abort: unable to find changeset or repository: .."

Does the above command line work for you? I'm on OSX, maybe it's my setup. When developing on talos I always ./mach talos-test, so it may have been broken for quite some time. I'm not sure that supporting running 'talos' locally directly (and not via ./mach) is something that we wish to support anymore...

I did verify that running locally via ./mach talos-test (after a clean build) still works with this patch applied. I wouldn't be opposed to landing this to fix the imports anyway, as it does appear in your try push to have no adverse effects. I added a gecko profiling talos test set (T-P-e10s) to your try push also.

Comment 14

[Florin Strugariu [:Bebe]]

I know about .mach and I understand that the recommendation and normal procedure is to run everything through .mach

When setting up my env I tried to fallow the wiki and managed to tun tests without any issues.

but when fallowing [1] I found the issue with the imports.
[1] https://wiki.mozilla.org/Performance_sheriffing/Talos/Running#Dev_Environment

There the '--develop' is used to run the tests.This potions skips the full mozversion repository info [2]
[2] https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/run_tests.py#182

Also, to run the tests successfully,  make sure you have MOZ_DEVELOPER_REPO_DIR ="c:/mozilla-source/mozilla-central" set as an environment variable.

In the end the command i used is:

$ talos --executablePath c:/mozilla-source/mozilla-central/obj-i686-pc-mingw32\\dist\\bin\\firefox.exe --profile C:/mozilla-Source/mozilla-central/testing/profiles --activeTests tart   

My scope was to run the talos tests from a python command so I can integrate and debug in PyCharm :D

Comment 15

[Joel Maher ( :jmaher ) (UTC -8)]

we support 2 modes for our tools:
1) in CI run currently by mozharness (so a try push)
2) ./mach talos-test || ./mach raptor-test, this is the interface for 98% of the use cases

So all bugs we work on should be confirmed to solve these use cases, anything outside of those use cases (like custom debugging) we are fine having code in place for it, but we will not test it regularly.

Thanks for working on this- ensuring both our primary use cases work is a must for this bug.

Comment 16

[Robert Wood [:rwood]]

Comment on attachment 9014031 [details] [diff] [review]
Bug 1477870.diff

(In reply to Florin Strugariu [:Bebe] from comment #14)

> Also, to run the tests successfully,  make sure you have
> MOZ_DEVELOPER_REPO_DIR ="c:/mozilla-source/mozilla-central" set as an
> environment variable.
> 

Even with MOZ_DEVELOPER_REPO_DIR set, running locally on my OSX directly via 'talos' doesn't work (same error I noted above).

However since it works for you, and also this patch is green in production and appears to have no adverse affect on running talos via ./mach talos-test, might as well go ahead and land and fix those imports. Thanks.

Comment 17

[Florin Strugariu [:Bebe]]

anything else I can do here? Can we land and close the issue

Comment 18

[Joel Maher ( :jmaher ) (UTC -8)]

while there might be other issues, this is a safe change to make.

Comment 19

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/04692d253f94
Talos fails to find module 'util' when run locally in moziila-build environment. r=rwood

Comment 20

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/04692d253f94