Closed Bug 1477870 Opened Last year Closed Last year

Talos fails to find module 'util' when run locally in moziila-build environment

Categories

(Testing :: Talos, defect)

Unspecified
Windows
defect
Not set

Tracking

(firefox64 fixed)

RESOLVED FIXED
mozilla64
Tracking Status
firefox64 --- fixed

People

(Reporter: aklotz, Assigned: Bebe)

Details

Attachments

(1 file, 1 obsolete file)

I am able to follow all of the setup steps until I reach https://wiki.mozilla.org/Buildbot/Talos/Running#Testing_your_Talos_Patch_Locally

From testing/talos, running:

> talos --help

results in:

Traceback (most recent call last):
  File "c:\Users\dblohm7\src\mc2\testing\talos\Scripts\talos-script.py", line 11, in <module>
    load_entry_point('talos', 'console_scripts', 'talos')()
  File "c:\Users\dblohm7\mozilla-build\python\lib\site-packages\pkg_resources\__init__.py", line 572, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "c:\Users\dblohm7\mozilla-build\python\lib\site-packages\pkg_resources\__init__.py", line 2752, in load_entry_point
    return ep.load()
  File "c:\Users\dblohm7\mozilla-build\python\lib\site-packages\pkg_resources\__init__.py", line 2405, in load
    return self.resolve()
  File "c:\Users\dblohm7\mozilla-build\python\lib\site-packages\pkg_resources\__init__.py", line 2411, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "c:\Users\dblohm7\src\mc2\testing\talos\talos\run_tests.py", line 18, in <module>
    import utils
ImportError: No module named utils
Blocks: 1468511, 1463483
Flags: needinfo?(rwood)
(For context, I am making changes to the xperf test and I'd like to test them locally)
Hi Aaron, I believe this is a legacy issue where we updated the talos requirements awhile back, but we don't force re-building the talos-venv each time you run locally.

Please try deleting the obj../testing/talos-venv folder and try running talos again. That will force it to update the virtualenv with the newest requirements. Thanks.
Flags: needinfo?(rwood)
No change.
Flags: needinfo?(rwood)
Hey Aaron sorry just noticed the wiki page you referenced is our old one, please see:

https://wiki.mozilla.org/Performance_sheriffing/Talos/Running

Try running the test using mach, i.e.:

./mach talos-test --suite xperf-e10s

:igoldan, I know you created the new Talos page, are you planning on removing the old one (or maybe make the one buildbot/talos automatically link to the new perf sheriff / talos one)?
Flags: needinfo?(rwood) → needinfo?(igoldan)
Note that those same steps are still present in the "Hacking on Talos Itself" section on that wiki page.
(In reply to Robert Wood [:rwood] from comment #4)
> :igoldan, I know you created the new Talos page, are you planning on
> removing the old one (or maybe make the one buildbot/talos automatically
> link to the new perf sheriff / talos one)?

I've just proposed the old page for deletion.
Flags: needinfo?(igoldan)
Given that this is looking like more of a documentation issue than anything, I'm removing this bug as a dependency.
No longer blocks: 1463483, 1468511
Attached patch Bug 1477870.diff (obsolete) — Splinter Review
I had the same issue with running talos from the command line

From what i found it's a import issue that can be easily fixed by updating the imports.

Also it looks like mozrunner was missing from the requirements file

With some help :igoldan from I ran the test on the try server and the changed don't affect integration with mach and treeherder
https://treeherder.mozilla.org/#/jobs?repo=try&revision=6165b00b1abb57297aa2233b63c6480df00b76e1

Please review and send feedback.
Assignee: nobody → bebe
Attachment #9013991 - Flags: review?(rwood)
Attachment #9013991 - Flags: feedback?(jmaher)
Comment on attachment 9013991 [details] [diff] [review]
Bug 1477870.diff

Review of attachment 9013991 [details] [diff] [review]:
-----------------------------------------------------------------

This patch changes the cycles for tresize to 1 instead of 20 (guessing that was meant to be temporary for testing?).
Attachment #9013991 - Flags: review?(rwood) → review-
Attached patch Bug 1477870.diffSplinter Review
Updated the Patch
Attachment #9013991 - Attachment is obsolete: true
Attachment #9013991 - Flags: feedback?(jmaher)
Attachment #9014031 - Flags: review?(rwood)
Attachment #9014031 - Flags: feedback?(jmaher)
:rwood Sorry forgot to remove that from the test code
Comment on attachment 9014031 [details] [diff] [review]
Bug 1477870.diff

Review of attachment 9014031 [details] [diff] [review]:
-----------------------------------------------------------------

this change sounds like a good idea and I have little concern.  as :rwood is on r? I will trust him to ensure it doesn't break anything new or nitpick any syntax or ordering.
Attachment #9014031 - Flags: feedback?(jmaher) → feedback+
(In reply to Florin Strugariu [:Bebe] from comment #10)
> Created attachment 9014031 [details] [diff] [review]
> Bug 1477870.diff
> 
> Updated the Patch

Hi Florin,

I still highly recommend using ./mach talos-test to run talos instead, as I'm not sure how support is for all the new features etc. when running directly via the 'talos' command. However thank you for fixing these import errors anyway.

I applied your patch locally on a clean build env, created the talos-venv and verified 'talos help' now works and doesn't error out like it did in the Description.

However when I tried to actually run a talos test directly like this:

(talos) Roberts-MacBook-Pro-1927:talos rwood$ talos --executablePath /Users/rwood/mozilla-unified/obj-x86_64-apple-darwin17.7.0/dist/NightlyDebug.app/Contents/MacOS/firefox --activeTests tart

It errors out further down:

"Abort: unable to find changeset or repository: .."

Does the above command line work for you? I'm on OSX, maybe it's my setup. When developing on talos I always ./mach talos-test, so it may have been broken for quite some time. I'm not sure that supporting running 'talos' locally directly (and not via ./mach) is something that we wish to support anymore...

I did verify that running locally via ./mach talos-test (after a clean build) still works with this patch applied. I wouldn't be opposed to landing this to fix the imports anyway, as it does appear in your try push to have no adverse effects. I added a gecko profiling talos test set (T-P-e10s) to your try push also.
Flags: needinfo?(bebe)
I know about .mach and I understand that the recommendation and normal procedure is to run everything through .mach

When setting up my env I tried to fallow the wiki and managed to tun tests without any issues.

but when fallowing [1] I found the issue with the imports.
[1] https://wiki.mozilla.org/Performance_sheriffing/Talos/Running#Dev_Environment

There the '--develop' is used to run the tests.This potions skips the full mozversion repository info [2]
[2] https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/run_tests.py#182

Also, to run the tests successfully,  make sure you have MOZ_DEVELOPER_REPO_DIR ="c:/mozilla-source/mozilla-central" set as an environment variable.

In the end the command i used is:

$ talos --executablePath c:/mozilla-source/mozilla-central/obj-i686-pc-mingw32\\dist\\bin\\firefox.exe --profile C:/mozilla-Source/mozilla-central/testing/profiles --activeTests tart   

My scope was to run the talos tests from a python command so I can integrate and debug in PyCharm :D
Flags: needinfo?(bebe)
we support 2 modes for our tools:
1) in CI run currently by mozharness (so a try push)
2) ./mach talos-test || ./mach raptor-test, this is the interface for 98% of the use cases

So all bugs we work on should be confirmed to solve these use cases, anything outside of those use cases (like custom debugging) we are fine having code in place for it, but we will not test it regularly.

Thanks for working on this- ensuring both our primary use cases work is a must for this bug.
Comment on attachment 9014031 [details] [diff] [review]
Bug 1477870.diff

(In reply to Florin Strugariu [:Bebe] from comment #14)

> Also, to run the tests successfully,  make sure you have
> MOZ_DEVELOPER_REPO_DIR ="c:/mozilla-source/mozilla-central" set as an
> environment variable.
> 

Even with MOZ_DEVELOPER_REPO_DIR set, running locally on my OSX directly via 'talos' doesn't work (same error I noted above).

However since it works for you, and also this patch is green in production and appears to have no adverse affect on running talos via ./mach talos-test, might as well go ahead and land and fix those imports. Thanks.
Attachment #9014031 - Flags: review?(rwood) → review+
anything else I can do here? Can we land and close the issue
Flags: needinfo?(rwood)
Flags: needinfo?(jmaher)
while there might be other issues, this is a safe change to make.
Flags: needinfo?(rwood)
Flags: needinfo?(jmaher)
Keywords: checkin-needed
Pushed by ebalazs@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/04692d253f94
Talos fails to find module 'util' when run locally in moziila-build environment. r=rwood
Keywords: checkin-needed
https://hg.mozilla.org/mozilla-central/rev/04692d253f94
Status: NEW → RESOLVED
Closed: Last year
Resolution: --- → FIXED
Target Milestone: --- → mozilla64
You need to log in before you can comment on or make changes to this bug.