Closed Bug 1656074 Opened 9 months ago Closed 9 months ago

./mach mozregression always says "mozregression was installed. please re-run your command."

Categories

(Testing :: mozregression, defect)

defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: tnikkel, Unassigned)

References

(Blocks 1 open bug)

Details

tip of m-c. I've tried rm -fr objdir and I still get the problem. Here is the install log (because there is one error in there)

Using base prefix '/usr/local/Cellar/python@3.8/3.8.3_1/Frameworks/Python.framework/Versions/3.8'
New python executable in /Users/tim/ffandroid/src/obj-x86_64-unknown-linux-android/_virtualenvs/init_py3/bin/python3.8
Also creating executable in /Users/tim/ffandroid/src/obj-x86_64-unknown-linux-android/_virtualenvs/init_py3/bin/python
Installing setuptools, pip, wheel...
done.
b'running build_ext\ncopying build/lib.macosx-10.15-x86_64-3.8/psutil/_psutil_osx.cpython-38-darwin.so -> psutil\ncopying build/lib.macosx-10.15-x86_64-3.8/psutil/_psutil_posix.cpython-38-darwin.so -> psutil\n'
Error processing command. Ignoring because optional. (optional:packages.txt:comm/build/virtualenv_packages.txt)
Collecting mozregression
Using cached https://files.pythonhosted.org/packages/ba/60/d0174f801a8b3093bb590a265338ac06bfd7797438d8f915b95e14bce771/mozregression-4.0.9-py3-none-any.whl
Collecting requests>=2.21.0
Using cached https://files.pythonhosted.org/packages/45/1e/0c169c6a5381e241ba7404532c16a21d86ab872c9bed8bdcd4c423954103/requests-2.24.0-py2.py3-none-any.whl
Collecting glean-sdk>=31.1.3
Using cached https://files.pythonhosted.org/packages/3a/a4/f6908edefc7963034289ccc5150624671a018fdee5d20959d5cc7a5526d2/glean_sdk-31.6.0-cp36-abi3-macosx_10_7_x86_64.whl
Collecting mozlog>=4.0
Using cached https://files.pythonhosted.org/packages/43/4f/f85945cdd29fd04d48ed9d9ca41f795d1a76f3c53a2a037d5e5dd4df1a0f/mozlog-6.1-py2.py3-none-any.whl
Collecting mozprocess>=1.0.0
Using cached https://files.pythonhosted.org/packages/53/35/4d8afbda598f933e862f094932603baf150f3c5ab1af94f41a3644773f10/mozprocess-1.2.0-py2.py3-none-any.whl
Collecting mozdevice<4,>=3.0.1
Using cached https://files.pythonhosted.org/packages/fa/02/399dd3ba3bbac0e1ac4fc56870e94cb17064cfae1e3348d9486e5cd86043/mozdevice-3.2.3-py2.py3-none-any.whl
Processing /Users/tim/Library/Caches/pip/wheels/f1/e4/16/4981ca97c2d65106b49861e0b35e2660695be7219a2d351ee0/configobj-5.0.6-cp38-none-any.whl
Requirement already satisfied: redo>=2.0.2 in ./third_party/python/redo (from mozregression) (2.0.3)
Processing /Users/tim/Library/Caches/pip/wheels/d7/d5/c4/11e57efb4dbee4eba14b68d3e7a57a6bf72a0c5de8327e8f30/mozinfo-1.2.2-py2.py3-none-any.whl
Collecting mozversion>=2.1.0
Using cached https://files.pythonhosted.org/packages/40/6f/2433c6556c4d5b67b124c28293c472b9dd4ae08426d21a9182a80fb06c62/mozversion-2.3.0-py2.py3-none-any.whl
Collecting colorama>=0.4.1
Using cached https://files.pythonhosted.org/packages/c9/dc/45cdef1b4d119eb96316b3117e6d5708a08029992b2fee2c143c7a0a5cc5/colorama-0.4.3-py2.py3-none-any.whl
Processing /Users/tim/Library/Caches/pip/wheels/6d/20/73/fa8e2af9862304621ed1c82c0ae83464c9fcd905cfed1fd5f7/mozfile-2.1.0-py2.py3-none-any.whl
Collecting taskcluster>=6.0.0
Using cached https://files.pythonhosted.org/packages/f6/a1/3551226cb8c7d564e5a35e384c9ca8f194665ee1db6ca92fe3458827f0d9/taskcluster-35.0.0-py3-none-any.whl
Collecting mozprofile>=2.2.0
Using cached https://files.pythonhosted.org/packages/6f/5f/ac810f46ea876046e93860f38710b6cafd1caac74fb34bd7d217aceaf58a/mozprofile-2.5.0-py2.py3-none-any.whl
Collecting beautifulsoup4>=4.7.1
Using cached https://files.pythonhosted.org/packages/66/25/ff030e2437265616a1e9b25ccc864e0371a0bc3adb7c5a404fd661c6f4f6/beautifulsoup4-4.9.1-py3-none-any.whl
Processing /Users/tim/Library/Caches/pip/wheels/b3/09/b9/b682485afe40c958580e07d8556e4eb5a88af90c1d75d9daf4/mozInstall-2.0.0-py2.py3-none-any.whl
Collecting mozrunner>=7.4.0
Using cached https://files.pythonhosted.org/packages/b9/48/fb5e6c47838776ed58c14f293f19c0f215f63a130219b367e929e2ea7b24/mozrunner-8.0.2-py2.py3-none-any.whl
Requirement already satisfied: certifi>=2017.4.17 in ./testing/web-platform/tests/tools/third_party/certifi (from requests>=2.21.0->mozregression) (2018.4.16)
Collecting chardet<4,>=3.0.2
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<3,>=2.5
Using cached https://files.pythonhosted.org/packages/a2/38/928ddce2273eaa564f6f50de919327bf3a00f091b5baba8dfa9460f3a8a8/idna-2.10-py2.py3-none-any.whl
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in ./third_party/python/urllib3/src (from requests>=2.21.0->mozregression) (1.25.9)
Collecting glean-parser==1.28.1
Using cached https://files.pythonhosted.org/packages/65/e5/c4bcfea46aa332dfad258f9ee84a0e13dfbcf7c9634bbdcab282f15b634f/glean_parser-1.28.1-py3-none-any.whl
Collecting cffi>=1
Using cached https://files.pythonhosted.org/packages/dc/00/3d48c7486bd9abfd7e0ab921014712042393dacb5fc0ce0651d0d9295298/cffi-1.14.1-cp38-cp38-macosx_10_9_x86_64.whl
Collecting blessings>=1.3
Using cached https://files.pythonhosted.org/packages/03/74/489f85a78247609c6b4f13733cbf3ba0d864b11aa565617b645d6fdf2a4a/blessings-1.7-py3-none-any.whl
Collecting mozterm
Using cached https://files.pythonhosted.org/packages/ab/eb/0c53684f5821d666372c6ea03d8c64558c8f74fda0fd5b24ae3dd2ab6a95/mozterm-1.0.0-py2.py3-none-any.whl
Collecting six>=1.10.0
Using cached https://files.pythonhosted.org/packages/ee/ff/48bde5c0f013094d729fe4b0316ba2a24774b3ff1c52d924a8a4cb04078a/six-1.15.0-py2.py3-none-any.whl
Collecting distro>=1.4.0
Using cached https://files.pythonhosted.org/packages/25/b7/b3c4270a11414cb22c6352ebc7a83aaa3712043be29daa05018fd5a5c956/distro-1.5.0-py2.py3-none-any.whl
Collecting taskcluster-urls>=12.1.0
Using cached https://files.pythonhosted.org/packages/24/7a/a2462a171675fab3de5f8e9d7e7d1624a7cd81e305c07c00fbe39b860626/taskcluster_urls-13.0.1-py3-none-any.whl
Collecting async-timeout>=2.0.0
Using cached https://files.pythonhosted.org/packages/e1/1e/5a4441be21b0726c4464f3f23c8b19628372f606755a9d2e46c187e65ec4/async_timeout-3.0.1-py3-none-any.whl
Collecting slugid>=2
Using cached https://files.pythonhosted.org/packages/6f/f7/0e7d06fa6914e2694d9766a827409fef1000f7548ba869b2476ff5817d86/slugid-2.0.0-py2.py3-none-any.whl
Collecting mohawk>=0.3.4
Using cached https://files.pythonhosted.org/packages/84/2a/56172c8d68b1439cc01a7c9eb5620006a77f91f313f8aa13877c6393d9cf/mohawk-1.1.0-py3-none-any.whl
Collecting aiohttp>=2.0.0
Using cached https://files.pythonhosted.org/packages/c2/f7/f0ad3dbace4762fef5d80aa4124b41bf218e4c4dd6d387a86cede707d9a4/aiohttp-3.6.2-py3-none-any.whl
Collecting soupsieve>1.2
Using cached https://files.pythonhosted.org/packages/6f/8f/457f4a5390eeae1cc3aeab89deb7724c965be841ffca6cfca9197482e470/soupsieve-2.0.1-py3-none-any.whl
Collecting appdirs>=1.4
Using cached https://files.pythonhosted.org/packages/3b/00/2344469e2084fb287c2e0b57b72910309874c3245463acd6cf5e3db69324/appdirs-1.4.4-py2.py3-none-any.whl
Collecting Jinja2>=2.10.1
Using cached https://files.pythonhosted.org/packages/30/9e/f663a2aa66a09d838042ae1a2c5659828bb9b41ea3a6efa20a20fd92b121/Jinja2-2.11.2-py2.py3-none-any.whl
Collecting diskcache>=4
Using cached https://files.pythonhosted.org/packages/ee/cc/d992e1d886d5ce15d2622c2e89b6de52b48312c6f05e34b7ee881b4ccb02/diskcache-4.1.0-py2.py3-none-any.whl
Requirement already satisfied: jsonschema>=3.0.2 in ./third_party/python/jsonschema (from glean-parser==1.28.1->glean-sdk>=31.1.3->mozregression) (3.2.0)
Collecting Click>=7
Using cached https://files.pythonhosted.org/packages/d2/3d/fa76db83bf75c4f8d338c2fd15c8d33fdd7ad23a9b5e57eb6c5de26b430e/click-7.1.2-py2.py3-none-any.whl
Collecting yamllint>=1.18.0
Using cached https://files.pythonhosted.org/packages/3d/cc/a68c1a8191a7d58188060bc0b1f8af38a5403772feb4ba8244059fc9eedb/yamllint-1.24.2-py2.py3-none-any.whl
Processing /Users/tim/Library/Caches/pip/wheels/a7/c1/ea/cf5bd31012e735dc1dfea3131a2d5eae7978b251083d6247bd/PyYAML-5.3.1-cp38-cp38-macosx_10_15_x86_64.whl
Collecting pycparser
Using cached https://files.pythonhosted.org/packages/ae/e7/d9c3a176ca4b02024debf82342dab36efadfc5776f9c8db077e8f6e71821/pycparser-2.20-py2.py3-none-any.whl
Collecting multidict<5.0,>=4.5
Using cached https://files.pythonhosted.org/packages/63/94/fff2e21a30e2b6b52b9d89df43c16e6b0e9563f50f7e2777aee77968332c/multidict-4.7.6-cp38-cp38-macosx_10_14_x86_64.whl
Collecting yarl<2.0,>=1.0
Using cached https://files.pythonhosted.org/packages/13/f6/ce23ae187cc848fb00ec4abe0ff12a35417dbaa254c5c201416fcd45a7c0/yarl-1.5.0-cp38-cp38-macosx_10_14_x86_64.whl
Collecting attrs>=17.3.0
Using cached https://files.pythonhosted.org/packages/a2/db/4313ab3be961f7a763066401fb77f7748373b6094076ae2bda2806988af6/attrs-19.3.0-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Using cached https://files.pythonhosted.org/packages/0c/12/37f68957526d1ec0883b521934b4e1b8ff3dd8e4fab858a5bf3e487bcee9/MarkupSafe-1.1.1-cp38-cp38-macosx_10_9_x86_64.whl
Processing /Users/tim/Library/Caches/pip/wheels/c2/85/ad/bc6d41e2c4b35c9fdfed48f0fcd411ffc4164e67755ddf9ebb/pyrsistent-0.16.0-cp38-cp38-macosx_10_15_x86_64.whl
Requirement already satisfied: setuptools in ./obj-x86_64-unknown-linux-android/_virtualenvs/init_py3/lib/python3.8/site-packages (from jsonschema>=3.0.2->glean-parser==1.28.1->glean-sdk>=31.1.3->mozregression) (41.6.0)
Collecting pathspec>=0.5.3
Using cached https://files.pythonhosted.org/packages/5d/d0/887c58853bd4b6ffc7aa9cdba4fc57d7b979b45888a6bd47e4568e1cf868/pathspec-0.8.0-py2.py3-none-any.whl
Collecting typing-extensions>=3.7.4
Using cached https://files.pythonhosted.org/packages/0c/0e/3f026d0645d699e7320b59952146d56ad7c374e9cd72cd16e7c74e657a0f/typing_extensions-3.7.4.2-py3-none-any.whl
ERROR: mozrunner 8.0.2 has requirement mozdevice<5,>=4.0.0, but you'll have mozdevice 3.2.3 which is incompatible.
Installing collected packages: chardet, idna, requests, appdirs, MarkupSafe, Jinja2, diskcache, Click, PyYAML, pathspec, yamllint, glean-parser, pycparser, cffi, glean-sdk, six, blessings, mozterm, mozlog, mozfile, distro, mozinfo, mozprocess, mozdevice, configobj, mozversion, colorama, taskcluster-urls, async-timeout, slugid, mohawk, multidict, typing-extensions, yarl, attrs, aiohttp, taskcluster, mozprofile, soupsieve, beautifulsoup4, mozinstall, mozrunner, mozregression, pyrsistent
Found existing installation: requests 2.9.1
Not uninstalling requests at /Users/tim/ffandroid/src/third_party/python/requests, outside environment /Users/tim/ffandroid/src/obj-x86_64-unknown-linux-android/_virtualenvs/init_py3/bin/..
Can't uninstall 'requests'. No files were found to uninstall.
Successfully installed Click-7.1.2 Jinja2-2.11.2 MarkupSafe-1.1.1 PyYAML-5.3.1 aiohttp-3.6.2 appdirs-1.4.4 async-timeout-3.0.1 attrs-19.3.0 beautifulsoup4-4.9.1 blessings-1.7 cffi-1.14.1 chardet-3.0.4 colorama-0.4.3 configobj-5.0.6 diskcache-4.1.0 distro-1.5.0 glean-parser-1.28.1 glean-sdk-31.6.0 idna-2.10 mohawk-1.1.0 mozdevice-3.2.3 mozfile-2.1.0 mozinfo-1.2.2 mozinstall-2.0.0 mozlog-6.1 mozprocess-1.2.0 mozprofile-2.5.0 mozregression-4.0.9 mozrunner-8.0.2 mozterm-1.0.0 mozversion-2.3.0 multidict-4.7.6 pathspec-0.8.0 pycparser-2.20 pyrsistent-0.16.0 requests-2.24.0 six-1.15.0 slugid-2.0.0 soupsieve-2.0.1 taskcluster-35.0.0 taskcluster-urls-13.0.1 typing-extensions-3.7.4.2 yamllint-1.24.2 yarl-1.5.0
WARNING: You are using pip version 19.3.1; however, version 20.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
mozregression was installed. please re-run your command. If you keep getting this message please manually run: 'pip install -U mozregression'.
tim@Timothys-MacBook-Pro src % ./mach mozregression --help
mozregression was installed. please re-run your command. If you keep getting this message please manually run: 'pip install -U mozregression'.

This seems like a mach issue rather than a mozregression one. :ahal do you have any ideas?

Flags: needinfo?(ahal)

ERROR: mozrunner 8.0.2 has requirement mozdevice<5,>=4.0.0, but you'll have mozdevice 3.2.3 which is incompatible.

Looks like the problem. Though the fact that message keeps appearing is confusing and seems like its own bug.

Could mozregression be installing its own version of mozrunner that is incompatible with the one in-tree? Timothy, could you have a copy of mozrunner on your PYTHONPATH somewhere outside of mozilla-central? What does $ ./mach python -c "import mozrunner; print(mozrunner.__file__)" say?

In general I'm not super happy with how mach mozregression does its bootstrapping. Maybe we could either vendor it, or possibly depend on bug 985141 and get it to install into an independent virtualenv (so it doesn't mix-in with the default one).

Flags: needinfo?(ahal) → needinfo?(tnikkel)

(In reply to Andrew Halberstadt [:ahal] from comment #2)

ERROR: mozrunner 8.0.2 has requirement mozdevice<5,>=4.0.0, but you'll have mozdevice 3.2.3 which is incompatible.

Looks like the problem. Though the fact that message keeps appearing is confusing and seems like its own bug.

Could mozregression be installing its own version of mozrunner that is incompatible with the one in-tree? Timothy, could you have a copy of mozrunner on your PYTHONPATH somewhere outside of mozilla-central? What does $ ./mach python -c "import mozrunner; print(mozrunner.__file__)" say?

Ah, this could be the source of the weirdness, in which case the recently released mozregression 4.0.11 may solve the problem-- it depends on the latest versions of mozdevice and mozrunner (same as the ones in-tree):

https://github.com/mozilla/mozregression/blob/master/setup.py

I'd recommend just trying again (maybe blowing away the objdir) -- it should work now

In general I'm not super happy with how mach mozregression does its bootstrapping. Maybe we could either vendor it, or possibly depend on bug 985141 and get it to install into an independent virtualenv (so it doesn't mix-in with the default one).

nod Yeah this isn't the first time this has come up. Incidentally the mach version of mozregression isn't really often used according to telemetry-- it's somewhat tempting to just take it out:

https://sql.telemetry.mozilla.org/queries/70610#177730

After pulling from m-c, mozregression now works, (I got the new version of mozregression).

(In reply to Andrew Halberstadt [:ahal] from comment #2)

ERROR: mozrunner 8.0.2 has requirement mozdevice<5,>=4.0.0, but you'll have mozdevice 3.2.3 which is incompatible.

Looks like the problem. Though the fact that message keeps appearing is confusing and seems like its own bug.

Could mozregression be installing its own version of mozrunner that is incompatible with the one in-tree? Timothy, could you have a copy of mozrunner on your PYTHONPATH somewhere outside of mozilla-central? What does $ ./mach python -c "import mozrunner; print(mozrunner.__file__)" say?

I only did this after updating my tree (and hence mozregression working again) and clearing the objdir and it found the mozrunner in the objdir of the tree.

I did a global find on my filesystem for mozrunner, it only found files inside other m-c trees and one venv I setup for mozregression before I knew I could run it from mach, but that venv wasn't active when I did any of these tests.

Feel free to close this bug now that it's working, I'm not sure how you want to deal with it.

Flags: needinfo?(tnikkel)

Thanks Timothy and sorry for the trouble. I think bug 1580625 actually covers the stuff that :ahal was talking about, so we can just resolve this one.

Status: NEW → RESOLVED
Closed: 9 months ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.