1667189 - Talos test refuses to run because of missing python packages

Status: RESOLVED DUPLICATE of bug 1669934

(Testing :: Talos, defect, P3)

Description

[Greg Tatum [:gregtatum]]

This is reproducing on two different macOS installs.

Steps to reproduce:

• ./mach clobber --full
• ./mach talos-test --activeTests damp --subtests console

Expected results:

• The packages to install and then the test to run

Actual Results:

• There are package install errors.

18:26:41     INFO - Calling [u'/Users/greg/dev/gecko/obj-ff-artifact/testing/talos-venv/bin/python', u'/Users/greg/dev/gecko/testing/talos/talos/run_tests.py', '--executablePath', u'/Users/greg/dev/gecko/obj-ff-artifact/dist/Nightly.app/Contents/MacOS/firefox', '--title', u'Magitek', u'--develop', u'--activeTests', u'damp', u'--subtests', u'console', '--log-tbpl-level=debug'] with output_timeout 3600
18:26:41    ERROR -  Traceback (most recent call last):
18:26:42     INFO -    File "/Users/greg/dev/gecko/testing/talos/talos/run_tests.py", line 17, in <module>
18:26:42     INFO -      from wptserve import server
18:26:42     INFO -  ImportError: No module named wptserve

I fixed this by manually pip installing via:

/Users/greg/dev/gecko/obj-ff-artifact/testing/talos-venv/bin/pip install simplejson
/Users/greg/dev/gecko/obj-ff-artifact/testing/talos-venv/bin/pip install wptserve

So it appears that simplejson and wptserve are missing in the requirements.

Comment 1

[Greg Tatum [:gregtatum]]

I'm getting a new error as well:

19:09:08     INFO -  ERROR: Could not find a version that satisfies the requirement pyrsistent>=0.14.0 (from jsonschema>=2.5.1->-r /Users/greg/dev/gecko/testing/talos/requirements.txt (line 1)) (from versions: none)
19:09:08     INFO -  ERROR: No matching distribution found for pyrsistent>=0.14.0 (from jsonschema>=2.5.1->-r /Users/greg/dev/gecko/testing/talos/requirements.txt (line 1))

I ran the command manually with a -v for a more verbose output:

/Users/greg/dev/gecko/obj-ff-artifact/testing/talos-venv/bin/pip install --timeout 120 -r /Users/greg/dev/gecko/testing/talos/requirements.txt --no-index --find-links https://pypi.pub.build.mozilla.org/pub/ -v

I got the output:

1 location(s) to search for versions of pyrsistent:
* https://pypi.pub.build.mozilla.org/pub/
Fetching project page and analyzing links: https://pypi.pub.build.mozilla.org/pub/
Getting page https://pypi.pub.build.mozilla.org/pub/
Looking up "https://pypi.pub.build.mozilla.org/pub/" in the cache
Request header has "max_age" as 0, cache bypassed
https://pypi.pub.build.mozilla.org:443 "GET /pub/ HTTP/1.1" 200 None
Updating cache with response from "https://pypi.pub.build.mozilla.org/pub/"
Caching b/c date exists and max-age > 0
Given no hashes to check 0 links for project 'pyrsistent': discarding no candidates
ERROR: Could not find a version that satisfies the requirement pyrsistent>=0.14.0 (from jsonschema>=2.5.1->-r /Users/greg/dev/gecko/testing/talos/requirements.txt (line 1)) (from versions: none)
ERROR: No matching distribution found for pyrsistent>=0.14.0 (from jsonschema>=2.5.1->-r /Users/greg/dev/gecko/testing/talos/requirements.txt (line 1))
Exception information:
Traceback (most recent call last):
  File "/Users/greg/Library/Python/3.7/lib/python/site-packages/pip/_internal/cli/base_command.py", line 188, in _main
    status = self.run(options, args)
  File "/Users/greg/Library/Python/3.7/lib/python/site-packages/pip/_internal/cli/req_command.py", line 185, in wrapper
    return func(self, options, args)
  File "/Users/greg/Library/Python/3.7/lib/python/site-packages/pip/_internal/commands/install.py", line 333, in run
    reqs, check_supported_wheels=not options.target_dir
  File "/Users/greg/Library/Python/3.7/lib/python/site-packages/pip/_internal/resolution/legacy/resolver.py", line 179, in resolve
    discovered_reqs.extend(self._resolve_one(requirement_set, req))
  File "/Users/greg/Library/Python/3.7/lib/python/site-packages/pip/_internal/resolution/legacy/resolver.py", line 362, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/Users/greg/Library/Python/3.7/lib/python/site-packages/pip/_internal/resolution/legacy/resolver.py", line 313, in _get_abstract_dist_for
    self._populate_link(req)
  File "/Users/greg/Library/Python/3.7/lib/python/site-packages/pip/_internal/resolution/legacy/resolver.py", line 279, in _populate_link
    req.link = self.finder.find_requirement(req, upgrade)
  File "/Users/greg/Library/Python/3.7/lib/python/site-packages/pip/_internal/index/package_finder.py", line 930, in find_requirement
    req)
DistributionNotFound: No matching distribution found for pyrsistent>=0.14.0 (from jsonschema>=2.5.1->-r /Users/greg/dev/gecko/testing/talos/requirements.txt (line 1))
Removed build tracker: '/private/var/folders/y7/0gx4dtxs223gpzqt_6rqjrmm0000gp/T/pip-req-tracker-ygMvDd'

I ran the command again, but this time without the mozilla URL.

/Users/greg/dev/gecko/obj-ff-artifact/testing/talos-venv/bin/pip install --timeout 120 -r /Users/greg/dev/gecko/testing/talos/requirements.txt

After that I got my same errors from above

What's really odd is that wptserve and simplejson are both listed in the testing/talos/requirements.txt

jsonschema>=2.5.1
mozlog>=6.0
mozcrash>=2.0
mozfile>=1.2
mozinfo>=0.8
mozprocess>=0.22
mozversion>=1.3
mozprofile>=0.25
mozrunner>=7.1.0
psutil>=3.1.1
simplejson>=2.1.1
requests>=2.9.1
wptserve>=2.0.0

Do you have any idea what's going on with this :sparky? I see this is marked as good first issue, but it's not clear to me what the fix is.

For the future me, if I get this again, the short fix to this problem is:

./obj-ff-artifact/testing/talos-venv/bin/pip install --timeout 120 -r ./testing/talos/requirements.txt
./obj-ff-artifact/testing/talos-venv/bin/pip install simplejson
./obj-ff-artifact/testing/talos-venv/bin/pip install wptserve

Comment 2

[Greg Tatum [:gregtatum]]

Attachment: command log example

I attached my full output running the steps described above, in case it's helpful.

Comment 3

[Greg Mierzwinski [:sparky]]

You'll have to add the missing package (pyrsistent) to here: https://pypi.pub.build.mozilla.org/pub/
Also note that your are mixing python2 with python3 in the logs you've provided. Talos is still python2.

Here are some recent similar bugs for how to do this, make sure the wheels get uploaded too: https://bugzilla.mozilla.org/show_bug.cgi?id=1666855

It's interesting that this is the first time someone hit this issue.

Comment 4

[Greg Tatum [:gregtatum]]

Also note that your are mixing python2 with python3 in the logs you've provided. Talos is still python2.

I believe this is the issue. I've had other mysterious errors in the build system, and I can now reproduce them using the steps outlined in: https://bugzilla.mozilla.org/show_bug.cgi?id=1669934#c15

I'm going to make this bug depend on Bug 1669934.