1239352 - Install marionette_requirements.txt in desktop_unittest.py

Status: RESOLVED FIXED

(Release Engineering :: Applications: MozharnessCore, defect)

Description

[Andrew Halberstadt [:ahal]]

Mochitest will soon need to use marionette. We'll need to install marionette and dependencies for desktop_unittest.py.

Seeing as marionette is getting pretty ubiquitous, I don't think it's worth special casing mochitest. We'll probably need it sooner or later in the other desktop_unittest.py based harnesses too.

Comment 1

[Andrew Halberstadt [:ahal]]

Attachment: MozReview Request: Bug 1239352 - Install marionette_requirements.txt in desktop_unittest.py, r=jlund

Review commit: https://reviewboard.mozilla.org/r/30765/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/30765/

Comment 2

[Jordan Lund (:jlund)]

Comment on attachment 8707575 [details]
MozReview Request: Bug 1239352 - Install marionette_requirements.txt in desktop_unittest.py, r=jlund

https://reviewboard.mozilla.org/r/30765/#review27765

Comment 3

[Andrew Halberstadt [:ahal]]

Here's a nominal try run:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=d8e45c102ee3

Comment 4

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/7c17147a801f

Comment 5

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/7c17147a801f

Comment 6

[aleth [:aleth]]

This broke all c-c xpcshell/mozmill tests:

 4:26:16    ERROR -  Traceback (most recent call last):
 04:26:16     INFO -    File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/pip-1.5.5-py2.7.egg/pip/basecommand.py", line 122, in main
 04:26:16     INFO -      status = self.run(options, args)
 04:26:16     INFO -    File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/pip-1.5.5-py2.7.egg/pip/commands/install.py", line 262, in run
 04:26:16     INFO -      for req in parse_requirements(filename, finder=finder, options=options, session=session):
 04:26:16     INFO -    File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/pip-1.5.5-py2.7.egg/pip/req.py", line 1631, in parse_requirements
 04:26:16     INFO -      req = InstallRequirement.from_line(line, comes_from, prereleases=getattr(options, "pre", None))
 04:26:16     INFO -    File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/pip-1.5.5-py2.7.egg/pip/req.py", line 172, in from_line
 04:26:16     INFO -      return cls(req, comes_from, url=url, prereleases=prereleases)
 04:26:16     INFO -    File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/pip-1.5.5-py2.7.egg/pip/req.py", line 70, in __init__
 04:26:16     INFO -      req = pkg_resources.Requirement.parse(req)
 04:26:16     INFO -    File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/pip-1.5.5-py2.7.egg/pip/_vendor/pkg_resources.py", line 2667, in parse
 04:26:16     INFO -      reqs = list(parse_requirements(s))
 04:26:16     INFO -    File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/pip-1.5.5-py2.7.egg/pip/_vendor/pkg_resources.py", line 2605, in parse_requirements
 04:26:16     INFO -      line, p, specs = scan_list(VERSION,LINE_END,line,p,(1,2),"version spec")
 04:26:16     INFO -    File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/pip-1.5.5-py2.7.egg/pip/_vendor/pkg_resources.py", line 2573, in scan_list
 04:26:16     INFO -      raise ValueError("Expected "+item_name+" in",line,"at",line[p:])
 04:26:16     INFO -  ValueError: ('Expected version spec in', '../tools/wptserve', 'at', '/tools/wptserve')
 04:26:16     INFO -  Storing debug log for failure in /home/cltbld/.pip/pip.log
 04:26:16  WARNING - Return code: 2
 04:26:16    FATAL - Could not install python package: /builds/slave/test/build/venv/bin/pip install --no-deps --download-cache /builds/slave/test/build/venv/cache --timeout 120 -r /builds/slave/test/build/tests/config/marionette_requirements.txt --no-index --find-links http://pypi.pvt.build.mozilla.org.proxxy1.srv.releng.use1.mozilla.com/pub --find-links http://pypi.pub.build.mozilla.org.proxxy1.srv.releng.use1.mozilla.com/pub --find-links http://pypi.pvt.build.mozilla.org/pub --find-links http://pypi.pub.build.mozilla.org/pub failed after 5 tries!

It appears tools/wptserve isn't in-tree?

Comment 7

[Andrew Halberstadt [:ahal]]

Hrm, tools/wptserve should exist in the tests.zip since bug 1229467. Gps, do you know if c-c uses test_archive.py or not?
https://hg.mozilla.org/mozilla-central/rev/21a130c2ba15

Comment 8

[aleth [:aleth]]

(In reply to Andrew Halberstadt [:ahal] from comment #7)
> Hrm, tools/wptserve should exist in the tests.zip since bug 1229467. Gps, do
> you know if c-c uses test_archive.py or not?
> https://hg.mozilla.org/mozilla-central/rev/21a130c2ba15

c-c uses test_archive.py, but maybe something isn't being packaged in testsuite-targets.mk for TB which is now required?

Comment 9

[aleth [:aleth]]

Hmm, looks like it's packaged in common.tests.zip.

Comment 10

[Andrew Halberstadt [:ahal]]

Attachment: Followup fix for c-c test bustage

Oh, I think I see the problem. The desktop_unittest.py script only extracts certain directories from the tests.zip by default. Looks like there's a thunderbird mozharness script I neglected to modify. Sorry.

Does this fix the issue?

Comment 11

[aleth [:aleth]]

Comment on attachment 8709158 [details] [diff] [review]
Followup fix for c-c test bustage

Review of attachment 8709158 [details] [diff] [review]:
-----------------------------------------------------------------

Aha, thanks! Looks like this should fix it indeed.

I don't know how to test this locally, and I'm not a testing/ peer, so f+ instead of r+.

Comment 12

[aleth [:aleth]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/76b4de2ccd6ca11a8121d17558aee62caf943c9a
Bug 1239352 - Follow-up fix for c-c test bustage. r=jlund

Comment 13

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/76b4de2ccd6c

Comment 14

[aleth [:aleth]]

(In reply to Andrew Halberstadt [:ahal] from comment #10)
> Does this fix the issue?

Problem solved, thanks.