Closed Bug 1185539 Opened 9 years ago Closed 8 years ago

'./mach talos-test tp5o' doesn't run

Categories

(Testing :: Talos, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED DUPLICATE of bug 1234500

People

(Reporter: BenWa, Unassigned)

References

Details

Attachments

(2 files, 4 obsolete files)

Attached file log
      No description provided.
I can reproduce the same thing, although I am doing './mach talos-test chromez', since tp5 will need the pageset which might not be readily available.

It looks like mach doesn't use mozharness proper, although mozharness just moved into the tree, so maybe a little update here:
https://dxr.mozilla.org/mozilla-central/source/testing/talos/mach_commands.py#81

trying to hack mozharness in the tree doesn't work, so that would be the next step.  The reason why the clone isn't happening is that we define a python webserver (instead of a hardcoded directory for apache) and mozharness doesn't clone talos in that instance:
https://dxr.mozilla.org/mozilla-central/source/testing/mozharness/mozharness/mozilla/testing/talos.py#530
Attached patch talos.patch (obsolete) — Splinter Review
With this patch I can go a bit further. The command "./mach talos-test chromez" now clone talos, but there are new issues:

23:18:19     INFO - Calling [u'/home/jp/dev/mozilla-central/testing/mozharness/venv/bin/python', u'/home/jp/dev/mozilla-central/testing/mozharness/build/talos_repo/talos/run_tests.py', '--noisy', '--debug', 'talos.yml'] with output_timeout 3600
23:18:19    ERROR -  Traceback (most recent call last):
23:18:19     INFO -    File "/home/jp/dev/mozilla-central/testing/mozharness/build/talos_repo/talos/run_tests.py", line 340, in <module>
23:18:19     INFO -      main()
23:18:19     INFO -    File "/home/jp/dev/mozilla-central/testing/mozharness/build/talos_repo/talos/run_tests.py", line 335, in main
23:18:19     INFO -      utils.startLogger(level)
23:18:19     INFO -    File "/home/jp/dev/mozilla-central/testing/mozharness/build/talos_repo/talos/utils.py", line 40, in startLogger
23:18:19     INFO -      log_levels = {'debug': mozlog.DEBUG, 'info': mozlog.INFO}
23:18:19     INFO -  AttributeError: 'module' object has no attribute 'DEBUG'
23:18:19    ERROR - Return code: 1

This happen because for whatever reason (I suspect dependencies are installed from testing/mozharness/mozharness/mozilla/testing/talos.py ?) mozlog 3.0 (latest) is installed - but Talos can't deal with that for now.

So basically Talos dependencies are not respected (because we explicitly ask for mozlog==2.6 in requirements.txt) and I wonder if we should try to fix that next ?
I saw that locally after manually running the clone myself. I was able to get Talos to work manually by running bin/activate. Not sure if that's helpful for this or not.
Attached patch talos.patch (obsolete) — Splinter Review
So, this goes a bit further. Now the rights dependencies are installed, but I still got an error:

08:59:15     INFO - Calling [u'/home/jp/dev/mozilla/testing/mozharness/venv/bin/python', u'/home/jp/dev/mozilla/testing/mozharness/build/talos_repo/talos/run_tests.py', '--noisy', '--debug', 'talos.yml'] with output_timeout 3600
08:59:16     INFO -  mozversion INFO | application_buildid: 20150717090553
08:59:16     INFO -  mozversion INFO | application_changeset: 8d262d1d0ae5
08:59:16     INFO -  mozversion INFO | application_display_name: Nightly
08:59:16     INFO -  mozversion INFO | application_name: Firefox
08:59:16     INFO -  mozversion INFO | application_version: 42.0a1
08:59:16     INFO -  mozversion INFO | platform_buildid: 20150717090553
08:59:16     INFO -  mozversion INFO | platform_changeset: 8d262d1d0ae5
08:59:16     INFO -  DEBUG : using testdate: 1437548356
08:59:16     INFO -  DEBUG : actual date: 1437548356
08:59:16     INFO -  JP-Precision-T1500:
08:59:16     INFO -  		Started Wed, 22 Jul 2015 08:59:16
08:59:16     INFO -  Running test tresize:
08:59:16     INFO -  		Started Wed, 22 Jul 2015 08:59:16
08:59:16     INFO -  DEBUG : operating with platform_type : linux_
08:59:16     INFO -  DEBUG : created profile
08:59:16     INFO -  DEBUG : command line: /home/jp/dev/mozilla/obj-x86_64-unknown-linux-gnu/dist/bin/firefox -profile /tmp/tmptx_XIr/profile getInfo.html
08:59:21     INFO -  DEBUG : Terminating:
08:59:21     INFO -  Failed tresize:
08:59:21     INFO -  		Stopped Wed, 22 Jul 2015 08:59:21
08:59:21    ERROR -  Traceback (most recent call last):
08:59:21     INFO -    File "/home/jp/dev/mozilla/testing/mozharness/build/talos_repo/talos/run_tests.py", line 267, in run_tests
08:59:21     INFO -      talos_results.add(mytest.runTest(browser_config, test))
08:59:21     INFO -    File "/home/jp/dev/mozilla/testing/mozharness/build/talos_repo/talos/ttest.py", line 297, in runTest
08:59:21     INFO -      return self._runTest(browser_config, test_config)
08:59:21     INFO -    File "/home/jp/dev/mozilla/testing/mozharness/build/talos_repo/talos/ttest.py", line 354, in _runTest
08:59:21     INFO -      self.initializeProfile(profile_dir, browser_config)
08:59:21     INFO -    File "/home/jp/dev/mozilla/testing/mozharness/build/talos_repo/talos/ttest.py", line 116, in initializeProfile
08:59:21     INFO -      self._ffsetup.InitializeNewProfile(profile_dir, browser_config)
08:59:21     INFO -    File "/home/jp/dev/mozilla/testing/mozharness/build/talos_repo/talos/ffsetup.py", line 151, in InitializeNewProfile
08:59:21     INFO -      raise TalosError("initalization has no output from browser")
08:59:21     INFO -  TalosError: initalization has no output from browser
08:59:21    ERROR - Return code: 2

Well, I suppose this is on the right way. :) But I'm a bit confused now by this error (one tab opens in my firefox instance, but this is /usr/bin/firefox, and not /home/jp/dev/mozilla/obj-x86_64-unknown-linux-gnu/dist/bin/firefox that is opened)
Attachment #8636801 - Attachment is obsolete: true
Attached image talos-error.png (obsolete) —
When I ran the command without any opened firefox instance, it is blocked at the line

09:09:15     INFO -  DEBUG : command line: /home/jp/dev/mozilla/obj-x86_64-unknown-linux-gnu/dist/bin/firefox -profile /tmp/tmpw3Skwl/profile getInfo.htm

and a firefox error window is opened (see the attached screenshot)
oh, the problem here is that getinfo.htm is invalid, we need http://localhost:{port}/getInfo.html

We need to pass in --develop from mach to get a python webserver properly setup.
Attached patch 1185539.patch (obsolete) — Splinter Review
Thanks, this is working well now locally!

We'll see on try how it will react to the changes made in testing/mozharness/mozharness/mozilla/testing/talos.py. I like that changes because it removes the hardcoded dependencies of talos, so I hope it will be all good. :)
Assignee: nobody → j.parkouss
Attachment #8637027 - Attachment is obsolete: true
Attachment #8637029 - Attachment is obsolete: true
Status: NEW → ASSIGNED
Attachment #8637154 - Flags: review?(jmaher)
Attached patch 1185539.patchSplinter Review
A minor change that uses ScriptMixin.read_from_file instead of using raw python open and read - this automatically log the action of reading "requirements.txt", and this could be a lot useful in case of issues. :)
Attachment #8637154 - Attachment is obsolete: true
Attachment #8637154 - Flags: review?(jmaher)
Attachment #8637170 - Flags: review?(jmaher)
Comment on attachment 8637170 [details] [diff] [review]
1185539.patch

Review of attachment 8637170 [details] [diff] [review]:
-----------------------------------------------------------------

this looks great.  I think these small changes will unlock a lot of things for folks.

try run up here:
https://hg.mozilla.org/try/pushloghtml?changeset=84b909d8af7f
Attachment #8637170 - Flags: review?(jmaher) → review+
This fails because mozinstall is required by mozharness but not by the requirements.txt in talos that is now used.

Pushed a new try with a fix (hardcode mozinstall dep in harness + add deps from talos requirements.txt):

https://treeherder.mozilla.org/#/jobs?repo=try&revision=45792341c2fe

So far, so good!
leave open, try is good so I am going to land it. This makes basically talos works with the mach command (ex "./mach talos-test chromez") but tp5o has another issue that we can discuss later.
Keywords: leave-open
the issue is we need to access tp5n.zip.  Either we put that in a public place, or come up with another solution.  The data is ~3 years old, that should be fair use.
I would be fine with a prompt saying something like:

tp5n pageset missing. Please install to '<DIR>' and re-run.

or

tp5n pageset missing. Please run:
wget <URL> | unzip tp5n.zip -e $DIR

where <URL> would be exactly as is. So we wouldn't provide the data publicly. The user could just find the URL, replace, run and re-run mach.

Or we could try to go with fair use, it would make things easier.
Working much better now!
See Also: → 1187127
parkous, would you mind if I uplifted your changes to aurora/beta/release?
It looks like there's a rm command so it's not possible to manually extract the tp5 pageset ATM. Is that right?
yeah, this is right.  Ideally we will get these pages in a repo that we can clone in an identical fashion from production and from 'mach'
(In reply to Armen Zambrano Gasparnian [:armenzg] from comment #17)
> parkous, would you mind if I uplifted your changes to aurora/beta/release?

Please do! Just check with jmaher, he's working on this at least on beta I think.
Also be sure to take th merged patch, not the one attached on the bug.
Assigning myself since there is no clear way to tackle this yet.
Assignee: j.parkouss → nobody
Status: ASSIGNED → NEW
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → DUPLICATE
Removing leave-open keyword from resolved bugs, per :sylvestre.
Keywords: leave-open
You need to log in before you can comment on or make changes to this bug.