Closed
Bug 650887
Opened 14 years ago
Closed 13 years ago
desktop talos runner in mozharness
Categories
(Release Engineering :: Applications: MozharnessCore, defect, P1)
Release Engineering
Applications: MozharnessCore
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: mozilla, Assigned: mozilla)
References
Details
(Whiteboard: [mozharness][talos][mozharness+talos])
Attachments
(1 file, 3 obsolete files)
8.97 KB,
patch
|
mozilla
:
review+
|
Details | Diff | Splinter Review |
This should get complexity out of buildbotcustom and allow the a-team,
developers, and releng to all use the same scripts.
By "port" I mean wrap run_tests.py in mozharness, for now.
Assignee | ||
Updated•14 years ago
|
Summary: port desktop talos to mozharness → desktop talos runner in mozharness
Assignee | ||
Updated•14 years ago
|
Assignee: nobody → aki
Comment 1•13 years ago
|
||
I'm not sure if this makes sense to do until bug 688604 is landed so that we can use the mozhttpd server. If not, there is no good way of using a webserver that doesn't require root privileges since, at least on linux, you're not allowed to open port 80
Assignee | ||
Comment 2•13 years ago
|
||
a) i have a talos zipfile that lets me use --develop without being blocked
b) we can install mozhttpd directly
c) we can start a webserver and test
I don't think this blocks. At most it blocks the ability to start a webserver automatically without a patched talos; there's plenty of other work that can happen here by using one of the workarounds above.
Comment 3•13 years ago
|
||
So it seems like the first step is to replace the existing buildbotcustom logic with a mozharness script. Is this an accurate assessment?
Assignee | ||
Comment 4•13 years ago
|
||
Correct. They'll probably run in parallel until we can prove desktop talosrunner is stable, but the goal is to replace it. A secondary goal is to allow a user to run the same script locally.
Comment 5•13 years ago
|
||
So I had incorrectly perceived the goal here to be getting something working for developers vs something to replace the existing buildbotcustom setup. Here's what I have now, which should mostly work except for the webserver part, just for posterity.
Comment 6•13 years ago
|
||
So this is another WIP slated towards buildbot as a consumer. It errs our at
10:27:23 INFO - utils.talosError: 'Found processes still running: firefox. Please close them before running talos.'
I don't have any firefox processes running so something is awry
Comment 7•13 years ago
|
||
(In reply to Jeff Hammel [:jhammel] from comment #6)
> Created attachment 572882 [details]
> WIP 2
>
> So this is another WIP slated towards buildbot as a consumer. It errs our at
>
> 10:27:23 INFO - utils.talosError: 'Found processes still running:
> firefox. Please close them before running talos.'
>
> I don't have any firefox processes running so something is awry
Ah, this is bug 700722. This should probably be fixed before this script is usable.
Depends on: 700722
Assignee | ||
Comment 8•13 years ago
|
||
Thanks, Jeff!
Looks like installing talos installs pyyaml by default, but I'm a bit worried about production, which can't get to pypi. Listing pyyaml explicitly might help us there.
For user-friendliness making this script and the other mozharness talos- and unittest- scripts "feel" the same in terms of actions and config naming would be good, but I'm fine leaving that for later.
It does sound like --develop and the process issue will make running this not so user-friendly, so getting those fixed properly would be good. If we want/need to roll something out before those are resolved, I think we can work around them.
Comment 9•13 years ago
|
||
Once https://github.com/escapewindow/mozharness/blob/talosrunner/scripts/device_talosrunner.py lands we should unify the options a bit, maybe have a common base class
Comment 10•13 years ago
|
||
Attachment #572583 -
Attachment is obsolete: true
Attachment #572882 -
Attachment is obsolete: true
Comment 11•13 years ago
|
||
This is currently blocked by bug 701573. The virtualenv-ed talos does not shell out to the correct python process, which is the fault of Talos. I will fix that first.
Depends on: 701573
Assignee | ||
Comment 12•13 years ago
|
||
(In reply to Jeff Hammel [:jhammel] from comment #11)
> This is currently blocked by bug 701573. The virtualenv-ed talos does not
> shell out to the correct python process, which is the fault of Talos. I
> will fix that first.
I'm working around this by setting PATH in the run_tests.py call (where I need to change it to use os.pathsep).
I'd love a real fix.
https://github.com/escapewindow/mozharness/blob/talosrunner/scripts/device_talosrunner.py#L340
Comment 13•13 years ago
|
||
Attachment #573691 -
Attachment is obsolete: true
Attachment #573963 -
Flags: review?(aki)
Assignee | ||
Comment 14•13 years ago
|
||
Comment on attachment 573963 [details] [diff] [review]
basic mozharness for talos
>+ def _fetch(self, url, dest):
>+ """fetch a static resource from the web"""
>+ # TODO: could be upstreamed
>+ f = file(dest, 'w')
>+ resource = urllib2.urlopen(url)
>+ f.write(resource.read())
>+ resource.close()
>+ f.close()
self.download_file() ?
http://hg.mozilla.org/build/mozharness/file/cbcc9a5d5549/mozharness/base/script.py#l147
>+ def _set_talos_dir(self):
>+ """XXX this is required as talos must be run out of its directory"""
>+ python = self.query_python_path()
>+ self.talos_dir = self.get_output_from_command([python, '-c', 'import os, talos; print os.path.dirname(os.path.abspath(talos.__file__))'])
>+ if not self.talos_dir:
>+ self.fatal("Talos directory could not be found")
>+ self.info('Talos directory: %s' % self.talos_dir)
I'm more of a "check talos out from hg and know where the path is" type person rather than "install talos from pip and figure out where it's installed" type person.
However, if this works, I'm not opposed.
>+ def install_pageloader(self):
>+ """install pageloader"""
>+ if not hasattr(self, 'talos_dir'):
>+ self._set_talos_dir()
>+ dest = os.path.join(self.talos_dir, 'page_load_test', 'pageloader.xpi')
>+ self._fetch(self.config['pageloader_url'], dest)
I solved this by checking out hg.m.o/build/pageloader into the talos directory. Tmtowtdi, but this may be part of what we want to standardize at some point.
https://github.com/escapewindow/mozharness/blob/talosrunner/configs/users/aki/tablet1.py#L42
https://github.com/escapewindow/mozharness/blob/talosrunner/scripts/device_talosrunner.py#L217
>+ def generate_config(self):
>+ """generate talos configuration"""
There may be more options we want here, but this is a good start.
There may be additional work that needs doing before this script is both fully featured for everything we do in production, and we have parity between the two talos scripts. But other than possibly s,_fetch,download_file, I don't see anything glaringly wrong here.
Thanks for the script!
Attachment #573963 -
Flags: review?(aki) → review+
Comment 15•13 years ago
|
||
(In reply to Aki Sasaki [:aki] from comment #14)
> Comment on attachment 573963 [details] [diff] [review] [diff] [details] [review]
> basic mozharness for talos
>
> >+ def _fetch(self, url, dest):
> >+ """fetch a static resource from the web"""
> >+ # TODO: could be upstreamed
> >+ f = file(dest, 'w')
> >+ resource = urllib2.urlopen(url)
> >+ f.write(resource.read())
> >+ resource.close()
> >+ f.close()
>
> self.download_file() ?
> http://hg.mozilla.org/build/mozharness/file/cbcc9a5d5549/mozharness/base/
> script.py#l147
Ah, I missed that somehow. Thanks. I'll change before I check in.
> >+ def _set_talos_dir(self):
> >+ """XXX this is required as talos must be run out of its directory"""
> >+ python = self.query_python_path()
> >+ self.talos_dir = self.get_output_from_command([python, '-c', 'import os, talos; print os.path.dirname(os.path.abspath(talos.__file__))'])
> >+ if not self.talos_dir:
> >+ self.fatal("Talos directory could not be found")
> >+ self.info('Talos directory: %s' % self.talos_dir)
>
> I'm more of a "check talos out from hg and know where the path is" type
> person rather than "install talos from pip and figure out where it's
> installed" type person.
I would like to see a method that is "check out from hg/download from tarball and install via `setup.py`. Would that be worth ticketing? I'm more inclined to use a direct .tar.gz/.zip download versus hg to avoid another dependency but agree that the way I've done things is a bit magic.
Also note that there are also upstream talos issues here. Talos should not require to be run from a specific directory, but it does. Talos should also separate the software pieces from the data pieces but it does not. We'll have to fix these too, probably this quarter. I just haven't gotten around to figuring out what all needs to be done there.
> However, if this works, I'm not opposed.
>
> >+ def install_pageloader(self):
> >+ """install pageloader"""
> >+ if not hasattr(self, 'talos_dir'):
> >+ self._set_talos_dir()
> >+ dest = os.path.join(self.talos_dir, 'page_load_test', 'pageloader.xpi')
> >+ self._fetch(self.config['pageloader_url'], dest)
>
> I solved this by checking out hg.m.o/build/pageloader into the talos
> directory. Tmtowtdi, but this may be part of what we want to standardize at
> some point.
>
> https://github.com/escapewindow/mozharness/blob/talosrunner/configs/users/
> aki/tablet1.py#L42
> https://github.com/escapewindow/mozharness/blob/talosrunner/scripts/
> device_talosrunner.py#L217
>
> >+ def generate_config(self):
> >+ """generate talos configuration"""
>
> There may be more options we want here, but this is a good start.
>
> There may be additional work that needs doing before this script is both
> fully featured for everything we do in production, and we have parity
> between the two talos scripts. But other than possibly
> s,_fetch,download_file, I don't see anything glaringly wrong here.
Indeed, will fix the _fetch -> download_file issue and push
> Thanks for the script!
Comment 16•13 years ago
|
||
Status: NEW → RESOLVED
Closed: 13 years ago
Resolution: --- → FIXED
Comment 17•13 years ago
|
||
Assignee | ||
Comment 18•13 years ago
|
||
(In reply to Jeff Hammel [:jhammel] from comment #15)
> I would like to see a method that is "check out from hg/download from
> tarball and install via `setup.py`. Would that be worth ticketing?
bug 697462 ?
> I'm more
> inclined to use a direct .tar.gz/.zip download versus hg to avoid another
> dependency but agree that the way I've done things is a bit magic.
The thing about the hg.m.o zip file is it's only tip, which means any checkin will get picked up immediately. We may want that behavior, but we may also want to lock a particular revision to a branch.
We do have talos tarballs already, so this isn't something that needs solving immediately.
> Also note that there are also upstream talos issues here. Talos should not
> require to be run from a specific directory, but it does. Talos should also
> separate the software pieces from the data pieces but it does not. We'll
> have to fix these too, probably this quarter. I just haven't gotten around
> to figuring out what all needs to be done there.
I'd love to see those things fixed, great to hear!
We may want followup bugs for a) --download support so this is user-at-desktop friendly, and b) to make device_talosrunner and this script share more backend methods.
Comment 19•13 years ago
|
||
Reopened due to the reopening of https://bugzilla.mozilla.org/show_bug.cgi?id=700722 . The code is still valid, it will just no longer work because of the downstream talos snafu. Unless there is a burning reason, I'd rather no backout the script
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Assignee | ||
Comment 20•13 years ago
|
||
I'm fine with keeping the script checked in, and adding patches to it as needed; nothing in production is calling this script.
Updated•13 years ago
|
Priority: P4 → P1
Whiteboard: [mozharness][talos] → [mozharness][talos][mozharness+talos]
Comment 21•13 years ago
|
||
re comment 19, bug 700722 is now fixed; closing
Status: REOPENED → RESOLVED
Closed: 13 years ago → 13 years ago
Resolution: --- → FIXED
Updated•11 years ago
|
Product: mozilla.org → Release Engineering
Updated•10 years ago
|
Component: Other → Mozharness
You need to log in
before you can comment on or make changes to this bug.
Description
•