Closed Bug 710779 Opened 13 years ago Closed 13 years ago

Modify peptest script to use mozbase/peptest in m-c

Categories

(Release Engineering :: General, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED INVALID

People

(Reporter: ahal, Assigned: ahal)

References

Details

(Whiteboard: [mozharness])

Attachments

(2 files)

Mozbase and peptest are now in m-c. The mozharness peptest script should have the capability to get these.
Assignee: nobody → ahalberstadt
Blocks: 700415
Status: NEW → ASSIGNED
Whiteboard: [mozharness]
Attachment #581701 - Flags: review?(aki)
Comment on attachment 581701 [details] [diff] [review]
Patch 1.0 - Fix extract error

Hit submit too soon by accident. This is the same patch from https://bugzilla.mozilla.org/show_bug.cgi?id=700415#c30

This doesn't fix the bug, that is upcoming. This fixes the extract issue and needs to be applied before the follow-up patch.
I started out putting this logic inside peptest.py, but it ended up looking pretty hacky (relative urls in the configs made it confusing). I decided just have the peptest script require full paths for peptest, mozbase and the test manifest, and handle the logic for finding said paths solely in the config file.

The 'test_config.py' file is an example of how one would go about this.
Attachment #581706 - Flags: review?(aki)
(In reply to Andrew Halberstadt [:ahal] from comment #3)
> Created attachment 581706 [details] [diff] [review]
> Patch 1.1 (inc) - use peptest/mozbase in m-c
> 
> I started out putting this logic inside peptest.py, but it ended up looking
> pretty hacky (relative urls in the configs made it confusing). I decided
> just have the peptest script require full paths for peptest, mozbase and the
> test manifest, and handle the logic for finding said paths solely in the
> config file.
> 
> The 'test_config.py' file is an example of how one would go about this.

That works, but will break specifying --work-dir on the commandline... not the biggest thing, but I prefer keeping things predictable and usable from both config file or commandline.

I'll probably write the mozharness-esque fix for this based on your patches, and add to the existing changes I've made to get this running so far.
Sounds good.

This is sort of tangential, but would there be an easy way to expose the command line configs to the config.py? It would be nice to e.g, make sure the virtualenv is always created inside the work_dir by default.

(Then again I'm probably not the one who will need to deal with configs all the time so I'll just let you implement things as you see fit)
(In reply to Andrew Halberstadt [:ahal] from comment #5)
> Sounds good.
> 
> This is sort of tangential, but would there be an easy way to expose the
> command line configs to the config.py? It would be nice to e.g, make sure
> the virtualenv is always created inside the work_dir by default.

I haven't found an elegant way to do that yet.

Also, that's not always wanted.
For example:

You want to run the same job 25,000 times. You want to clobber the work_dir every time. The python modules never change in that time frame. Would you prefer to:

a) re-create the same virtualenv, and have that be a noop operation, or
b) download and reinstall all the virtualenv modules every single time?

I can think of more times when I'd want the virtualenv to stick around than I can wanting to blow it away, but maybe I'm not thinking along the same lines here.

Also, I'm going to solve a lot of these problems by overriding query_abs_dirs() and specifying an abs_mozbase_dir and abs_peptest_dir.
This is my first stab at it:
http://hg.mozilla.org/users/asasaki_mozilla.com/mozharness/rev/6d6114250661

I added a mozharness.base.python.query_virtualenv_path():
http://hg.mozilla.org/users/asasaki_mozilla.com/mozharness/rev/6d6114250661#l3.33
If you specify an abs_virtualenv_dir in query_abs_dirs(), it'll use that.
If you specify an absolute path in c['virtualenv_path'], it'll use that.
If you specify a relative path there, it'll use that path under base_work_dir.
If I'm wrong, and we usually want to clobber the venv along with everything else, I'll change that default.

I changed your assert to be more mozharness-esque:
http://hg.mozilla.org/users/asasaki_mozilla.com/mozharness/rev/6d6114250661#l4.12

I added a PepTest.query_abs_dirs(); testing will tell if I got everything.
http://hg.mozilla.org/users/asasaki_mozilla.com/mozharness/rev/6d6114250661#l5.58

I simplified the _install*() methods:
http://hg.mozilla.org/users/asasaki_mozilla.com/mozharness/rev/6d6114250661#l5.117

I changed the behavior of automatically doing things and force you to explicitly ask for them, in places like this:
http://hg.mozilla.org/users/asasaki_mozilla.com/mozharness/rev/6d6114250661#l5.190
I made a number of other changes like that in earlier commits.

Ah, I think I need to change that app_install_dir at the bottom to a dirs['abs_app_install_dir'].  I will commence testing and debugging.
Since this is rolled into the patch on bug 700415, closing this bug.
Status: ASSIGNED → RESOLVED
Closed: 13 years ago
Resolution: --- → INVALID
Attachment #581701 - Flags: review?(aki)
Attachment #581706 - Flags: review?(aki)
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: