Closed Bug 1007262 Opened 6 years ago Closed 4 years ago

talos should use in-tree mozbase

Categories

(Testing :: Talos, defect)

defect
Not set

Tracking

(firefox43 fixed)

RESOLVED FIXED
mozilla43
Tracking Status
firefox43 --- fixed

People

(Reporter: aki, Assigned: parkouss)

References

Details

Attachments

(1 file)

IIRC, Joel ran into problems with conflicting versions of mozbase in-tree (for mach) and in-house pypi (for Talos).  By keeping a separate set of mozbase version dependencies for Talos, we're going to regularly hit this problem.

We'll need to make Talos compatible with the in-tree versions of mozbase; this may be easiest if we check Talos into the tree, but may not be a hard requirement.  To keep Talos in lockstep, we can then have Talos download the test zip and install mozbase from there, like unittests already do.  Once this happens, any incompatibilities between Talos and in-tree mozbase will show up in pushes, and we will no longer need to depend on our in-house pypi for Talos.
An alternative solution to this might be to block on bug 985141 and then have Talos specify the older versions of mozbase as dependencies. I don't know when or if that will happen though.
Depends on: 1189714
Depends on: 1180664
Depends on: 1190265
Depends on: 1191197
Bug 1007262 - talos should use in-tree mozbase. r=ahal
Attachment #8660356 - Flags: review?(ahalberstadt)
So, this patch should use in-tree mozbase deps.

This works fine locally (./mach talos-test chromez), and should be good on try. I checked on linux:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=47c6b3273964

this looks good, since in the log file (section "create-virtualenv") I don't see that mozbase packages are downloaded anymore.

Full push to try here:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=b21ba5964b97

Also note that MozBaseMixin in mozharnes can't be used, it break things locally.

:ahal, I asked you for the review, feel free to redirect if needed! Thanks. :)
Assignee: nobody → j.parkouss
Status: NEW → ASSIGNED
Comment on attachment 8660356 [details]
MozReview Request: Bug 1007262 - talos should use in-tree mozbase. r=jmaher

https://reviewboard.mozilla.org/r/19117/#review17225

::: testing/mozharness/mozharness/mozilla/testing/talos.py:514
(Diff revision 1)
> +            # locally, install mozbase local deps - we find them
> +            # looking for */setup.py files in the mozbase src dir
> +            mozbase_dir = os.path.join(
> +                os.path.dirname(os.path.dirname(self.talos_json)),
> +                'mozbase'
> +            )
> +            modules = [os.path.dirname(fname)
> +                       for fname in glob.glob(os.path.join(mozbase_dir,
> +                                                           '*',
> +                                                           'setup.py'))]
> +            self.run_command(
> +                [self.query_python_path("pip"), 'install', '-e'] + modules

If running through mach, mozbase modules should already be on sys.path. Obviously a virtualenv is better.. just commenting this might not be necessary if you expect people to mostly use mach. If people aren't using mach for some reason, I think it's reasonable to make them set up mozbase manually (this how all the other test harnesses work).

This is fine as is though, I'll leave it up to you.
Attachment #8660356 - Flags: review?(ahalberstadt) → review+
Comment on attachment 8660356 [details]
MozReview Request: Bug 1007262 - talos should use in-tree mozbase. r=jmaher

Bug 1007262 - talos should use in-tree mozbase. r=jmaher
Attachment #8660356 - Attachment description: MozReview Request: Bug 1007262 - talos should use in-tree mozbase. r=ahal → MozReview Request: Bug 1007262 - talos should use in-tree mozbase. r=jmaher
Attachment #8660356 - Flags: review?(jmaher)
Ok, updated with our irc discussion.
Comment on attachment 8660356 [details]
MozReview Request: Bug 1007262 - talos should use in-tree mozbase. r=jmaher

https://reviewboard.mozilla.org/r/19117/#review17265

thanks for doing this.
https://hg.mozilla.org/mozilla-central/rev/91040fc155c4
Status: ASSIGNED → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla43
You need to log in before you can comment on or make changes to this bug.