Talos mozharness should download talos only once

RESOLVED FIXED

Status

Release Engineering
Mozharness
RESOLVED FIXED
5 years ago
4 years ago

People

(Reporter: jyeo, Assigned: jyeo)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments)

(Assignee)

Description

5 years ago
Currently talos is downloaded twice.
1. It is cloned from http://hg.mozilla.org/build/talos.
2. It is downloaded from puppetagain and installed into virtualenv using pip.

A possible solution is to clone from the mercurial repo, and use pip to install from that local directory.

Comment 1

5 years ago
Change the virtualenv_modules here:
http://hg.mozilla.org/build/mozharness/file/86e067433595/mozharness/mozilla/testing/talos.py#l127

If it's a string, it'll install it via whatever pypi server we're pointing at.
If it's a dict, it'll use the value as path.

kwargs['config'].setdefault(
    'virtualenv_modules', [
        {"talos": "path/to/talos"},
        "mozinstall"
    ]
)

where path/to/talos/ is a directory containing setup.py.  e.g. http://hg.mozilla.org/build/mozharness/file/86e067433595/scripts/desktop_unittest.py#l69

Comment 2

5 years ago
(if you take this approach, we have to clone talos before we can create the virtualenv.)
(Assignee)

Comment 3

5 years ago
Created attachment 759919 [details] [diff] [review]
Install talos from it's mercurial repo

I changed talos_url to talos_path so that it will install from the local directory instead of the url.
Assignee: nobody → yshun
Status: NEW → ASSIGNED
Attachment #759919 - Flags: feedback?(aki)

Comment 4

5 years ago
Comment on attachment 759919 [details] [diff] [review]
Install talos from it's mercurial repo

a) You might want to

             virtualenv_modules = self.config.get('virtualenv_modules', [])[:]

so you don't edit your config accidentally, or this might barf when I fix bug 706036.

b) is there a reason you're editing self.config['virtualenv_modules'] rather than making it use the cloned talos every time?  Trying not to affect jetperf?
Attachment #759919 - Flags: feedback?(aki) → feedback+
(Assignee)

Comment 5

5 years ago
(In reply to Aki Sasaki [:aki] from comment #4)
> Comment on attachment 759919 [details] [diff] [review]
> Install talos from it's mercurial repo
> 
> a) You might want to
> 
>              virtualenv_modules = self.config.get('virtualenv_modules',
> [])[:]

Okay.

> b) is there a reason you're editing self.config['virtualenv_modules'] rather
> than making it use the cloned talos every time?  Trying not to affect
> jetperf?

Not really. I'm just trying to minimize the amount of changes to the code.
(Assignee)

Comment 6

5 years ago
Created attachment 759933 [details] [diff] [review]
Install talos from it's mercurial repo
Attachment #759933 - Flags: feedback?(aki)

Updated

5 years ago
Attachment #759933 - Flags: feedback?(aki) → feedback+
(Assignee)

Comment 7

5 years ago
Logs show that all platforms are installing from local repository. Fixed as of http://hg.mozilla.org/users/asasaki_mozilla.com/ash-mozharness/rev/de8df7a74f8b.
(Assignee)

Updated

5 years ago
Attachment #759933 - Flags: checked-in+
(Assignee)

Updated

5 years ago
Attachment #759919 - Flags: checked-in+
(Assignee)

Updated

5 years ago
Status: ASSIGNED → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering

Updated

4 years ago
Component: General Automation → Mozharness
You need to log in before you can comment on or make changes to this bug.