Closed Bug 1246585 Opened 8 years ago Closed 8 years ago

mozharness 'create_virtualenv' step takes 8x longer on Linux compared to OS X and Windows

Categories

(Testing :: Firefox UI Tests, defect)

Product:
Testing
Component:
Firefox UI Tests
Version:
45 Branch
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(firefox44 fixed, firefox45 fixed, firefox46 fixed, firefox47 fixed)

Status:
RESOLVED FIXED
Tracking Flags:
Tracking Status
firefox44 --- fixed
firefox45 --- fixed
firefox46 --- fixed
firefox47 --- fixed

People

(Reporter: whimboo, Assigned: whimboo)

References

(
URL
)

Details

I have checked a couple of our Firefox UI jobs on Treeherder and noticed that the execution times for our tests on Linux take twice the time as on OS X and Windows. This is by using mozharness as runner.

A first look showed me that we have huge wait times while installing Python packages in the create_virtualenv step:

 07:21:36     INFO -  Downloading/unpacking psutil>=0.7.1
 07:21:36     INFO -    http://pypi.pub.build.mozilla.org/pub uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available
 07:21:36     INFO -    http://pypi.pub.build.mozilla.org.proxxy1.srv.releng.scl3.mozilla.com/pub uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org.proxxy1.srv.releng.scl3.mozilla.com has it available
 07:21:36     INFO -    http://pypi.pub.build.mozilla.org/pub uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available
 07:23:37     INFO -    Running setup.py (path:/home/mozauto/jenkins/workspace/mozilla-central_update/build/venv/build/psutil/setup.py) egg_info for package psutil

Not sure why the proxxy1 index is queried for given that this should not happen in developer mode.
The problem is clearly located in the create_virtualenv step. Here what happens on those Linux boxes:

07:12:36 INFO - MultiFileLogger online at 20160207 07:12:36 in /home/mozauto/jenkins/workspace/mozilla-central_functional
[..]
07:21:01 INFO - ##### Running install step.


Compared to OS X:

05:25:18 INFO - MultiFileLogger online at 20160207 05:25:18 in /Users/mozauto/jenkins/workspace/mozilla-central_functional
[..]
05:26:11 INFO - ##### Running install step.

As you can see we have 8.5 minutes compared to 53s.
Assignee: nobody → hskupin
Status: NEW → ASSIGNED
This goes back to release but keep in mind that we always make use of the mozharness version from mozilla-central for now. This might change so that we use the version of the branch /revision we are testing. That means we should take care of it down to beta (which becomes the next ESR) but not release.
status-firefox44: --- → wontfix
status-firefox45: --- → affected
status-firefox46: --- → affected
status-firefox47: --- → affected
Summary: Text execution times are twice that long on Linux compared to OS X and Windows → mozharness 'create_virtualenv' step takes 8x longer on Linux compared to OS X and Windows
Interestingly this is only a problem on production but not for staging Linux nodes. Maybe bug 1241811 plays into account here and it's something related to the pip v8 warning. I will see this later.
Looks like that this is really the case here. Having mm-ub-1404-1 updated I executed an update job and now the times are fine with 3 minutes again:

https://treeherder.mozilla.org/#/jobs?repo=mozilla-aurora&revision=f19a31a82ae3&filter-searchStr=firefox%20ui%20update&filter-tier=1&filter-tier=2&filter-tier=3&selectedJob=1890599

Marking the dependency.
Depends on: 1241811
Fixed on all branches with bug 1241811 done.
Status: ASSIGNED → RESOLVED
Closed: 8 years ago
status-firefox44: wontfix → fixed
status-firefox45: affected → fixed
status-firefox46: affected → fixed
status-firefox47: affected → fixed
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.