make psutil a dependency of Talos

RESOLVED FIXED in Firefox 43

Status

Testing
Talos
RESOLVED FIXED
3 years ago
2 years ago

People

(Reporter: parkouss, Assigned: parkouss)

Tracking

unspecified
mozilla43
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox43 fixed)

Details

Attachments

(1 attachment, 1 obsolete attachment)

(Assignee)

Description

3 years ago
Adding psutil as a Talos dependency will help us in at least two ways:

 - add some counters (bug 1193707 for example)
 - remove custom Talos code to get process list and kill them (ffprocess.py)

There is a working patch in bug 1193707 comment 10 - this needs to be reworked a bit, but it is on the good road I think.
(Assignee)

Updated

3 years ago
Blocks: 1194076
(Assignee)

Comment 1

3 years ago
:gps, we would need to install psutil when running talos jobs on harness.

We added in internal pypi:

[ ]	psutil-3.1.1-cp27-none-win32.whl	12-Aug-2015 05:48 	86K	 
[ ]	psutil-3.1.1-cp27-none-win_amd64.whl	12-Aug-2015 05:48 	88K	 
[ ]	psutil-3.1.1.tar.gz	12-Aug-2015 05:47 	241K	 

And with the following mozharness patch, I am able to require psutil from talos, even on windows:

https://hg.mozilla.org/try/rev/27c2318c59b5

try is good, and looking at the logs the right psutil is installed:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=f4dd950feab2

The patch basically does two things:

 - require pip >= 1.5 before installing dependencies (pip >=1.5 will use wheels when available)
 - install psutil >= 0.7.1 instead of ==0.7.1 in testing/mozharness/mozharness/base/python.py

I'm a bit worried about the second point, as it will not impact talos only. But that should work if:

 - only mozsystemmonitor uses the psutil installed in harness (https://dxr.mozilla.org/mozilla-central/source/testing/mozharness/mozharness/base/python.py#443)
 - mozsystemmonitor works with psutil >=0.7.1 (https://dxr.mozilla.org/mozilla-central/source/testing/mozbase/mozsystemmonitor/setup.py#28)


So, do you think that patch is applicable ? or should I try to impact talos only ? or something else ?
Flags: needinfo?(gps)
gps, it would be great to hear from you here so we can move forward.

Comment 3

3 years ago
Sorry for delayed response - on PTO for a long weekend and traveling yesterday.

mozsystemmonitor likely hasn't been explicitly tested with psutil 3.1. You should manually run tests against that version and make sure things still work. Upgrading the in-tree copy of psutil to 3.1.1 would also be a good idea to ensure we stay compatible with 3.1+ going forward.
Flags: needinfo?(gps)

Comment 4

3 years ago
I would also be very aggressive about using the newest available psutil version. If you are going to deploy 3.1.1 somewhere, we should see about upgrading everything to using 3.1.1+ if possible.
(Assignee)

Updated

2 years ago
Depends on: 1196253
I just ran the mozsystemmonitor tests (testing/mozbase/mozsystemmonitor/mozsystemmonitor/test/test_resource_monitor.py) locally with psutil 3.1.1 and didn't encounter any failures.
(Assignee)

Comment 6

2 years ago
Created attachment 8650466 [details] [diff] [review]
1194074.patch

So I just added some comments, but this is the same patch as in comment 1. So try was good for Talos, and as :chmanchester run the mozsystemmonitor tests with psutil 3.1.1, we should be good to land I think.
Assignee: nobody → j.parkouss
Status: NEW → ASSIGNED
Attachment #8650466 - Flags: review?(gps)

Updated

2 years ago
Attachment #8650466 - Flags: review?(gps) → review+
https://hg.mozilla.org/mozilla-central/rev/6e2dea2674da
Status: ASSIGNED → RESOLVED
Last Resolved: 2 years ago
status-firefox43: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla43
Created attachment 8652567 [details]
MozReview Request: Bug 1194074 - Install psutil 3.1.1 in the desktop_unittests vitualenv so it will be available to desktop unittests. r=jlund

Bug 1194074 - Install psutil 3.1.1 in the desktop_unittests vitualenv so it will be available to desktop unittests. r=jlund
Attachment #8652567 - Flags: review?(jlund)
Comment on attachment 8652567 [details]
MozReview Request: Bug 1194074 - Install psutil 3.1.1 in the desktop_unittests vitualenv so it will be available to desktop unittests. r=jlund

Sorry, wrong bug.
Attachment #8652567 - Flags: review?(jlund)
Attachment #8652567 - Attachment is obsolete: true
You need to log in before you can comment on or make changes to this bug.