Closed
Bug 908296
Opened 11 years ago
Closed 11 years ago
psutil breaks in Debian wheezy
Categories
(Firefox Build System :: General, defect)
Tracking
(Not tracked)
RESOLVED
FIXED
mozilla26
People
(Reporter: micahflee, Assigned: gps)
References
Details
(Whiteboard: [Run `hg status -in python/psutil | xargs rm` to reset psutil])
Attachments
(1 file)
286.86 KB,
patch
|
glandium
:
review+
|
Details | Diff | Splinter Review |
User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130806 Firefox/17.0 Iceweasel/17.0.8 (Nightly/Aurora) Build ID: 20130806234539 Steps to reproduce: I'm running an up-to-date Debian wheezy, but I can't build mozilla-central due to psutil fatal errors. You should be able to reproduce by cloning the repo in wheezy, running: ./mach bootstrap Then: ./mach build Actual results: micah@spock:~/projects/mozilla-central$ ./mach build /home/micah/eff_projects/httpse/mozilla-central-httpse-tests/python/psutil/psutil/_pslinux.py:102: RuntimeWarning: couldn't determine platform's TOTAL_PHYMEM warnings.warn("couldn't determine platform's TOTAL_PHYMEM", RuntimeWarning) Error running mach: ['build'] The error occurred in code that was called by the mach command. This is either a bug in the called code itself or in the way that mach is calling it. You should consider filing a bug for this issue. If filing a bug, please include the full output of mach, including this error message. The details of the failure are as follows: AttributeError: 'module' object has no attribute 'get_sysinfo' File "/home/micah/eff_projects/httpse/mozilla-central-httpse-tests/python/mozbuild/mozbuild/mach_commands.py", line 299, in build monitor.init(warnings_path) File "/home/micah/eff_projects/httpse/mozilla-central-httpse-tests/python/mozbuild/mozbuild/controller/building.py", line 277, in init self.resources = SystemResourceMonitor(poll_interval=1.0) File "/home/micah/eff_projects/httpse/mozilla-central-httpse-tests/testing/mozbase/mozsystemmonitor/mozsystemmonitor/resourcemonitor.py", line 180, in __init__ virt = psutil.virtual_memory() File "/home/micah/eff_projects/httpse/mozilla-central-httpse-tests/python/psutil/psutil/__init__.py", line 1154, in virtual_memory return _psplatform.virtual_memory() File "/home/micah/eff_projects/httpse/mozilla-central-httpse-tests/python/psutil/psutil/_pslinux.py", line 137, in virtual_memory total, free, buffers, shared, _, _ = _psutil_linux.get_sysinfo() I think this is a bug with the version of psutil that's included with mozilla-central. The version I have installed in Debian seems to work to include: micah@spock:~/projects/mozilla-central$ python Python 2.7.3 (default, Jan 2 2013, 13:56:14) [GCC 4.7.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import psutil >>> But when I load the mozilla-central version I get an error: micah@spock:~/projects/mozilla-central/python/psutil$ ls CREDITS examples HISTORY LICENSE MANIFEST.in PKG-INFO psutil README setup.py test micah@spock:~/projects/mozilla-central/python/psutil$ python Python 2.7.3 (default, Jan 2 2013, 13:56:14) [GCC 4.7.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import psutil psutil/_pslinux.py:102: RuntimeWarning: couldn't determine platform's TOTAL_PHYMEM warnings.warn("couldn't determine platform's TOTAL_PHYMEM", RuntimeWarning) >>> Expected results: "./mach build" should have built mozilla-central.
Assignee | ||
Comment 1•11 years ago
|
||
Perhaps we should upgrade the psutil in m-c. Will submit a patch to try momentarily.
Assignee | ||
Comment 2•11 years ago
|
||
Archive obtained from https://psutil.googlecode.com/files/psutil-1.0.1.tar.gz and extracted over existing source code without modifications. https://tbpl.mozilla.org/?tree=Try&rev=b242a872ce53
Attachment #794322 -
Flags: review?(mh+mozilla)
Assignee | ||
Updated•11 years ago
|
Assignee: nobody → gps
Comment 3•11 years ago
|
||
Comment on attachment 794322 [details] [diff] [review] Upgrade psutil to version 1.0.1 Review of attachment 794322 [details] [diff] [review]: ----------------------------------------------------------------- rs=me
Attachment #794322 -
Flags: review?(mh+mozilla) → review+
Assignee | ||
Comment 4•11 years ago
|
||
https://hg.mozilla.org/integration/mozilla-inbound/rev/ed0b6a11532d
Status: UNCONFIRMED → ASSIGNED
Component: Untriaged → Build Config
Ever confirmed: true
Product: Firefox → Core
Version: 17 Branch → Trunk
Assignee | ||
Comment 5•11 years ago
|
||
Despite the conversions on IRC, I forgot to touch a virtualenv file when I landed this, so it caused some minor bustage on inbound. Pushed a trivial followup touching populate_virtualenv.py in a non-significant way. https://hg.mozilla.org/integration/mozilla-inbound/rev/1ad8a6674785
Comment 6•11 years ago
|
||
https://hg.mozilla.org/mozilla-central/rev/ed0b6a11532d https://hg.mozilla.org/mozilla-central/rev/1ad8a6674785
Status: ASSIGNED → RESOLVED
Closed: 11 years ago
Flags: in-testsuite+
Resolution: --- → FIXED
Target Milestone: --- → mozilla26
Comment 7•11 years ago
|
||
Backed out at the request of #developers for causing OSX mach bustage. https://hg.mozilla.org/integration/mozilla-inbound/rev/31aa3eb1a40d
Target Milestone: mozilla26 → ---
Updated•11 years ago
|
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Comment 8•11 years ago
|
||
The update to psutil 1.0.1 broke mach under OS X (confirmed independently by two developers under both OSX 10.8 and OSX 10.9). The specific issue (which seems to be the same problem fixed by 1ad8a6674785) is the absence of a TCPS_ESTABLISHED attribute on the _psutil_osx module:
> $ ./mach build
> Error running mach:
>
> ['build']
>
> The error occurred in code that was called by the mach command. This is either
> a bug in the called code itself or in the way that mach is calling it.
>
> You should consider filing a bug for this issue.
>
> If filing a bug, please include the full output of mach, including this error
> message.
>
> The details of the failure are as follows:
>
> AttributeError: 'module' object has no attribute 'TCPS_ESTABLISHED'
>
> File "/Users/Adam/devel/mozilla/mozilla-inbound/python/mozbuild/mozbuild/mach_commands.py", line 293, in build
> from mozbuild.controller.building import BuildMonitor
> File "/Users/Adam/devel/mozilla/mozilla-inbound/python/mozbuild/mozbuild/controller/building.py", line 22, in <module>
> import psutil
> File "/Users/Adam/devel/mozilla/mozilla-inbound/python/psutil/psutil/__init__.py", line 95, in <module>
> import psutil._psosx as _psplatform
> File "/Users/Adam/devel/mozilla/mozilla-inbound/python/psutil/psutil/_psosx.py", line 48, in <module>
> _TCP_STATES_TABLE = {_psutil_osx.TCPS_ESTABLISHED : CONN_ESTABLISHED,
Comment 9•11 years ago
|
||
Removing python/psutil/build python/psutil/_psutil_osx.so python/psutil/_psutil_posix.so fixed my OS X build.
Comment 10•11 years ago
|
||
Wow. I'm surprised we have binary build output showing up in the tree in locations other than obj-*. Would it be possible to move these outputs into the same directory as the rest of the binary build objects so that clobbers clean them up?
Assignee | ||
Comment 11•11 years ago
|
||
This is a dependency problem. The virtualenv needs to be repopulated before psutil can be imported or the aforementioned TCPS_ESTABLISHED error will occur. By backing this out, you've likely made the problem worse, as now developers who've upgraded will get errors due to newer psutil being used on import. The correct solution is to tell people to run `hg status -in python/psutil | xargs rm`. (In reply to Adam Roach [:abr] from comment #10) > Wow. I'm surprised we have binary build output showing up in the tree in > locations other than obj-*. Would it be possible to move these outputs into > the same directory as the rest of the binary build objects so that clobbers > clean them up? Known issue. It really sucks. It will take some knowledge of Python packaging to fix. It needs to be fixed. Fortunately, we've only upgraded psutil twice in over a year, so as annoying as it is, it's not frequent enough to highly prioritize, IMO.
Comment 12•11 years ago
|
||
(In reply to Gregory Szorc [:gps] from comment #11) > The correct solution is to tell people to run `hg status -in > python/psutil | xargs rm`. This kind of "hallway lore" is very brittle -- most people who need it won't have access to it when they need it. Can we be careful to at least include this information in the bug the next time we update the package? Or, even better, put a try/catch around the package inclusion that takes care of running this command for developers?
Assignee | ||
Comment 13•11 years ago
|
||
Relanded because the backout will cause bustage as well. https://hg.mozilla.org/integration/mozilla-inbound/rev/037e6a08032f
Whiteboard: [Run `hg status -in python/psutil | xargs rm` to reset psutil]
Comment 14•11 years ago
|
||
FWIW, I don't think it's acceptable to break most developers trees on OS X and let them all figure out by themselves how to unbreak them.
Comment 15•11 years ago
|
||
(In reply to Gregory Szorc [:gps] from comment #11) > The correct solution is to tell people to run `hg status -in python/psutil | xargs rm`. For those using a clone of the Git mirror, an equivalent is: git clean -xf python/psutil/ # -x to include ignored files; -f to force the clean (replace with -n to see what would be cleaned first)
Comment 16•11 years ago
|
||
(In reply to Peter Van der Beken [:peterv] from comment #14) > FWIW, I don't think it's acceptable to break most developers trees on OS X > and let them all figure out by themselves how to unbreak them. I definitely agree! We lived without it before, and now that will be painful while bisecting on OS X after that patch. What about landing a patch that would fix all bustages instead?
Assignee | ||
Comment 17•11 years ago
|
||
Applied some wallpaper to catch all Exceptions when importing psutil. This should limit the impact of this patch. People building a changeset with new psutil but not this changeset may still run into issues, however. Perhaps we should cherry-pick it to central before the inbound merge? https://hg.mozilla.org/integration/mozilla-inbound/rev/7be399d00f9a
Comment 18•11 years ago
|
||
https://hg.mozilla.org/mozilla-central/rev/037e6a08032f https://hg.mozilla.org/mozilla-central/rev/7be399d00f9a
Status: REOPENED → RESOLVED
Closed: 11 years ago → 11 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla26
Updated•6 years ago
|
Product: Core → Firefox Build System
You need to log in
before you can comment on or make changes to this bug.
Description
•