908296 - psutil breaks in Debian wheezy

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Micah Lee]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130806 Firefox/17.0 Iceweasel/17.0.8 (Nightly/Aurora)
Build ID: 20130806234539

Steps to reproduce:

I'm running an up-to-date Debian wheezy, but I can't build mozilla-central due to psutil fatal errors. You should be able to reproduce by cloning the repo in wheezy, running:

./mach bootstrap

Then:

./mach build


Actual results:

micah@spock:~/projects/mozilla-central$ ./mach build
/home/micah/eff_projects/httpse/mozilla-central-httpse-tests/python/psutil/psutil/_pslinux.py:102: RuntimeWarning: couldn't determine platform's TOTAL_PHYMEM
  warnings.warn("couldn't determine platform's TOTAL_PHYMEM", RuntimeWarning)
Error running mach:

    ['build']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.

You should consider filing a bug for this issue.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

AttributeError: 'module' object has no attribute 'get_sysinfo'

  File "/home/micah/eff_projects/httpse/mozilla-central-httpse-tests/python/mozbuild/mozbuild/mach_commands.py", line 299, in build
    monitor.init(warnings_path)
  File "/home/micah/eff_projects/httpse/mozilla-central-httpse-tests/python/mozbuild/mozbuild/controller/building.py", line 277, in init
    self.resources = SystemResourceMonitor(poll_interval=1.0)
  File "/home/micah/eff_projects/httpse/mozilla-central-httpse-tests/testing/mozbase/mozsystemmonitor/mozsystemmonitor/resourcemonitor.py", line 180, in __init__
    virt = psutil.virtual_memory()
  File "/home/micah/eff_projects/httpse/mozilla-central-httpse-tests/python/psutil/psutil/__init__.py", line 1154, in virtual_memory
    return _psplatform.virtual_memory()
  File "/home/micah/eff_projects/httpse/mozilla-central-httpse-tests/python/psutil/psutil/_pslinux.py", line 137, in virtual_memory
    total, free, buffers, shared, _, _ = _psutil_linux.get_sysinfo()


I think this is a bug with the version of psutil that's included with mozilla-central. The version I have installed in Debian seems to work to include:

micah@spock:~/projects/mozilla-central$ python
Python 2.7.3 (default, Jan  2 2013, 13:56:14) 
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import psutil
>>> 


But when I load the mozilla-central version I get an error:

micah@spock:~/projects/mozilla-central/python/psutil$ ls
CREDITS  examples  HISTORY  LICENSE  MANIFEST.in  PKG-INFO  psutil  README  setup.py  test
micah@spock:~/projects/mozilla-central/python/psutil$ python
Python 2.7.3 (default, Jan  2 2013, 13:56:14) 
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import psutil
psutil/_pslinux.py:102: RuntimeWarning: couldn't determine platform's TOTAL_PHYMEM
  warnings.warn("couldn't determine platform's TOTAL_PHYMEM", RuntimeWarning)
>>> 



Expected results:

"./mach build" should have built mozilla-central.

Comment 1

[Gregory Szorc [:gps]]

Perhaps we should upgrade the psutil in m-c. Will submit a patch to try momentarily.

Comment 2

[Gregory Szorc [:gps]]

Attachment: Upgrade psutil to version 1.0.1

Archive obtained from
https://psutil.googlecode.com/files/psutil-1.0.1.tar.gz and extracted
over existing source code without modifications.

https://tbpl.mozilla.org/?tree=Try&rev=b242a872ce53

Comment 3

[Mike Hommey [:glandium]]

Comment on attachment 794322 [details] [diff] [review]
Upgrade psutil to version 1.0.1

Review of attachment 794322 [details] [diff] [review]:
-----------------------------------------------------------------

rs=me

Comment 4

[Gregory Szorc [:gps]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/ed0b6a11532d

Comment 5

[Gregory Szorc [:gps]]

Despite the conversions on IRC, I forgot to touch a virtualenv file when I landed this, so it caused some minor bustage on inbound. Pushed a trivial followup touching populate_virtualenv.py in a non-significant way.

https://hg.mozilla.org/integration/mozilla-inbound/rev/1ad8a6674785

Comment 6

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/ed0b6a11532d
https://hg.mozilla.org/mozilla-central/rev/1ad8a6674785

Comment 7

[Ryan VanderMeulen [:RyanVM]]

Backed out at the request of #developers for causing OSX mach bustage.
https://hg.mozilla.org/integration/mozilla-inbound/rev/31aa3eb1a40d

Comment 8

[Adam Roach [:abr]]

The update to psutil 1.0.1 broke mach under OS X (confirmed independently by two developers under both OSX 10.8 and OSX 10.9). The specific issue (which seems to be the same problem fixed by 1ad8a6674785) is the absence of a TCPS_ESTABLISHED attribute on the _psutil_osx module:

> $ ./mach build
> Error running mach:
> 
>     ['build']
> 
> The error occurred in code that was called by the mach command. This is either
> a bug in the called code itself or in the way that mach is calling it.
> 
> You should consider filing a bug for this issue.
> 
> If filing a bug, please include the full output of mach, including this error
> message.
> 
> The details of the failure are as follows:
> 
> AttributeError: 'module' object has no attribute 'TCPS_ESTABLISHED'
> 
>   File "/Users/Adam/devel/mozilla/mozilla-inbound/python/mozbuild/mozbuild/mach_commands.py", line 293, in build
>     from mozbuild.controller.building import BuildMonitor
>   File "/Users/Adam/devel/mozilla/mozilla-inbound/python/mozbuild/mozbuild/controller/building.py", line 22, in <module>
>     import psutil
>   File "/Users/Adam/devel/mozilla/mozilla-inbound/python/psutil/psutil/__init__.py", line 95, in <module>
>     import psutil._psosx as _psplatform
>   File "/Users/Adam/devel/mozilla/mozilla-inbound/python/psutil/psutil/_psosx.py", line 48, in <module>
>     _TCP_STATES_TABLE = {_psutil_osx.TCPS_ESTABLISHED : CONN_ESTABLISHED,

Comment 9

[Peter Van der Beken [:peterv]]

Removing

python/psutil/build
python/psutil/_psutil_osx.so
python/psutil/_psutil_posix.so

fixed my OS X build.

Comment 10

[Adam Roach [:abr]]

Wow. I'm surprised we have binary build output showing up in the tree in locations other than obj-*. Would it be possible to move these outputs into the same directory as the rest of the binary build objects so that clobbers clean them up?

Comment 11

[Gregory Szorc [:gps]]

This is a dependency problem. The virtualenv needs to be repopulated before psutil can be imported or the aforementioned TCPS_ESTABLISHED error will occur. By backing this out, you've likely made the problem worse, as now developers who've upgraded will get errors due to newer psutil being used on import. The correct solution is to tell people to run `hg status -in python/psutil | xargs rm`.

(In reply to Adam Roach [:abr] from comment #10)
> Wow. I'm surprised we have binary build output showing up in the tree in
> locations other than obj-*. Would it be possible to move these outputs into
> the same directory as the rest of the binary build objects so that clobbers
> clean them up?

Known issue. It really sucks. It will take some knowledge of Python packaging to fix. It needs to be fixed. Fortunately, we've only upgraded psutil twice in over a year, so as annoying as it is, it's not frequent enough to highly prioritize, IMO.

Comment 12

[Adam Roach [:abr]]

(In reply to Gregory Szorc [:gps] from comment #11)
> The correct solution is to tell people to run `hg status -in
> python/psutil | xargs rm`.
 
This kind of "hallway lore" is very brittle -- most people who need it won't have access to it when they need it. Can we be careful to at least include this information in the bug the next time we update the package? Or, even better, put a try/catch around the package inclusion that takes care of running this command for developers?

Comment 13

[Gregory Szorc [:gps]]

Relanded because the backout will cause bustage as well.

https://hg.mozilla.org/integration/mozilla-inbound/rev/037e6a08032f

Comment 14

[Peter Van der Beken [:peterv]]

FWIW, I don't think it's acceptable to break most developers trees on OS X and let them all figure out by themselves how to unbreak them.

Comment 15

[Myk Melez [:myk] [@mykmelez]]

(In reply to Gregory Szorc [:gps] from comment #11)
> The correct solution is to tell people to run `hg status -in python/psutil | xargs rm`.

For those using a clone of the Git mirror, an equivalent is:

git clean -xf python/psutil/

# -x to include ignored files; -f to force the clean (replace with -n to see what would be cleaned first)

Comment 16

[Guillaume Abadie]

(In reply to Peter Van der Beken [:peterv] from comment #14)
> FWIW, I don't think it's acceptable to break most developers trees on OS X
> and let them all figure out by themselves how to unbreak them.
I definitely agree!
We lived without it before, and now that will be painful while bisecting on OS X after that patch. What about landing a patch that would fix all bustages instead?

Comment 17

[Gregory Szorc [:gps]]

Applied some wallpaper to catch all Exceptions when importing psutil. This should limit the impact of this patch. People building a changeset with new psutil but not this changeset may still run into issues, however. Perhaps we should cherry-pick it to central before the inbound merge?

https://hg.mozilla.org/integration/mozilla-inbound/rev/7be399d00f9a

Comment 18

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/037e6a08032f
https://hg.mozilla.org/mozilla-central/rev/7be399d00f9a