805835 - virtualenv.py fails in fix_lib64 if target for symlink already exist

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Maxim Philippov]

Attachment: sugested patch

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:19.0) Gecko/19.0 Firefox/19.0
Build ID: 20121022162853

Steps to reproduce:

Updated from trunk and started compilation through `make -f client.mk`


Actual results:

Compilation failed at start, because fix_lib64 from `python/virtualenv/virtualenv.py' tried to create symlink to an existing target

make -f client.mk clean
Adding client.mk options from /home/maxim/projects/mozilla/.mozconfig:
    MOZ_MAKE_FLAGS=-j4
make[1]: Entering directory `/home/maxim/projects/mozilla'
cp /home/maxim/projects/mozilla/.mozconfig obj-x86_64-unknown-linux-gnu/.mozconfig
cd obj-x86_64-unknown-linux-gnu
/home/maxim/projects/mozilla/configure
loading cache ./config.cache
checking host system type... x86_64-unknown-linux-gnu
checking target system type... x86_64-unknown-linux-gnu
checking build system type... x86_64-unknown-linux-gnu
checking for gawk... (cached) gawk
checking for python2.7... (cached) /usr/bin/python2.6
Creating Python environment
Traceback (most recent call last):
  File "/home/maxim/projects/mozilla/python/virtualenv/virtualenv.py", line 2471, in <module>
    main()
  File "/home/maxim/projects/mozilla/python/virtualenv/virtualenv.py", line 942, in main
    never_download=options.never_download)
  File "/home/maxim/projects/mozilla/python/virtualenv/virtualenv.py", line 1043, in create_environment
    site_packages=site_packages, clear=clear))
  File "/home/maxim/projects/mozilla/python/virtualenv/virtualenv.py", line 1186, in install_python
    fix_lib64(lib_dir)
  File "/home/maxim/projects/mozilla/python/virtualenv/virtualenv.py", line 1564, in fix_lib64
    os.path.join(os.path.dirname(lib_parent), 'lib64'))
OSError: [Errno 17] File exists
Traceback (most recent call last):
  File "/home/maxim/projects/mozilla/build/virtualenv/populate_virtualenv.py", line 321, in <module>
    manager.ensure()
  File "/home/maxim/projects/mozilla/build/virtualenv/populate_virtualenv.py", line 81, in ensure
    return self.build()
  File "/home/maxim/projects/mozilla/build/virtualenv/populate_virtualenv.py", line 258, in build
    self.create()
  File "/home/maxim/projects/mozilla/build/virtualenv/populate_virtualenv.py", line 105, in create
    raise Exception('Error creating virtualenv.')
Exception: Error creating virtualenv.
*** Fix above errors and then restart with               "make -f client.mk build"

Comment 1

[nirvansk815]

Can confirm bug. Suggested patch worked for me, thanks:)

Comment 2

[Gregory Szorc [:gps]]

This patch is against the upstream virtualenv package. We will likely never take it directly in mozilla-central. You should report the issue at http://www.virtualenv.org/

I suspect that if you delete the virtualenv directory and build again, things will magically work.

  rm -rf /path/to/obdir/_virtualenv

Comment 3

[Gregory Szorc [:gps]]

You may also want to upgrade to Python 2.7, as the tree will likely require it within the next few months.

Comment 4

[Maxim Philippov]

updated from trunk, distclean-ed, built, updated, started incremental build - unable to reproduce problem. 
I'll try to report the issue at virtualenv project. Thanks for python upgrade advice.

Comment 5

[Maxim Philippov]

Looks like this issue is fixed in https://github.com/pypa/virtualenv/issues/334

Comment 6

[Gregory Szorc [:gps]]

I guess we'll pull this in with the next virtualenv release. Until then, issue is resolved per comment #4.

Comment 7

[Phil Ringnalda (:philor)]

https://tbpl.mozilla.org/php/getParsedLog.php?id=16667918&tree=Mozilla-Inbound
https://tbpl.mozilla.org/php/getParsedLog.php?id=16668454&tree=Mozilla-Inbound

Comment 8

[Gregory Szorc [:gps]]

Well, crap. Since we're seeing it on our build infra, I suppose we should fix this.

I reckon the easy fix is to cherry pick the upstream virtualenv fix into the tree...

Comment 9

[Gregory Szorc [:gps]]

Attachment: Cherry pick upstream patches, v1

I looked at the commit history between 1.8.2..develop of the virtualenv repository and identified 3 commits referencing this issue. I cherry picked those 3 commits into our in-tree copy of virtualenv. It seems to work locally. But, I never experienced any issue. Perhaps the original reporter can confirm that this patch fixes it.

Try at https://tbpl.mozilla.org/?tree=Try&rev=67387aef163a

Comment 10

[Gregory Szorc [:gps]]

Can you please confirm this patch to mozilla-central fixes things?

Comment 11

[Maxim Philippov]

I confirm this patch fixes the issue.

Comment 12

[Mike Hommey [:glandium]]

Comment on attachment 677592 [details] [diff] [review]
Cherry pick upstream patches, v1

Review of attachment 677592 [details] [diff] [review]:
-----------------------------------------------------------------

I won't pretend to understand what that changes, but it does apply the patches mentioned in the patch summary, so r+.

Comment 13

[Gregory Szorc [:gps]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/51dc263099d4

Comment 14

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/51dc263099d4