907642 - Specifying absolute path for MOZ_OBJDIR on another hard drive breaks build

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Frank Wein [:mcsmurf]]

I'm currently trying to build Firefox on Windows with MOZ_OBJDIR being on another hard drive, so I wanted to specify an absolute path. It turns out this currently breaks the build:
$ mach build
 0:00.18 C:/mozilla-build/msys/bin/sh.exe -c c:/mozilla-build/python/python.exe c:/mozilla/tree-hg/comm-central/mozilla/build/pymake/make.py -f client.mk -s
 0:01.79 Adding client.mk options from c:/mozilla/tree-hg/comm-central/mozilla/.mozconfig:
 0:01.79     MOZ_OBJDIR=f:/mozilla/firefox-objdir-test
 0:01.79     MOZ_MAKE_FLAGS=-j6
[...]
 0:05.69 New python executable in f:/mozilla/firefox-objdir-test/_virtualenv\Scripts\python2.7.exe
 0:05.69 Also creating executable in f:/mozilla/firefox-objdir-test/_virtualenv\Scripts\python.exe
 0:05.87 Installing setuptools................done.
 0:06.40 Installing pip...................done.
 0:06.46 Traceback (most recent call last):
 0:06.47   File "c:/mozilla/tree-hg/comm-central/mozilla/build/virtualenv/populate_virtualenv.py", line 382, in <mod
ule>
 0:06.47     manager.populate()
 0:06.47   File "c:/mozilla/tree-hg/comm-central/mozilla/build/virtualenv/populate_virtualenv.py", line 277, in popu
late
 0:06.47     handle_package(package)
 0:06.47   File "c:/mozilla/tree-hg/comm-central/mozilla/build/virtualenv/populate_virtualenv.py", line 217, in hand
le_package
 0:06.47     f.write("%s\n" % os.path.relpath(path, python_lib))
 0:06.47   File "f:\mozilla\firefox-objdir-test\_virtualenv\lib\ntpath.py", line 512, in relpath
 0:06.47     % (path_prefix, start_prefix))
 0:06.47 ValueError: path is on drive c:, start on drive f:
 0:06.47 Traceback (most recent call last):
 0:06.47   File "c:/mozilla/tree-hg/comm-central/mozilla/build/virtualenv/populate_virtualenv.py", line 384, in <mod
ule>
 0:06.47     manager.ensure()
 0:06.47   File "c:/mozilla/tree-hg/comm-central/mozilla/build/virtualenv/populate_virtualenv.py", line 103, in ensu
re
 0:06.47     return self.build()
 0:06.47   File "c:/mozilla/tree-hg/comm-central/mozilla/build/virtualenv/populate_virtualenv.py", line 327, in buil
d
 0:06.47     raise Exception('Error populating virtualenv.')
 0:06.47 Exception: Error populating virtualenv.

My .mozconfig file:
ac_add_options --enable-debug
ac_add_options --enable-tests
ac_add_options --enable-application=browser
mk_add_options MOZ_OBJDIR=f:/mozilla/firefox-objdir-test
mk_add_options MOZ_MAKE_FLAGS="-j6"

I've also tried other variants like f:\\mozilla\\firefox-objdir-test or /f/mozilla/firefox-objdir-test, this just gives me different errors:
 0:01.81 No rule to make target 'f:\\mozilla\\firefox-objdir-test/Makefile' needed by ['<command-line>', 'f:\\\\mozilla\\\\firefox-objdir-test/Makefile']
and
 0:01.03 c:\mozilla\tree-hg\comm-central\mozilla\client.mk:137:2:For Windows Pymake builds, MOZ_OBJDIR must be a Windows [and not MSYS] style path.

Comment 1

[Gregory Szorc [:gps]]

Ugh. Can you work around this for the time being?

Comment 2

[Frank Wein [:mcsmurf]]

Yes, I use NTFS symbolic links for now :) (for example as reference for others: run the command "mklink /D obj-i686-pc-mingw32 F:\mozilla\firefox-objdir" with admin privs once in the source folder)

Comment 4

[(not currently active) Ted Mielczarek]

It looks like this is just what os.path.relpath gives when you give it two paths on different drives:
>>> os.path.relpath("c:\foo", "d:\foo")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "c:\mozilla-build\python\lib\ntpath.py", line 512, in relpath
    % (path_prefix, start_prefix))
ValueError: path is on drive C:, start on drive d:

I guess we just need smarter handling here that writes the abs path if the paths are on different drives:
http://hg.mozilla.org/mozilla-central/annotate/8f8a683dfc42/build/virtualenv/populate_virtualenv.py#l234

Comment 5

[Gregory Szorc [:gps]]

Indeed. Let's tackle this after populate_virtualenv.py is moved so we can avoid bit rot.

Comment 6

[Benjamin Smedberg]

Attachment: bug918869-virtualenv

Comment 7

[Benjamin Smedberg]

Please ignore the toolkit/xre change, that's for another bug

Comment 8

[Gregory Szorc [:gps]]

Comment on attachment 807811 [details] [diff] [review]
bug918869-virtualenv

Review of attachment 807811 [details] [diff] [review]:
-----------------------------------------------------------------

::: toolkit/xre/nsWindowsDllBlocklist.cpp
@@ +132,5 @@
>    { "prnx.dll", MAKE_VERSION(1, 3, 334, 9) },
>  
> +  // Older belgian ID card software causes Firefox to crash or hang on
> +  // shutdown, bug 831285 and 918399.
> +  { "beid35cardlayer.dll", MAKE_VERSION(3, 5, 6, 6968) },

o rly?

Comment 9

[Gregory Szorc [:gps]]

It looks like the patch was attributed to bug 918869, which has been marked as a dupe of this. Anyway, this landed in 97e1d3a2f64e.