1604667 - mozbuild: Don't use relativesrcdir on Windows

Status: RESOLVED FIXED

(Firefox Build System :: General, enhancement, P3)

Description

[Marc Streckfuß [:MeFisto94]]

This bug is a real corner-case and as it looks relativesrcdir's are/used to be present in many Makefiles and thus it might be difficult to support, if possible at all.
If not, maybe this issue at least helps others with that issue:

The case: Building Servo on Windows (Mozilla-Build) fails when trying to compile mozjs_sys (Spidermonkey), because servo's source code is hosted on Drive E:\ (in my case) and CARGO_HOME is set on Drive C:\ (~/.cargo), which leads to:

Traceback (most recent call last):
  File "c:/Users/Marc/.cargo/git/checkouts/mozjs-fa11ffc7d4f1cc2d/5906588/mozjs/js/src/../../configure.py", line 132, in <module>
    sys.exit(main(sys.argv))
  File "c:/Users/Marc/.cargo/git/checkouts/mozjs-fa11ffc7d4f1cc2d/5906588/mozjs/js/src/../../configure.py", line 43, in main
    return config_status(config)
  File "c:/Users/Marc/.cargo/git/checkouts/mozjs-fa11ffc7d4f1cc2d/5906588/mozjs/js/src/../../configure.py", line 127, in config_status
    return config_status(args=[], **encode(sanitized_config, encoding))
  File "c:\Users\Marc\.cargo\git\checkouts\mozjs-fa11ffc7d4f1cc2d\5906588\mozjs\python\mozbuild\mozbuild\config_status.py", line 146, in config_status
    the_backend.consume(definitions)
  File "c:\Users\Marc\.cargo\git\checkouts\mozjs-fa11ffc7d4f1cc2d\5906588\mozjs\python\mozbuild\mozbuild\backend\base.py", line 128, in consume
    if (not self.consume_object(obj) and
  File "c:\Users\Marc\.cargo\git\checkouts\mozjs-fa11ffc7d4f1cc2d\5906588\mozjs\python\mozbuild\mozbuild\backend\recursivemake.py", line 438, in consume_object
    consumed = CommonBackend.consume_object(self, obj)
  File "c:\Users\Marc\.cargo\git\checkouts\mozjs-fa11ffc7d4f1cc2d\5906588\mozjs\python\mozbuild\mozbuild\backend\common.py", line 134, in consume_object
    with self._get_preprocessor(obj) as pp:
  File "e:\mozilla-build\python\Lib\contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "c:\Users\Marc\.cargo\git\checkouts\mozjs-fa11ffc7d4f1cc2d\5906588\mozjs\python\mozbuild\mozbuild\backend\base.py", line 320, in _get_preprocessor
    srcdir_rel=mozpath.relpath(srcdir, mozpath.dirname(obj.output_path)),
  File "c:\Users\Marc\.cargo\git\checkouts\mozjs-fa11ffc7d4f1cc2d\5906588\mozjs\python\mozbuild\mozpack\path.py", line 33, in relpath
    rel = normsep(os.path.relpath(path, start))
  File "e:\Marc\Documents\Coding\servo\target\debug\build\mozjs_sys-88004c1deb65babf\out\_virtualenvs\init\lib\ntpath.py", line 529, in relpath
    % (path_prefix, start_prefix))
ValueError: path is on drive c:, start on drive e:

So the obvious workaround is to have servo on C:.
I am uncertain how easy this could be fixed. I guess this rare occasion didn't happen before as on Unix/Mac Systems, there are no real "drive boundaries" as on Windows, but for Windows there can't be relative paths over drive boundaries.
In addition to that, relativesrcpath is usually for source-code located inside the topsrcdir anyway.

BUT:
Is there a real reason to use relative paths here? If so, what is/was the intent there?
Especially since in the given situation, the absolute path is added as well:

pp.context.update(
            top_srcdir=obj.topsrcdir,
            topobjdir=obj.topobjdir,
            srcdir=srcdir,
            srcdir_rel=mozpath.relpath(srcdir, mozpath.dirname(obj.output_path)),
            relativesrcdir=mozpath.relpath(srcdir, obj.topsrcdir) or '.',
            DEPTH=mozpath.relpath(obj.topobjdir, mozpath.dirname(obj.output_path)) or '.',
        )

I guess the Preprocessor isn't the big problem here anyway, but the Makefiles that are built are.

As I said, the easy workaround is to move the servo folder to C, maybe even symlinking is enough. Maybe I can also figure out a way for mozjs_sys to just copy the contents into a servo folder.

Comment 1

[Ricky Stewart]

Attachment: Bug 1604667 - Handle case where relativesrcdir doesn't exist on Windows

Comment 2

[Ricky Stewart]

Marc, can you patch this revision and see whether it fixes your problem? I can't easily reproduce this on my own machine.

Comment 3

[Marc Streckfuß [:MeFisto94]]

Sorry for the late reply, I also had to get the build to reproduce locally as well.
It compiles for me, so it has indeed solved the problem.
Note that I had to apply the patch manually as the version of mozjs I had locally was different from your version, but I guess this is just because servo is using an older version [and my checkout of servo is also a month old, as well].

Thanks for the fix!

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/319b53c8e372
Handle case where relativesrcdir doesn't exist on Windows r=firefox-build-system-reviewers,mshal

Comment 5

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/319b53c8e372