build_gecko::bindings cannot find system header with compiling on Nix

REOPENED
Unassigned

Status

()

Core
Build Config
REOPENED
3 months ago
2 months ago

People

(Reporter: nbp, Unassigned)

Tracking

unspecified
Points:
---

Firefox Tracking Flags

(firefox57 wontfix)

Details

Attachments

(1 attachment)

(Reporter)

Description

3 months ago
/home/nicolas/mozilla/_build/firefox/bugzil.la/js-startup-cache/final/x64/gcc/dbg/dist/include/mozilla/Compiler.h:49:12: fatal error: 'cstddef' file not found, err: true
/home/nicolas/mozilla/_build/firefox/bugzil.la/js-startup-cache/final/x64/gcc/dbg/dist/include/mozilla/mozalloc.h:16:12: fatal error: 'new' file not found, err: true
/home/nicolas/mozilla/_build/firefox/bugzil.la/js-startup-cache/final/x64/gcc/dbg/dist/include/mozilla/mozalloc.h:16:12: fatal error: 'new' file not found, err: true

stack backtrace:
   0:       0x1a94267ec3 - std::sys::imp::backtrace::tracing::imp::unwind_backtrace::hcdf51e4c9dc54357
   1:       0x1a94264574 - std::sys_common::backtrace::_print::h9da91fd31a37d0f1
   2:       0x1a9426d5e3 - std::panicking::default_hook::{{closure}}::h46820a72bf0cb624
   3:       0x1a9426d352 - std::panicking::default_hook::h4c1ef1cc83189c8e
   4:       0x1a9426daa7 - std::panicking::rust_panic_with_hook::h99016f44bdcb8544
   5:       0x1a9426d974 - std::panicking::begin_panic_new::hae931f4b9fe56a90
   6:       0x1a9426d879 - std::panicking::begin_panic_fmt::h3b8db4524c33692e
   7:       0x1a93e44744 - build_script_build::build_gecko::bindings::write_binding_file::h9bae0af8e30c7ed9
                               at /home/nicolas/mozilla/contrib-push/servo/components/style/build_gecko.rs:291
   8:       0x1a93e4a2ab - build_script_build::build_gecko::bindings::generate_bindings::h299bb67754ac7068
                               at /home/nicolas/mozilla/contrib-push/servo/components/style/build_gecko.rs:549
   9:       0x1a94274a4c - __rust_maybe_catch_panic
  10:       0x1a93e271f3 - std::panicking::try::h69501e86c368d31c
                               at /checkout/src/libstd/panicking.rs:458
  11:       0x1a93e1638d - std::panic::catch_unwind::heddd79a6fbad9261
                               at /checkout/src/libstd/panic.rs:361
  12:       0x1a93e26b91 - std::thread::Builder::spawn::{{closure}}::h533351ffa6d6c3a2
                               at /checkout/src/libstd/thread/mod.rs:363
  13:       0x1a93e3011d - <F as alloc::boxed::FnBox<A>>::call_box::h65c0a6674ab6fc32
                               at /checkout/src/liballoc/boxed.rs:651
  14:       0x1a9426cafb - std::sys::imp::thread::Thread::new::thread_start::h10fad04495d944f7
  15:     0x7f2762ca0483 - start_thread
  16:     0x7f27623c27ae - __clone
  17:                0x0 - <unknown>
You want to run `xcode-select --install` first.
Status: NEW → RESOLVED
Last Resolved: 3 months ago
Resolution: --- → DUPLICATE
Duplicate of bug: 1395929
(Reporter)

Comment 2

3 months ago
(In reply to Nathan Froyd [:froydnj] from comment #1)
> You want to run `xcode-select --install` first.

If I were on a Mac maybe, but I am on Linux
Status: RESOLVED → REOPENED
Resolution: DUPLICATE → ---
(Reporter)

Updated

3 months ago
Flags: needinfo?(nfroyd)
You are on Linux?  What is wrong with your system that the compiler can't find includes in the usual places?
Flags: needinfo?(nfroyd)
(Reporter)

Comment 4

3 months ago
Created attachment 8903700 [details]
full error message on panic.
(Reporter)

Comment 5

3 months ago
(In reply to Nathan Froyd [:froydnj] from comment #3)
> You are on Linux?  What is wrong with your system that the compiler can't
> find includes in the usual places?

My compiler can find the headers, there is just no usual place.
They are all under one of the /nix/store/* directories.
(In reply to Nicolas B. Pierron [:nbp] from comment #5)
> (In reply to Nathan Froyd [:froydnj] from comment #3)
> > You are on Linux?  What is wrong with your system that the compiler can't
> > find includes in the usual places?
> 
> My compiler can find the headers, there is just no usual place.
> They are all under one of the /nix/store/* directories.

...

I'm assuming that you're using the clang that came in via `mach bootstrap` to compile stylo, then, correct?  (Possibly unwittingly, since configure will auto-detect the presence of the `mach bootstrap`'d clang and not really tell you.)

If that's the case, then I think you are just going to have to use a nix-aware clang instead; I doubt that modifying our clang to work on nix is going to be a worthwhile investment.  You can do this by setting LLVM_CONFIG in your mozconfig to point to an llvm-config from a nix-aware clang package.
Flags: needinfo?(nicolas.b.pierron)
Summary: build_gecko::bindings cannot find system header. → build_gecko::bindings cannot find system header with compiling on Nix
(Reporter)

Comment 7

3 months ago
(In reply to Nathan Froyd [:froydnj] from comment #6)
> (In reply to Nicolas B. Pierron [:nbp] from comment #5)
> > (In reply to Nathan Froyd [:froydnj] from comment #3)
> > > You are on Linux?  What is wrong with your system that the compiler can't
> > > find includes in the usual places?
> > 
> > My compiler can find the headers, there is just no usual place.
> > They are all under one of the /nix/store/* directories.
> 
> ...
> 
> I'm assuming that you're using the clang that came in via `mach bootstrap`
> to compile stylo, then, correct?  (Possibly unwittingly, since configure
> will auto-detect the presence of the `mach bootstrap`'d clang and not really
> tell you.)

No, `mach bootstrap` never worked on my system.  I am using the Nix expression from nixpkgs-mozilla [1] to produce an isolated environment which include libclang.

[1] https://github.com/mozilla/nixpkgs-mozilla/blob/master/pkgs/gecko/default.nix#L61

> If that's the case, then I think you are just going to have to use a
> nix-aware clang instead; I doubt that modifying our clang to work on nix is
> going to be a worthwhile investment.  You can do this by setting LLVM_CONFIG
> in your mozconfig to point to an llvm-config from a nix-aware clang package.

I do not use LLVM_CONFIG because it never worked for me, instead I used:

ac_add_options --with-libclang-path=$(dirname $libclang)
ac_add_options --with-clang-path=$clang
Flags: needinfo?(nicolas.b.pierron)
(Reporter)

Comment 8

3 months ago
From what I recall my build only broke today, and the only difference I noticed was the automatic update of rustc to 1.20.
If you are already using a nix-aware clang, then...I am not sure what else can be done.  Is there some sort of setup where the C headers and the C++ headers are not found in a common directory, because they belong to different packages?  We might need to invoke clang++ in bindgen if that's the case, because presumably plain `clang` in this case cannot find the C++ headers.   Or somehow communicate that C++ header paths need to be added, or something.
(In reply to Nicolas B. Pierron [:nbp] from comment #8)
> From what I recall my build only broke today, and the only difference I
> noticed was the automatic update of rustc to 1.20.

This is an interesting piece of information!  I am having a hard time seeing how a rustc change would affect the header search paths for bindgen, though...
(Reporter)

Comment 11

3 months ago
(In reply to Nathan Froyd [:froydnj] from comment #9)
> If you are already using a nix-aware clang, then...I am not sure what else
> can be done.  Is there some sort of setup where the C headers and the C++
> headers are not found in a common directory, because they belong to
> different packages?

Yes, we have a compiler wrapper which is used to provide the command line which instruct the compiler of the system headers location.

> We might need to invoke clang++ in bindgen if that's
> the case, because presumably plain `clang` in this case cannot find the C++
> headers. […]

Ok, if we are invoking clang++ to get the headers, and not just relying on libclang, than I can understand why the headers might not be found, as I set the default compiler to be gcc, and the clang that I provide is not aware of the system headers location.

On the other hand, I tried using the stable version of nixpkgs (17.03), with rustc 1.20, a similar mozconfg file, on the same revision of Firefox, and successfully managed to build. (clang 3.9.1 instead of clang 4.0.1)

I will re-try with the newer version if I can reproduce the issue the original issue once more, and try to bisect nixpkgs if I can reproduce this issue.

Thanks Nathan.

Updated

2 months ago
status-firefox57: --- → wontfix
You need to log in before you can comment on or make changes to this bug.