1420301 - stylo: bindgen panicked at 'function not loaded: clang_createIndex'

Status: RESOLVED FIXED

(Core :: CSS Parsing and Computation, defect)

Description

[Ralph Giles (:rillian)]

Stylo debug builds fail with rust 1.23.0-beta.1 on win32:

thread '<unnamed>' panicked at 'function not loaded: clang_createIndex', Z:\build\build\src\third_party\rust\clang-sys\src\lib.rs:1455:0
22:39:56     INFO -  stack backtrace:
22:39:56     INFO -     0:     0x7ff6b3542be3 - std::sys_common::backtrace::_print
22:39:56     INFO -                                 at C:\projects\rust\src\libstd\sys_common\backtrace.rs:91
22:39:56     INFO -     1:     0x7ff6b354f543 - std::panicking::default_hook::{{closure}}
22:39:56     INFO -                                 at C:\projects\rust\src\libstd\panicking.rs:383
22:39:56     INFO -     2:     0x7ff6b354f1b5 - std::panicking::default_hook
22:39:56     INFO -                                 at C:\projects\rust\src\libstd\panicking.rs:397
22:39:56     INFO -     3:     0x7ff6b354fb15 - std::panicking::rust_panic_with_hook
22:39:56     INFO -                                 at C:\projects\rust\src\libstd\panicking.rs:577
22:39:56     INFO -     4:     0x7ff6b343d328 - std::panicking::begin_panic<str*>
22:39:56     INFO -                                 at C:\projects\rust\src\libstd\panicking.rs:538
22:39:56     INFO -     5:     0x7ff6b3461748 - clang_sys::clang_createIndex::{{closure}}
22:39:56     INFO -                                 at Z:\build\build\src\third_party\rust\clang-sys\src\link.rs:92
22:39:56     INFO -     6:     0x7ff6b3460f05 - clang_sys::with_library::{{closure}}<unsafe extern "C" fn(i32, i32) -> mut libc::c_void*,closure>
22:39:56     INFO -                                 at Z:\build\build\src\third_party\rust\clang-sys\src\link.rs:79
22:39:56     INFO -     7:     0x7ff6b345a96d - std::thread::local::LocalKey<core::cell::RefCell<core::option::Option<alloc::arc::Arc<clang_sys::SharedLibrary>>>>::try_with<core::cell::RefCell<core::option::Option<alloc::arc::Arc<clang_sys::SharedLibrary>>>,closure,core::option::Option<unsafe extern "C" fn(i32, i32) -> mut libc::c_void*>>
22:39:56     INFO -                                 at C:\projects\rust\src\libstd\thread\local.rs:377
22:39:56     INFO -     8:     0x7ff6b3459e8c - std::thread::local::LocalKey<core::cell::RefCell<core::option::Option<alloc::arc::Arc<clang_sys::SharedLibrary>>>>::with<core::cell::RefCell<core::option::Option<alloc::arc::Arc<clang_sys::SharedLibrary>>>,closure,core::option::Option<unsafe extern "C" fn(i32, i32) -> mut libc::c_void*>>
22:39:56     INFO -                                 at C:\projects\rust\src\libstd\thread\local.rs:288
22:39:56     INFO -     9:     0x7ff6b344a234 - clang_sys::clang_createIndex
22:39:56     INFO -                                 at Z:\build\build\src\third_party\rust\clang-sys\src\link.rs:89
22:39:56     INFO -    10:     0x7ff6b338b874 - bindgen::ir::context::BindgenContext::new
22:39:56     INFO -                                 at Z:\build\build\src\third_party\rust\bindgen\src\ir\context.rs:541
22:39:56     INFO -    11:     0x7ff6b339bd7e - bindgen::Bindings::generate
22:39:56     INFO -                                 at Z:\build\build\src\third_party\rust\bindgen\src\lib.rs:1627
22:39:56     INFO -    12:     0x7ff6b339aa05 - bindgen::Builder::generate
22:39:56     INFO -                                 at Z:\build\build\src\third_party\rust\bindgen\src\lib.rs:1097
22:39:56     INFO -    13:     0x7ff6b3352886 - build_script_build::build_gecko::bindings::write_binding_file
22:39:56     INFO -                                 at Z:\build\build\src\servo\components\style\build_gecko.rs:279
22:39:56     INFO -    14:     0x7ff6b335a2c3 - build_script_build::build_gecko::bindings::generate_bindings
22:39:56     INFO -                                 at Z:\build\build\src\servo\components\style\build_gecko.rs:546
22:39:56     INFO -    15:     0x7ff6b3550511 - panic_unwind::__rust_maybe_catch_panic
22:39:56     INFO -                                 at C:\projects\rust\src\libpanic_unwind\lib.rs:101
22:39:56     INFO -    16:     0x7ff6b3331e94 - std::panicking::try<(),std::panic::AssertUnwindSafe<closure>>
22:39:56     INFO -                                 at C:\projects\rust\src\libstd\panicking.rs:459
22:39:56     INFO -    17:     0x7ff6b33453f2 - std::panic::catch_unwind<std::panic::AssertUnwindSafe<closure>,()>
22:39:56     INFO -                                 at C:\projects\rust\src\libstd\panic.rs:365
22:39:56     INFO -    18:     0x7ff6b3345fcd - std::thread::{{impl}}::spawn::{{closure}}<closure,()>
22:39:56     INFO -                                 at C:\projects\rust\src\libstd\thread\mod.rs:405
22:39:56     INFO -    19:     0x7ff6b334cb74 - alloc::boxed::{{impl}}::call_box<(),closure>
22:39:56     INFO -                                 at C:\projects\rust\src\liballoc\boxed.rs:772
22:39:56     INFO -    20:     0x7ff6b354e3ab - std::sys::imp::thread::{{impl}}::new::thread_start
22:39:56     INFO -                                 at C:\projects\rust\src\libstd\sys\windows\thread.rs:54
22:39:56     INFO -    21:     0x7ff81d8913d1 - BaseThreadInitThunk

Comment 1

[Emilio Cobos Álvarez (:emilio)]

What version of libclang is this using? I don't think clang_createIndex should be missing from any libclang ever... This means that we failed to dynamically load the symbol, which looks suspicious...

Also, is this really only on debug? I'd expect all builds w/ the same setup to be affected.

Comment 2

[Emilio Cobos Álvarez (:emilio)]

Hmm, I've been able to repro. It's not a panic, but a segfault on my machine... I'm trying to figure out what's going on. I've verified that the issue is with clang_sys runtime-loading support, if I load libclang statically, it builds just fine.

Comment 3

[Emilio Cobos Álvarez (:emilio)]

Fascinating... If I modify clang-sys the following way:

diff --git a/src/link.rs b/src/link.rs
index 543a5a5..0871d6a 100644
--- a/src/link.rs
+++ b/src/link.rs
@@ -24,6 +24,7 @@ macro_rules! link {
         #[cfg($cfg)]
         pub fn $name(library: &mut super::SharedLibrary) {
             let symbol = unsafe { library.library.get(stringify!($name).as_bytes()) }.ok();
+            println!("Loaded {}: {:?}", stringify!($name), symbol);
             library.functions.$name = symbol.map(|s| *s);
         }
 
@@ -92,6 +93,7 @@ macro_rules! link {
                         _ => panic!(concat!("function not loaded: ", stringify!($name))),
                     }
                 }).expect("a `libclang` shared library is not loaded on this thread");
+                println!(concat!("Calling ", stringify!($name), ": {:?}"), f);
                 f($($pname), *)
             }
 
@@ -125,6 +127,7 @@ macro_rules! link {
                 format!("the `libclang` shared library could not be opened: {}", file.display())
             });
             let mut library = SharedLibrary::new(try!(library));
+            println!("Loaded library at {}", file.display());
             $(load::$name(&mut library);)+
             Ok(library)
         }

I suddenly get a build. Which makes absolutely no sense.

Comment 4

[Emilio Cobos Álvarez (:emilio)]

Only the first one is required, actually. The others make no difference... 

Furthermore:

diff --git a/src/link.rs b/src/link.rs
index 543a5a5..685b408 100644
--- a/src/link.rs
+++ b/src/link.rs
@@ -24,6 +24,7 @@ macro_rules! link {
         #[cfg($cfg)]
         pub fn $name(library: &mut super::SharedLibrary) {
             let symbol = unsafe { library.library.get(stringify!($name).as_bytes()) }.ok();
+            println!("foo");
             library.functions.$name = symbol.map(|s| *s);
         }

Makes the build still fail.

While:

diff --git a/src/link.rs b/src/link.rs
index 543a5a5..685b408 100644
--- a/src/link.rs
+++ b/src/link.rs
@@ -24,6 +24,7 @@ macro_rules! link {
         #[cfg($cfg)]
         pub fn $name(library: &mut super::SharedLibrary) {
             let symbol = unsafe { library.library.get(stringify!($name).as_bytes()) }.ok();
+            println!("foo: {:?}", symbol);
             library.functions.$name = symbol.map(|s| *s);
         }

Fixes it.

Comment 5

[Emilio Cobos Álvarez (:emilio)]

It looks like clang-sys is relying on type inference to work in a particular way...

Comment 6

[Emilio Cobos Álvarez (:emilio)]

https://github.com/KyleMayes/clang-sys/pull/69 fixes it for me

Comment 7

[Emilio Cobos Álvarez (:emilio)]

I've talked with Simon because I couldn't reason about my theory of type inference, and it does seem like a miscompilation bug in rustc :/.

Comment 8

[Emilio Cobos Álvarez (:emilio)]

Filed https://github.com/rust-lang/rust/issues/46239

Comment 9

[Emilio Cobos Álvarez (:emilio)]

Attachment: Bug 1420301: Update clang-sys to work around a compiler bug.

See: https://github.com/rust-lang/rust/issues/46239

Review commit: https://reviewboard.mozilla.org/r/202936/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/202936/

Comment 10

[Ralph Giles (:rillian)]

I looks like it's libclang built from svn r311608 plus some backport patches. https://searchfox.org/mozilla-central/source/build/build-clang/clang-win32.json

We use the same version for 64-bit windows, which one additional patch.

Anyway, glad you could reproduce and track down the issue so quickly. I also see a segfault on my Linux machine. Hopefully we can work around in clang-sys and/or get a fix into the next rustc beta.

Comment 11

[Ralph Giles (:rillian)]

Comment on attachment 8931812 [details]
Bug 1420301: Update clang-sys to work around a compiler bug.

https://reviewboard.mozilla.org/r/202936/#review208276

Comment 12

[Mozilla Autoland]

hg error in cmd: hg push -r tip ssh://hg.mozilla.org/integration/autoland: pushing to ssh://hg.mozilla.org/integration/autoland
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 12 changes to 12 files
remote: (cbd7ca1ed61c modifies servo/Cargo.lock from Servo; not enforcing peer review)
remote: (1 changesets contain changes to protected servo/ directory: cbd7ca1ed61c)
remote: ************************************************************************
remote: you do not have permissions to modify files under servo/
remote: 
remote: the servo/ directory is kept in sync with the canonical upstream
remote: repository at https://github.com/servo/servo
remote: 
remote: changes to servo/ are only allowed by the syncing tool and by sheriffs
remote: performing cross-repository "merges"
remote: 
remote: to make changes to servo/, submit a Pull Request against the servo/servo
remote: GitHub project
remote: ************************************************************************
remote: transaction abort!
remote: rollback completed
remote: pretxnchangegroup.e_prevent_vendored hook failed
abort: push failed on remote

Comment 13

[Ralph Giles (:rillian)]

Oops, looks like I'm not allowed to land this. Can you push it, please?

Comment 14

[Emilio Cobos Álvarez (:emilio)]

Comment on attachment 8931812 [details]
Bug 1420301: Update clang-sys to work around a compiler bug.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/202936/diff/1-2/

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/cb600422ac4d
Update clang-sys to work around a compiler bug. r=rillian

Comment 16

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/cb600422ac4d

Comment 17

[Ryan VanderMeulen [:RyanVM]]

I'm wondering if we should backport this to Beta as well. By the time 58 goes to release, Rust 1.23 will probably be officially released.

Comment 18

[Emilio Cobos Álvarez (:emilio)]

They will probably uplift to beta the fix in https://github.com/rust-lang/rust/pull/46253.

Comment 19

[Gerry Chang [:gchang]]

Too late for Beta58. Mark 58 won't fix.