1586915 - build osclientcerts in-tree (macos)

Status: VERIFIED FIXED

(Core :: Security: PSM, enhancement, P1)

Description

[Dana Keeler (she/her) (use needinfo) [:keeler] (on leave)]

osclientcerts (currently at https://github.com/mozkeeler/osclientcerts) will provide support for Firefox to use client certificates stored in OS certificate storage. We need to be able to build it as part of the build process for MacOS (bug 1584401 is the corresponding bug for Windows).

Comment 1

[Stephen A Pohl [:spohl]]

Hi Dana, I had a look at this and it appears that we're running into the same problem that we faced in
https://phabricator.services.mozilla.com/D19721, which resulted in my comment https://phabricator.services.mozilla.com/D19721#549205. The one question I had was whether or not you had confirmed that there was indeed no Objective-C API to do the same, even if it were to throw up a dialog to end users or similar. In particular I'm thinking of SecurityFoundation, SFAuthorization or similar.

Comment 2

[Dana Keeler (she/her) (use needinfo) [:keeler] (on leave)]

Attachment: bug 1586915 - build osclientcerts in-tree for macOS r?jcj

This patch implements osclientcerts for macOS.
Because the SDK we build with isn't recent enough, some of the functions we
need aren't guaranteed to be available. To handle this, we load the Security
framework at runtime and attempt to locate the symbols we need. If this
succeeds, then operation proceeds as normal. Otherwise, the module will report
that there are no certificates/keys available.

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/28a30a7e2666
build osclientcerts in-tree for macOS r=jcj,mstange

Comment 4

[Cosmin Sabou [:CosminS]]

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&resultStatus=testfailed%2Cbusted%2Cexception&searchStr=windows%2Cmingw%2Call&tochange=d2c7a9b66822875ce9cd1ab769f8d90e25b437d9&fromchange=28a30a7e266603c7c6d657f4400860288c807115&selectedJob=288403912

Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=288403912&repo=autoland

Backout link: https://hg.mozilla.org/integration/autoland/rev/012c3f1626b3e9bcd803d19aaf9584a81c5c95de

[task 2020-02-11T21:22:41.388Z] 21:22:41 INFO - note: Function pointers must be non-null (in this struct field)
[task 2020-02-11T21:22:41.388Z] 21:22:41 INFO - --> /builds/worker/workspace/build/src/third_party/rust/clang-sys/src/lib.rs:1602:5
[task 2020-02-11T21:22:41.388Z] 21:22:41 INFO - |
[task 2020-02-11T21:22:41.388Z] 21:22:41 INFO - 1602 | pub abortQuery: extern "C" fn(CXClientData, *mut c_void) -> c_int,
[task 2020-02-11T21:22:41.388Z] 21:22:41 INFO - | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[task 2020-02-11T21:22:41.389Z] 21:22:41 INFO - Compiling bindgen v0.51.1
[task 2020-02-11T21:22:41.389Z] 21:22:41 INFO - Running CARGO_PKG_VERSION=0.51.1 CARGO_PKG_DESCRIPTION='Automatically generates Rust FFI bindings to C and C++ libraries.' CARGO_PKG_HOMEPAGE='https://rust-lang.github.io/rust-bindgen/' CARGO_PKG_VERSION_PRE= CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_NAME=bindgen CARGO_PKG_VERSION_MINOR=51 CARGO_PKG_REPOSITORY='https://github.com/rust-lang/rust-bindgen' CARGO=/builds/worker/fetches/rustc/bin/cargo LD_LIBRARY_PATH='/builds/worker/workspace/build/src/obj-firefox/debug/deps:/builds/worker/fetches/rustc/lib:/builds/worker/fetches/mingw32/lib64:/builds/worker/fetches/clang/lib' CARGO_PKG_AUTHORS='Jyun-Yan You <[e-mail]>:Emilio Cobos Álvarez <[e-mail]>:Nick Fitzgerald <[e-mail]>:The Servo project developers' OUT_DIR=/builds/worker/workspace/build/src/obj-firefox/debug/build/bindgen-779ec2fef274997b/out CARGO_PKG_VERSION_PATCH=1 CARGO_MANIFEST_DIR=/builds/worker/workspace/build/src/third_party/rust/bindgen /builds/worker/fetches/sccache/sccache /builds/worker/fetches/rustc/bin/rustc --crate-name bindgen /builds/worker/workspace/build/src/third_party/rust/bindgen/src/lib.rs --color never --crate-type lib --emit=dep-info,metadata,link -C opt-level=1 -C debuginfo=2 -C debug-assertions=on -C metadata=e6b7c9f30d83111e -C extra-filename=-e6b7c9f30d83111e --out-dir /builds/worker/workspace/build/src/obj-firefox/debug/deps -C linker=/builds/worker/workspace/build/src/build/cargo-host-linker -L dependency=/builds/worker/workspace/build/src/obj-firefox/debug/deps --extern bitflags=/builds/worker/workspace/build/src/obj-firefox/debug/deps/libbitflags-caf210eb60cb5209.rmeta --extern cexpr=/builds/worker/workspace/build/src/obj-firefox/debug/deps/libcexpr-073aec5cb679b44d.rmeta --extern cfg_if=/builds/worker/workspace/build/src/obj-firefox/debug/deps/libcfg_if-affc25c675703e0b.rmeta --extern clang_sys=/builds/worker/workspace/build/src/obj-firefox/debug/deps/libclang_sys-bc7fcd28120905c5.rmeta --extern lazy_static=/builds/worker/workspace/build/src/obj-firefox/debug/deps/liblazy_static-5915516f32eeea53.rmeta --extern peeking_take_while=/builds/worker/workspace/build/src/obj-firefox/debug/deps/libpeeking_take_while-63e97fce878f37fa.rmeta --extern proc_macro2=/builds/worker/workspace/build/src/obj-firefox/debug/deps/libproc_macro2-ad2d52aeb9a6e356.rmeta --extern quote=/builds/worker/workspace/build/src/obj-firefox/debug/deps/libquote-a4cf4fb65c56da69.rmeta --extern regex=/builds/worker/workspace/build/src/obj-firefox/debug/deps/libregex-2ec8145a1beb984b.rmeta --extern rustc_hash=/builds/worker/workspace/build/src/obj-firefox/debug/deps/librustc_hash-5d1c74a1ae32c0ca.rmeta --extern shlex=/builds/worker/workspace/build/src/obj-firefox/debug/deps/libshlex-1c831afe12c94ef3.rmeta --cap-lints warn -L native=/builds/worker/workspace/build/src/obj-firefox/debug/build/libloading-5eee4ffd9a6de817/out
[task 2020-02-11T21:22:41.389Z] 21:22:41 INFO - Compiling osclientcerts-static v0.1.4 (/builds/worker/workspace/build/src/security/manager/ssl/osclientcerts)
[task 2020-02-11T21:22:41.389Z] 21:22:41 INFO - Running CARGO_PKG_VERSION=0.1.4 CARGO_PKG_DESCRIPTION='Platform-specific support for client authentication certificates in Firefox' CARGO_PKG_HOMEPAGE= CARGO_PKG_VERSION_PRE= CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_NAME=osclientcerts-static CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_REPOSITORY='https://github.com/mozkeeler/osclientcerts' CARGO=/builds/worker/fetches/rustc/bin/cargo LD_LIBRARY_PATH='/builds/worker/workspace/build/src/obj-firefox/debug/deps:/builds/worker/fetches/rustc/lib:/builds/worker/fetches/mingw32/lib64:/builds/worker/fetches/clang/lib' CARGO_PKG_AUTHORS='Dana Keeler <[e-mail]>' CARGO_PKG_VERSION_PATCH=4 CARGO_MANIFEST_DIR=/builds/worker/workspace/build/src/security/manager/ssl/osclientcerts /builds/worker/fetches/sccache/sccache /builds/worker/fetches/rustc/bin/rustc --edition=2018 --crate-name build_script_build security/manager/ssl/osclientcerts/build.rs --color never --crate-type bin --emit=dep-info,link -C opt-level=1 -C debuginfo=2 -C debug-assertions=on -C metadata=8a5ae4e72ead2ebb -C extra-filename=-8a5ae4e72ead2ebb --out-dir /builds/worker/workspace/build/src/obj-firefox/debug/build/osclientcerts-static-8a5ae4e72ead2ebb -C linker=/builds/worker/workspace/build/src/build/cargo-host-linker -L dependency=/builds/worker/workspace/build/src/obj-firefox/debug/deps --extern bindgen=/builds/worker/workspace/build/src/obj-firefox/debug/deps/libbindgen-e6b7c9f30d83111e.rlib -L native=/builds/worker/workspace/build/src/obj-firefox/debug/build/libloading-5eee4ffd9a6de817/out
[task 2020-02-11T21:22:41.390Z] 21:22:41 INFO - Running /builds/worker/workspace/build/src/obj-firefox/debug/build/osclientcerts-static-8a5ae4e72ead2ebb/build-script-build
[task 2020-02-11T21:22:41.390Z] 21:22:41 INFO - Running CARGO_PKG_VERSION=0.1.4 CARGO_PKG_DESCRIPTION='Platform-specific support for client authentication certificates in Firefox' CARGO_PKG_HOMEPAGE= CARGO_PKG_VERSION_PRE= CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_NAME=osclientcerts-static CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_REPOSITORY='https://github.com/mozkeeler/osclientcerts' CARGO=/builds/worker/fetches/rustc/bin/cargo LD_LIBRARY_PATH='/builds/worker/workspace/build/src/obj-firefox/debug/deps:/builds/worker/fetches/rustc/lib:/builds/worker/fetches/mingw32/lib64:/builds/worker/fetches/clang/lib' CARGO_PKG_AUTHORS='Dana Keeler <[e-mail]>' OUT_DIR=/builds/worker/workspace/build/src/obj-firefox/i686-pc-windows-gnu/debug/build/osclientcerts-static-dcf0894aa3c24ef6/out CARGO_PKG_VERSION_PATCH=4 CARGO_MANIFEST_DIR=/builds/worker/workspace/build/src/security/manager/ssl/osclientcerts /builds/worker/fetches/sccache/sccache /builds/worker/fetches/rustc/bin/rustc --edition=2018 --crate-name osclientcerts_static security/manager/ssl/osclientcerts/src/lib.rs --color never --crate-type staticlib --emit=dep-info,link -C opt-level=1 -C panic=abort -C debuginfo=2 -C debug-assertions=on -C metadata=c9c89da6e130fd57 -C extra-filename=-c9c89da6e130fd57 --out-dir /builds/worker/workspace/build/src/obj-firefox/i686-pc-windows-gnu/debug/deps --target i686-pc-windows-gnu -C linker=/builds/worker/workspace/build/src/build/cargo-linker -L dependency=/builds/worker/workspace/build/src/obj-firefox/i686-pc-windows-gnu/debug/deps -L dependency=/builds/worker/workspace/build/src/obj-firefox/debug/deps --extern byteorder=/builds/worker/workspace/build/src/obj-firefox/i686-pc-windows-gnu/debug/deps/libbyteorder-114d8244af376b61.rlib --extern env_logger=/builds/worker/workspace/build/src/obj-firefox/i686-pc-windows-gnu/debug/deps/libenv_logger-686887b70e82a39a.rlib --extern lazy_static=/builds/worker/workspace/build/src/obj-firefox/i686-pc-windows-gnu/debug/deps/liblazy_static-3794392dc2077794.rlib --extern log=/builds/worker/workspace/build/src/obj-firefox/i686-pc-windows-gnu/debug/deps/liblog-645964c3adac50c1.rlib --extern pkcs11=/builds/worker/workspace/build/src/obj-firefox/i686-pc-windows-gnu/debug/deps/libpkcs11-960ad69ab3e0e497.rlib --extern sha2=/builds/worker/workspace/build/src/obj-firefox/i686-pc-windows-gnu/debug/deps/libsha2-675470ce479f9eba.rlib --extern winapi=/builds/worker/workspace/build/src/obj-firefox/i686-pc-windows-gnu/debug/deps/libwinapi-bb3cdab82b2b44a9.rlib -C opt-level=2 -C debuginfo=2 -C force-frame-pointers=yes --cap-lints warn -L native=/builds/worker/workspace/build/src/third_party/rust/winapi-i686-pc-windows-gnu/lib
[task 2020-02-11T21:22:41.390Z] 21:22:41 INFO - error: couldn't read /builds/worker/workspace/build/src/obj-firefox/i686-pc-windows-gnu/debug/build/osclientcerts-static-dcf0894aa3c24ef6/out/bindings.rs: No such file or directory (os error 2)
[task 2020-02-11T21:22:41.390Z] 21:22:41 INFO - --> security/manager/ssl/osclientcerts/src/backend_windows.rs:7:1
[task 2020-02-11T21:22:41.390Z] 21:22:41 INFO - |
[task 2020-02-11T21:22:41.391Z] 21:22:41 INFO - 7 | include!(concat!(env!("OUT_DIR"), "/bindings.rs"));
[task 2020-02-11T21:22:41.391Z] 21:22:41 INFO - | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[task 2020-02-11T21:22:41.391Z] 21:22:41 ERROR - error: aborting due to previous error
[task 2020-02-11T21:22:41.391Z] 21:22:41 INFO - error: could not compile osclientcerts-static.
[task 2020-02-11T21:22:41.391Z] 21:22:41 INFO - Caused by:
[task 2020-02-11T21:22:41.392Z] 21:22:41 INFO - process didn't exit successfully: CARGO_PKG_VERSION=0.1.4 CARGO_PKG_DESCRIPTION='Platform-specific support for client authentication certificates in Firefox' CARGO_PKG_HOMEPAGE= CARGO_PKG_VERSION_PRE= CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_NAME=osclientcerts-static CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_REPOSITORY='https://github.com/mozkeeler/osclientcerts' CARGO=/builds/worker/fetches/rustc/bin/cargo LD_LIBRARY_PATH='/builds/worker/workspace/build/src/obj-firefox/debug/deps:/builds/worker/fetches/rustc/lib:/builds/worker/fetches/mingw32/lib64:/builds/worker/fetches/clang/lib' CARGO_PKG_AUTHORS='Dana Keeler <[e-mail]>' OUT_DIR=/builds/worker/workspace/build/src/obj-firefox/i686-pc-windows-gnu/debug/build/osclientcerts-static-dcf0894aa3c24ef6/out CARGO_PKG_VERSION_PATCH=4 CARGO_MANIFEST_DIR=/builds/worker/workspace/build/src/security/manager/ssl/osclientcerts /builds/worker/fetches/sccache/sccache /builds/worker/fetches/rustc/bin/rustc --edition=2018 --crate-name osclientcerts_static security/manager/ssl/osclientcerts/src/lib.rs --color never --crate-type staticlib --emit=dep-info,link -C opt-level=1 -C panic=abort -C debuginfo=2 -C debug-assertions=on -C metadata=c9c89da6e130fd57 -C extra-filename=-c9c89da6e130fd57 --out-dir /builds/worker/workspace/build/src/obj-firefox/i686-pc-windows-gnu/debug/deps --target i686-pc-windows-gnu -C linker=/builds/worker/workspace/build/src/build/cargo-linker -L dependency=/builds/worker/workspace/build/src/obj-firefox/i686-pc-windows-gnu/debug/deps -L dependency=/builds/worker/workspace/build/src/obj-firefox/debug/deps --extern byteorder=/builds/worker/workspace/build/src/obj-firefox/i686-pc-windows-gnu/debug/deps/libbyteorder-114d8244af376b61.rlib --extern env_logger=/builds/worker/workspace/build/src/obj-firefox/i686-pc-windows-gnu/debug/deps/libenv_logger-686887b70e82a39a.rlib --extern lazy_static=/builds/worker/workspace/build/src/obj-firefox/i686-pc-windows-gnu/debug/deps/liblazy_static-3794392dc2077794.rlib --extern log=/builds/worker/workspace/build/src/obj-firefox/i686-pc-windows-gnu/debug/deps/liblog-645964c3adac50c1.rlib --extern pkcs11=/builds/worker/workspace/build/src/obj-firefox/i686-pc-windows-gnu/debug/deps/libpkcs11-960ad69ab3e0e497.rlib --extern sha2=/builds/worker/workspace/build/src/obj-firefox/i686-pc-windows-gnu/debug/deps/libsha2-675470ce479f9eba.rlib --extern winapi=/builds/worker/workspace/build/src/obj-firefox/i686-pc-windows-gnu/debug/deps/libwinapi-bb3cdab82b2b44a9.rlib -C opt-level=2 -C debuginfo=2 -C force-frame-pointers=yes --cap-lints warn -L native=/builds/worker/workspace/build/src/third_party/rust/winapi-i686-pc-windows-gnu/lib (exit code: 1)
[task 2020-02-11T21:22:41.392Z] 21:22:41 INFO - /builds/worker/workspace/build/src/config/makefiles/rust.mk:277: recipe for target 'force-cargo-library-build' failed
[task 2020-02-11T21:22:41.392Z] 21:22:41 ERROR - make[4]: *** [force-cargo-library-build] Error 101
[task 2020-02-11T21:22:41.392Z] 21:22:41 INFO - make[4]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/security/manager/ssl/osclientcerts'
[task 2020-02-11T21:22:41.392Z] 21:22:41 INFO - make[4]: Entering directory '/builds/worker/workspace/build/src/obj-firefox/dom/bindings'
[task 2020-02-11T21:22:41.393Z] 21:22:41 INFO - /builds/worker/workspace/build/src/config/recurse.mk:74: recipe for target 'security/manager/ssl/osclientcerts/target' failed
[task 2020-02-11T21:22:41.393Z] 21:22:41 ERROR - make[3]: *** [security/manager/ssl/osclientcerts/target] Error 2
[task 2020-02-11T21:22:41.393Z] 21:22:41 INFO - make[3]: *** Waiting for unfinished jobs....

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/69fb848f9514
build osclientcerts in-tree for macOS r=jcj,mstange

Comment 6

[Razvan Maries]

https://hg.mozilla.org/mozilla-central/rev/69fb848f9514

Comment 7

[Julien Cristau [:jcristau] (back April 22)]

Adding this to the draft 75beta release notes:

Release Note Request (optional, but appreciated)
[Why is this notable]: we relnoted this in 72 for Windows
[Affects Firefox for Android]:
[Suggested wording]: Experimental support for using client certificates from the OS certificate store can be enabled on macOS by setting the preference security.osclientcerts.autoload to true (this feature has been available on Windows since Firefox 72).
[Links (documentation, blog post, etc)]:

Comment 10

[Miruna Curtean]

Verified on macOS 10.14 and 10.15 across Beta 75.0b6 (20200319224147) and Nightly 76.0a1 (20200319215651) and verified that the Ec and RSA client certificates can be successfully loaded using TLS 1.2 or TLS 1.3 servers.