1603860 - Missing -std=gnu++17 with custom export CXX=...

Status: RESOLVED WONTFIX

(Firefox Build System :: General, defect)

Description

[Rico Tzschichholz]

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:72.0) Gecko/20100101 Firefox/72.0

Steps to reproduce:

Compiling on Ubuntu with a clean mozilla-central checkout using clang 9 and gcc 9.2.1

.mozconfig:

ac_add_options --disable-updater
mk_add_options "export CC=clang-9"
mk_add_options "export CXX=clang++-9"

$ ./mach build -v

Actual results:

 1:26.93 clang++-9 -o Unified_cpp_js_src_debugger0.o -c  -I/holding/mozilla-central/obj-x86_64-pc-linux-gnu/dist/system_wrappers -include /holding/mozilla-central/config/gcc_hidden.h -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -DNDEBUG=1 -DTRIMMED=1 -DWASM_SUPPORTS_HUGE_MEMORY -DJS_CACHEIR_SPEW -DJS_STRUCTURED_SPEW -DJS_HAS_CTYPES -DFFI_BUILDING -DEXPORT_JS_API -DMOZ_HAS_MOZGLUE -I/holding/mozilla-central/js/src/debugger -I/holding/mozilla-central/obj-x86_64-pc-linux-gnu/js/src/debugger -I/holding/mozilla-central/obj-x86_64-pc-linux-gnu/js/src -I/holding/mozilla-central/js/src -I/holding/mozilla-central/obj-x86_64-pc-linux-gnu/dist/include -I/holding/mozilla-central/obj-x86_64-pc-linux-gnu/dist/include/nspr -fPIC -DMOZILLA_CLIENT -include /holding/mozilla-central/obj-x86_64-pc-linux-gnu/js/src/js-confdefs.h -Qunused-arguments -Qunused-arguments -Wall -Wbitfield-enum-conversion -Wempty-body -Wignored-qualifiers -Woverloaded-virtual -Wpointer-arith -Wshadow-field-in-constructor-modified -Wsign-compare -Wtype-limits -Wunreachable-code -Wunreachable-code-return -Wwrite-strings -Wno-invalid-offsetof -Wclass-varargs -Wfloat-overflow-conversion -Wfloat-zero-conversion -Wloop-analysis -Wc++2a-compat -Wcomma -Wimplicit-fallthrough -Wstring-conversion -Wtautological-overlap-compare -Wtautological-unsigned-enum-zero-compare -Wtautological-unsigned-zero-compare -Wno-error=tautological-type-limit-compare -Wno-inline-new-delete -Wno-error=deprecated-declarations -Wno-error=array-bounds -Wno-error=backend-plugin -Wno-error=return-std-move -Wno-error=atomic-alignment -Wformat -Wformat-security -Wno-gnu-zero-variadic-macro-arguments -Wno-noexcept-type -Wno-unknown-warning-option -Wno-return-type-c-linkage -fno-sized-deallocation -fno-aligned-new -fno-rtti -fno-exceptions -fno-math-errno -pthread -pipe -g -O3 -fno-omit-frame-pointer -funwind-tables -fno-strict-aliasing -Werror=format -Wno-shadow  -MD -MP -MF .deps/Unified_cpp_js_src_debugger0.o.pp  -fcolor-diagnostics  Unified_cpp_js_src_debugger0.cpp
 1:27.71 In file included from Unified_cpp_js_src_debugger0.cpp:2:
 1:27.71 In file included from /holding/mozilla-central/js/src/debugger/DebugScript.cpp:7:
 1:27.71 In file included from /holding/mozilla-central/js/src/debugger/DebugScript.h:14:
 1:27.71 In file included from /holding/mozilla-central/js/src/jsapi.h:18:
 1:27.71 /holding/mozilla-central/obj-x86_64-pc-linux-gnu/dist/include/mozilla/RefPtr.h:116:46: error: no template named 'is_convertible_v' in namespace 'std'; did you mean 'is_convertible'?
 1:27.71             typename = std::enable_if_t<std::is_convertible_v<I*, T*>>>
 1:27.71                                         ~~~~~^~~~~~~~~~~~~~~~
 1:27.71                                              is_convertible
 1:27.71 /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/type_traits:1335:12: note: 'is_convertible' declared here
 1:27.71     struct is_convertible
 1:27.71            ^

Expected results:

Dropping the custom exports in .mozconfig results in the expected inclusion of -std=gnu++17:

 1:20.86 /usr/bin/clang++ -std=gnu++17 -o Unified_cpp_js_src_debugger0.o -c  -I/holding/mozilla-central/obj-x86_64-pc-linux-gnu/dist/system_wrappers -include /holding/mozilla-central/config/gcc_hidden.h -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -DNDEBUG=1 -DTRIMMED=1 -DWASM_SUPPORTS_HUGE_MEMORY -DJS_CACHEIR_SPEW -DJS_STRUCTURED_SPEW -DJS_HAS_CTYPES -DFFI_BUILDING -DEXPORT_JS_API -DMOZ_HAS_MOZGLUE -I/holding/mozilla-central/js/src/debugger -I/holding/mozilla-central/obj-x86_64-pc-linux-gnu/js/src/debugger -I/holding/mozilla-central/obj-x86_64-pc-linux-gnu/js/src -I/holding/mozilla-central/js/src -I/holding/mozilla-central/obj-x86_64-pc-linux-gnu/dist/include -I/holding/mozilla-central/obj-x86_64-pc-linux-gnu/dist/include/nspr -fPIC -DMOZILLA_CLIENT -include /holding/mozilla-central/obj-x86_64-pc-linux-gnu/js/src/js-confdefs.h -Qunused-arguments -Qunused-arguments -Wall -Wbitfield-enum-conversion -Wempty-body -Wignored-qualifiers -Woverloaded-virtual -Wpointer-arith -Wshadow-field-in-constructor-modified -Wsign-compare -Wtype-limits -Wunreachable-code -Wunreachable-code-return -Wwrite-strings -Wno-invalid-offsetof -Wclass-varargs -Wfloat-overflow-conversion -Wfloat-zero-conversion -Wloop-analysis -Wc++2a-compat -Wcomma -Wimplicit-fallthrough -Wstring-conversion -Wtautological-overlap-compare -Wtautological-unsigned-enum-zero-compare -Wtautological-unsigned-zero-compare -Wno-error=tautological-type-limit-compare -Wno-inline-new-delete -Wno-error=deprecated-declarations -Wno-error=array-bounds -Wno-error=backend-plugin -Wno-error=return-std-move -Wno-error=atomic-alignment -Wformat -Wformat-security -Wno-gnu-zero-variadic-macro-arguments -Wno-noexcept-type -Wno-unknown-warning-option -Wno-return-type-c-linkage -fno-sized-deallocation -fno-aligned-new -fno-rtti -fno-exceptions -fno-math-errno -pthread -pipe -g -O3 -fno-omit-frame-pointer -funwind-tables -fno-strict-aliasing -Werror=format -Wno-shadow  -MD -MP -MF .deps/Unified_cpp_js_src_debugger0.o.pp  -fcolor-diagnostics  Unified_cpp_js_src_debugger0.cpp

Comment 1

[Sylvestre Ledru [:Sylvestre]]

I am experiencing the same issue in the ci https://relman-ci.moz.tools/job/firefox-clang-last/1100/consoleFull
Nathan, rings a bell?

Comment 2

[Nathan Froyd [:froydnj]]

(In reply to Sylvestre Ledru [:Sylvestre] from comment #1)

I am experiencing the same issue in the ci https://relman-ci.moz.tools/job/firefox-clang-last/1100/consoleFull
Nathan, rings a bell?

It looks like the same sort of issue, but I'm mystified as to what's going on. Is the clang used in that job obtainable? What does clang++-10 -dM -E -x c++ /dev/null -o - say? And what does clang++-10 -dM -E -x c++ /dev/null -o - -std=gnu++17 say?

Comment 3

[Sylvestre Ledru [:Sylvestre]]

Attachment: defines C++17 and without

I guess you want the diff between the two :)

Comment 4

[Nathan Froyd [:froydnj]]

OK, so that makes sense. So why isn't this block in toolchain.configure being executed? https://searchfox.org/mozilla-central/source/build/moz.configure/toolchain.configure#528-537

Comment 5

[Ricky Stewart]

Is there a reason why you're not doing this instead?

ac_add_options CC=clang-9
ac_add_options CXX=clang++-9

This should work and doesn't skip over all the extra logic we add in the configure stage (which is why you have the behavior observed in comment 4).

It's unclear to me why anyone would use mk_add_options instead -- what's the use case there?

Comment 6

[Sylvestre Ledru [:Sylvestre]]

Sorry but we have to come up with a better plan.
I think we need to keep the export CC=clang++-10 support for tools like scan-build.

if you run tools running the build of Firefox, they export CC/CXX to custom compilers.

For example, scan-build is started:

scan-build-10 ./mach build 

or

scan-build-10  --use-cc=clang-10 --use-c++=clang++-10 ./mach build

Or at least, fail the configure with the old model.

Comment 7

[Ricky Stewart]

I just ran ~/.mozbuild/clang/bin/scan-build ./mach build and it does inject a custom compiler:

checking for the target C compiler... /home/rstewart/.mozbuild/clang/libexec/ccc-analyzer

The values of CC and CXX get properly populated as you can tell by checking the config.statusd:

$ cat objdir/config.statusd/substs/CXX
"/home/rstewart/.mozbuild/clang/libexec/c++-analyzer -std=gnu++17"

Is this not WAI? Otherwise, can you provide a more precise list of steps on how to reproduce an issue?

Comment 8

[Sylvestre Ledru [:Sylvestre]]

Sure, in the .mozconfig, I have:

export CC=clang-10
export CXX=clang++-10

For example, full log:
https://relman-ci.moz.tools/job/firefox-clang-last/1102/consoleFull

cat > .mozconfig <<EOF
export CC=clang-$CLANG_VERSION
export CXX=clang++-$CLANG_VERSION
ac_add_options --enable-debug
# --disable-unified-compilation
ac_add_options --enable-warnings-as-errors 

export CXXFLAGS="-Wno-error=pessimizing-move -Wno-error=large-by-value-copy=128 -Wempty-init-stmt -Wno-error=implicit-int-float-conversion -Werror=int-in-bool "

mk_add_options MOZ_MAKE_FLAGS="-j14"
EOF

To generate the mozconfig

I really don't mind moving to ac_add_options but we should probably fail the configure if the export solution is used

Comment 9

[Ricky Stewart]

Acknowledged. chmanchester, mshal, and I have been discussing the goofy behavior of .mozconfig files and I'm going to open a followup bug to address this. Meanwhile, every issue raised in this thread has an easy workaround so I'm going to re-close the bug.

Comment 10

[Ricky Stewart]

Sylvestre, I can't reproduce that failure locally. Setting export CC= and ac_add_options CC= both work equally well for me. Have you verified that using ac_add_options fixes your issue? We're wondering if your issue is instead being caused by the tree not building well with clang 10 rather than being related to the original bug report.

Comment 11

[Sylvestre Ledru [:Sylvestre]]

(In reply to Nathan Froyd [:froydnj] from comment #4)

OK, so that makes sense. So why isn't this block in toolchain.configure being executed? https://searchfox.org/mozilla-central/source/build/moz.configure/toolchain.configure#528-537

It does execute this part.

For example:

/usr/bin/clang++-10 -std=gnu++17 -o Unified_cpp_js_src_debugger0.o -c  -I/var/lib/jenkins/workspace/firefox-clang-last/obj-x86_64-pc-linux-gnu/dist/system_wrappers -include /var/lib/jenkins/workspace/firefox-clang-last/config/gcc_hidden.h -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -ftrivial-auto-var-init=pattern -DDEBUG=1 -DWASM_SUPPORTS_HUGE_MEMORY -DJS_CACHEIR_SPEW -DJS_STRUCTURED_SPEW -DJS_HAS_CTYPES -DFFI_BUILDING -DEXPORT_JS_API -DMOZ_HAS_MOZGLUE -I/var/lib/jenkins/workspace/firefox-clang-last/js/src/debugger -I/var/lib/jenkins/workspace/firefox-clang-last/obj-x86_64-pc-linux-gnu/js/src/debugger -I/var/lib/jenkins/workspace/firefox-clang-last/obj-x86_64-pc-linux-gnu/js/src -I/var/lib/jenkins/workspace/firefox-clang-last/js/src -I/var/lib/jenkins/workspace/firefox-clang-last/obj-x86_64-pc-linux-gnu/dist/include -I/var/lib/jenkins/workspace/firefox-clang-last/obj-x86_64-pc-linux-gnu/dist/include/nspr -fPIC -DMOZILLA_CLIENT -include /var/lib/jenkins/workspace/firefox-clang-last/obj-x86_64-pc-linux-gnu/js/src/js-confdefs.h -Qunused-arguments -Qunused-arguments -Wall -Wbitfield-enum-conversion -Wempty-body -Wignored-qualifiers -Woverloaded-virtual -Wpointer-arith -Wshadow-field-in-constructor-modified -Wsign-compare -Wtype-limits -Wunreachable-code -Wunreachable-code-return -Wwrite-strings -Wno-invalid-offsetof -Wclass-varargs -Wempty-init-stmt -Wfloat-overflow-conversion -Wfloat-zero-conversion -Wloop-analysis -Wc++2a-compat -Wcomma -Wimplicit-fallthrough -Werror=non-literal-null-conversion -Wstring-conversion -Wtautological-overlap-compare -Wtautological-unsigned-enum-zero-compare -Wtautological-unsigned-zero-compare -Wno-error=tautological-type-limit-compare -Wno-inline-new-delete -Wno-error=deprecated-declarations -Wno-error=array-bounds -Wno-error=backend-plugin -Wno-error=return-std-move -Wno-error=atomic-alignment -Wno-error=deprecated-copy -Wformat -Wformat-security -Wno-gnu-zero-variadic-macro-arguments -Wno-noexcept-type -Wno-unknown-warning-option -Wno-return-type-c-linkage -fno-sized-deallocation -fno-aligned-new -Wno-error=pessimizing-move -Wno-error=large-by-value-copy=128 -Wempty-init-stmt -Wno-error=implicit-int-float-conversion -Werror=int-in-bool -fno-rtti -fno-exceptions -fno-math-errno -pthread -pipe -g -O3 -fno-omit-frame-pointer -funwind-tables -Werror -fno-strict-aliasing -Werror=format -Wno-shadow  -MD -MP -MF .deps/Unified_cpp_js_src_debugger0.o.pp   Unified_cpp_js_src_debugger0.cpp

has -std=gnu++17

We're wondering if your issue is instead being caused by the tree not building well with clang 10

Not sure what you mean here.

ac_add_option

it fail with it too:

 In file included from Unified_cpp_js_src_debugger0.cpp:2:
 In file included from /var/lib/jenkins/workspace/firefox-clang-last/js/src/debugger/DebugScript.cpp:7:
 In file included from /var/lib/jenkins/workspace/firefox-clang-last/js/src/debugger/DebugScript.h:14:
 In file included from /var/lib/jenkins/workspace/firefox-clang-last/js/src/jsapi.h:18:
 /var/lib/jenkins/workspace/firefox-clang-last/obj-x86_64-pc-linux-gnu/dist/include/mozilla/RefPtr.h:116:46: error: no template named 'is_convertible_v' in namespace 'std'; did you mean 'is_convertible'?
             typename = std::enable_if_t<std::is_convertible_v<I*, T*>>>
                                         ~~~~~^~~~~~~~~~~~~~~~
                                              is_convertible
 /usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0/type_traits:1558:12: note: 'is_convertible' declared here
     struct is_convertible
            ^

I can't reproduce that failure locally.

Did you try with libstdc++6 ?

Comment 12

[Sylvestre Ledru [:Sylvestre]]

I upgraded to Debian buster (10.2) from stretch (9) and I don't see this issue anymore.

Comment 13

[Ricky Stewart]

OK. In the absence of any further information here I'm going to re-close since this confirms it's an environmental issue rather than anything related to the OP.

Comment 14

[Sylvestre Ledru [:Sylvestre]]

The bug is valid but I agree that we might not want to spend time on it.