1419798 - FUZZING definitions assume LIBFUZZER or AFL compiler.

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Jesse Schwartzentruber (:truber)]

We should be able to build --enable-fuzzing builds without libfuzzer or AFL.

Currently we get unused function errors because the fuzzing interfaces (MOZ_FUZZING_INTERFACE_STREAM/MOZ_FUZZING_INTERFACE_RAW) are unused if either libfuzzer or AFL are not used.

For example:

[task 2017-11-22T01:45:40.025Z] 01:45:40     INFO -  gmake[4]: Entering directory '/builds/worker/workspace/build/src/obj-firefox/media/webrtc/signaling/fuzztest'
[task 2017-11-22T01:45:40.029Z] 01:45:40     INFO -  /builds/worker/workspace/build/src/sccache2/sccache /builds/worker/workspace/build/src/gcc/bin/g++ -o sdp_parser_libfuzz.o -c -I/builds/worker/workspace/build/src/obj-firefox/dist/stl_wrappers -I/builds/worker/workspace/build/src/obj-firefox/dist/system_wrappers -include /builds/worker/workspace/build/src/config/gcc_hidden.h -DDEBUG=1 -DSTATIC_EXPORTABLE_JS_API -DMOZILLA_INTERNAL_API -DIMPL_LIBXUL -I/builds/worker/workspace/build/src/media/webrtc/signaling/fuzztest -I/builds/worker/workspace/build/src/obj-firefox/media/webrtc/signaling/fuzztest -I/builds/worker/workspace/build/src/ipc/chromium/src -I/builds/worker/workspace/build/src/media/mtransport -I/builds/worker/workspace/build/src/media/webrtc -I/builds/worker/workspace/build/src/media/webrtc/signaling/src/common/time_profiling -I/builds/worker/workspace/build/src/media/webrtc/signaling/src/peerconnection -I/builds/worker/workspace/build/src/obj-firefox/dist/include -I/builds/worker/workspace/build/src/obj-firefox/dist/include/nspr -I/builds/worker/workspace/build/src/obj-firefox/dist/include/nss -fPIC -DMOZILLA_CLIENT -include /builds/worker/workspace/build/src/obj-firefox/mozilla-config.h -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Wall -Wempty-body -Wignored-qualifiers -Woverloaded-virtual -Wpointer-arith -Wsign-compare -Wtype-limits -Wunreachable-code -Wwrite-strings -Wno-invalid-offsetof -Wc++14-compat -Wduplicated-cond -Wno-error=maybe-uninitialized -Wno-error=deprecated-declarations -Wno-error=array-bounds -Wno-error=free-nonheap-object -Wformat -fno-sized-deallocation -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fno-exceptions -fno-strict-aliasing -fno-rtti -ffunction-sections -fdata-sections -fno-exceptions -fno-math-errno -pthread -D_GLIBCXX_USE_CXX11_ABI=0 -pipe -g -freorder-blocks -Os -fno-omit-frame-pointer -Werror  -MD -MP -MF .deps/sdp_parser_libfuzz.o.pp   /builds/worker/workspace/build/src/media/webrtc/signaling/fuzztest/sdp_parser_libfuzz.cpp
[task 2017-11-22T01:45:40.029Z] 01:45:40     INFO -  /builds/worker/workspace/build/src/media/webrtc/signaling/fuzztest/sdp_parser_libfuzz.cpp:25:1: error: 'int RunSdpParserFuzzing(const uint8_t*, size_t)' defined but not used [-Werror=unused-function]
[task 2017-11-22T01:45:40.029Z] 01:45:40     INFO -   RunSdpParserFuzzing(const uint8_t* data, size_t size) {
[task 2017-11-22T01:45:40.029Z] 01:45:40     INFO -   ^~~~~~~~~~~~~~~~~~~
[task 2017-11-22T01:45:40.029Z] 01:45:40     INFO -  cc1plus: all warnings being treated as errors
[task 2017-11-22T01:45:40.029Z] 01:45:40     INFO -  /builds/worker/workspace/build/src/config/rules.mk:1028: recipe for target 'sdp_parser_libfuzz.o' failed
[task 2017-11-22T01:45:40.029Z] 01:45:40     INFO -  gmake[4]: *** [sdp_parser_libfuzz.o] Error 1
[task 2017-11-22T01:45:40.029Z] 01:45:40     INFO -  gmake[4]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/media/webrtc/signaling/fuzztest'
[task 2017-11-22T01:45:40.029Z] 01:45:40     INFO -  /builds/worker/workspace/build/src/config/recurse.mk:73: recipe for target 'media/webrtc/signaling/fuzztest/target' failed
[task 2017-11-22T01:45:40.030Z] 01:45:40     INFO -  gmake[3]: *** [media/webrtc/signaling/fuzztest/target] Error 2
[task 2017-11-22T01:45:40.031Z] 01:45:40     INFO -  gmake[3]: *** Waiting for unfinished jobs....

Comment 1

[Jesse Schwartzentruber (:truber)]

Attachment: Bug 1419798 - Only build fuzzing interfaces if clang or AFL compilers are used.

Review commit: https://reviewboard.mozilla.org/r/202080/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/202080/

Comment 2

[Mike Hommey [:glandium]]

Comment on attachment 8930985 [details]
Bug 1419798 - Only build fuzzing interfaces if clang or AFL compilers are used.

https://reviewboard.mozilla.org/r/202080/#review207730

::: js/moz.configure:261
(Diff revision 1)
> +    set_config('AFLFUZZER', enable_aflfuzzer)
> +    set_define('AFLFUZZER', enable_aflfuzzer)

Rather than adding this, can't we just use FUZZING as meaning that we have either AFL or libfuzzer, and actively reject --enable-fuzzing if we have neither? Or is --enable-fuzzing still useful without AFL of libfuzzer?

Comment 3

[Jesse Schwartzentruber (:truber)]

Sorry I should have added more motivation. There are features behind FUZZING that are not libfuzzer/AFL interfaces, such as the fuzzing functions (bug 1346339) and purpose-built in-process fuzzers (eg. bug 1232119).

I discussed with decoder having another define for this instead of adding AFLFUZZER, such as:

    FUZZING_INTERFACES = FUZZING and (AFL or LIBFUZZER)

That gives future fuzzers a single flag to use which has the meaning you thought FUZZING had.

Comment 4

[Jesse Schwartzentruber (:truber)]

Comment on attachment 8930985 [details]
Bug 1419798 - Only build fuzzing interfaces if clang or AFL compilers are used.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/202080/diff/1-2/

Comment 5

[Mike Hommey [:glandium]]

Comment on attachment 8930985 [details]
Bug 1419798 - Only build fuzzing interfaces if clang or AFL compilers are used.

https://reviewboard.mozilla.org/r/202080/#review208782

This new version is clearer. Thanks.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/10f9b7960bc7
Only build fuzzing interfaces if clang or AFL compilers are used. r=glandium

Comment 7

[Tiberius Oros[:tiberius_oros]]

https://hg.mozilla.org/mozilla-central/rev/10f9b7960bc7

Comment 8

[Tiberius Oros[:tiberius_oros]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/10f9b7960bc7