1776344 - Warning printing code at the end of the build doesn't ignore warnings from third-party projects

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Ryan VanderMeulen [:RyanVM]]

Attachment: Warnings list

Looking at https://searchfox.org/mozilla-central/rev/45ecfdfaa2f74083b11c1414ddb4c1a640b0d19d/python/mozbuild/mozbuild/controller/building.py#1478, it appears that the intent of that code is to ignore warnings from third-party projects listed in ThirdPartyPaths.txt and Generated.txt when printing all build warnings out at the end of the build.

However, in my local Windows build, nearly all of the printed warnings comes from directories which should have been ignored (see attachment).

Comment 1

[Ryan VanderMeulen [:RyanVM]]

Reproducible for me on Linux as well.

Comment 2

[Ryan VanderMeulen [:RyanVM]]

If I print the LOCAL_SUPPRESS_DIRS tuple, this is what it looks like:

('browser/components/newtab/vendor/\n', 'browser/components/pocket/content/panels/js/vendor/\n', 'browser/components/storybook/node_modules/\n', 'browser/components/translation/cld2/\n', 'browser/extensions/formautofill/content/third-party/\n', 'browser/extensions/formautofill/test/fixtures/third_party/\n', 'browser/extensions/screenshots/build/raven.js\n', 'devtools/client/inspector/markup/test/lib_*\n', 'devtools/client/jsonview/lib/require.js\n', 'devtools/client/shared/build/babel.js\n', 'devtools/client/shared/source-map/\n', 'devtools/client/shared/sourceeditor/codemirror/\n', 'devtools/client/shared/sourceeditor/test/cm_mode_ruby.js\n', 'devtools/client/shared/sourceeditor/test/codemirror/\n', 'devtools/client/shared/vendor/\n', 'devtools/shared/acorn/\n', 'devtools/shared/heapsnapshot/CoreDump.pb.cc\n', 'devtools/shared/heapsnapshot/CoreDump.pb.h\n', 'devtools/shared/jsbeautify/\n', 'devtools/shared/node-properties/\n', 'devtools/shared/qrcode/decoder/\n', 'devtools/shared/qrcode/encoder/\n', 'devtools/shared/sprintfjs/\n', 'devtools/shared/storage/vendor/\n', 'dom/canvas/test/webgl-conf/checkout/\n', 'dom/imptests/\n', 'dom/media/gmp/rlz/\n', 'dom/media/gmp/widevine-adapter/content_decryption_module_export.h\n', 'dom/media/gmp/widevine-adapter/content_decryption_module_ext.h\n', 'dom/media/gmp/widevine-adapter/content_decryption_module.h\n', 'dom/media/gmp/widevine-adapter/content_decryption_module_proxy.h\n', 'dom/media/platforms/ffmpeg/ffmpeg57/\n', 'dom/media/platforms/ffmpeg/ffmpeg58/\n', 'dom/media/platforms/ffmpeg/ffmpeg59/\n', 'dom/media/platforms/ffmpeg/libav53/\n', 'dom/media/platforms/ffmpeg/libav54/\n', 'dom/media/platforms/ffmpeg/libav55/\n', 'dom/media/webaudio/test/blink/\n', 'dom/media/webrtc/transport/third_party/\n', 'dom/media/webspeech/recognition/endpointer.cc\n', 'dom/media/webspeech/recognition/endpointer.h\n', 'dom/media/webspeech/recognition/energy_endpointer.cc\n', 'dom/media/webspeech/recognition/energy_endpointer.h\n', 'dom/media/webspeech/recognition/energy_endpointer_params.cc\n', 'dom/media/webspeech/recognition/energy_endpointer_params.h\n', 'dom/media/webvtt/vtt.jsm\n', 'dom/tests/mochitest/ajax/\n', 'dom/tests/mochitest/dom-level1-core/\n', 'dom/tests/mochitest/dom-level2-core/\n', 'dom/tests/mochitest/dom-level2-html/\n', 'dom/u2f/tests/pkijs/\n', 'dom/webauthn/cbor-cpp/\n', 'dom/webauthn/tests/pkijs/\n', 'editor/libeditor/tests/browserscope/lib/richtext/\n', 'editor/libeditor/tests/browserscope/lib/richtext2/\n', 'extensions/spellcheck/hunspell/src/\n', 'function2/\n', 'gfx/angle/checkout/\n', 'gfx/cairo/\n', 'gfx/graphite2/\n', 'gfx/harfbuzz/\n', 'gfx/ots/\n', 'gfx/qcms/\n', 'gfx/sfntly/\n', 'gfx/skia/\n', 'gfx/vr/service/openvr/\n', 'gfx/vr/service/openvr/headers/openvr.h\n', 'gfx/vr/service/openvr/src/README\n', 'gfx/vr/service/openvr/src/dirtools_public.cpp\n', 'gfx/vr/service/openvr/src/dirtools_public.h\n', 'gfx/vr/service/openvr/src/envvartools_public.cpp\n', 'gfx/vr/service/openvr/src/envvartools_public.h\n', 'gfx/vr/service/openvr/src/hmderrors_public.cpp\n', 'gfx/vr/service/openvr/src/hmderrors_public.h\n', 'gfx/vr/service/openvr/src/ivrclientcore.h\n', 'gfx/vr/service/openvr/src/openvr_api_public.cpp\n', 'gfx/vr/service/openvr/src/pathtools_public.cpp\n', 'gfx/vr/service/openvr/src/pathtools_public.h\n', 'gfx/vr/service/openvr/src/sharedlibtools_public.cpp\n', 'gfx/vr/service/openvr/src/sharedlibtools_public.h\n', 'gfx/vr/service/openvr/src/strtools_public.cpp\n', 'gfx/vr/service/openvr/src/strtools_public.h\n', 'gfx/vr/service/openvr/src/vrpathregistry_public.cpp\n', 'gfx/vr/service/openvr/src/vrpathregistry_public.h\n', 'gfx/wr/\n', 'gfx/ycbcr/\n', 'intl/icu/\n', 'ipc/chromium/src/third_party/\n', 'js/src/ctypes/libffi/\n', 'js/src/dtoa.c\n', 'js/src/editline/\n', 'js/src/jit/arm64/vixl/\n', 'js/src/octane/\n', 'js/src/vtune/disable_warnings.h\n', 'js/src/vtune/ittnotify_config.h\n', 'js/src/vtune/ittnotify.h\n', 'js/src/vtune/ittnotify_static.c\n', 'js/src/vtune/ittnotify_static.h\n', 'js/src/vtune/ittnotify_types.h\n', 'js/src/vtune/jitprofiling.c\n', 'js/src/vtune/jitprofiling.h\n', 'js/src/vtune/legacy/\n', 'js/src/zydis/\n', 'layout/docs/css-gap-decorations/\n', 'media/ffvpx/\n', 'media/kiss_fft/\n', 'media/libaom/\n', 'media/libcubeb/\n', 'media/libdav1d/\n', 'media/libjpeg/\n', 'media/libmkv/\n', 'media/libnestegg/\n', 'media/libogg/\n', 'media/libopus/\n', 'media/libpng/\n', 'media/libsoundtouch/\n', 'media/libspeex_resampler/\n', 'media/libtheora/\n', 'media/libtremor/\n', 'media/libvorbis/\n', 'media/libvpx/\n', 'media/libwebp/\n', 'media/libyuv/\n', 'media/mozva/va\n', 'media/mp4parse-rust/\n', 'media/openmax_dl/\n', 'media/openmax_il/\n', 'media/webrtc/signaling/gtest/MockCall.h\n', 'mfbt/double-conversion/double-conversion/\n', 'mfbt/lz4/.*\n', 'mobile/android/geckoview/src/thirdparty/\n', 'modules/brotli/\n', 'modules/fdlibm/\n', 'modules/freetype2/\n', 'modules/woff2/\n', 'modules/xz-embedded/\n', 'modules/zlib/\n', 'mozglue/misc/decimal/\n', 'mozglue/tests/glibc_printf_tests/\n', 'netwerk/dns/nsIDNKitInterface.h\n', 'netwerk/sctp/src/\n', 'netwerk/srtp/src/\n', 'nsprpub/\n', 'other-licenses/\n', 'parser/expat/\n', 'remote/cdp/test/browser/chrome-remote-interface.js\n', 'remote/test/puppeteer/\n', 'security/nss/\n', 'security/sandbox/chromium/\n', 'security/sandbox/chromium-shim/\n', 'testing/gtest/gmock/\n', 'testing/gtest/gtest/\n', 'testing/mochitest/MochiKit/\n', 'testing/mochitest/pywebsocket3/\n', 'testing/mochitest/tests/MochiKit-1.4.2/\n', 'testing/modules/sinon-7.2.7.js\n', 'testing/mozbase/mozproxy/mozproxy/backends/mitm/scripts/catapult/\n', 'testing/talos/talos/tests/dromaeo/\n', 'testing/talos/talos/tests/kraken/\n', 'testing/talos/talos/tests/v8_7/\n', 'testing/web-platform/tests/resources/webidl2/\n', 'testing/web-platform/tests/tools/third_party/\n', 'testing/xpcshell/dns-packet/\n', 'testing/xpcshell/node-ip/\n', 'testing/xpcshell/node-http2/\n', 'testing/xpcshell/odoh-wasm/\n', 'third_party/\n', 'toolkit/components/certviewer/content/vendor/\n', 'toolkit/components/jsoncpp/\n', 'toolkit/components/normandy/vendor/\n', 'toolkit/components/passwordmgr/PasswordRulesParser.jsm\n', 'toolkit/components/protobuf/\n', 'toolkit/components/url-classifier/chromium/\n', 'toolkit/components/utils/mozjexl.js\n', 'toolkit/components/viaduct/fetch_msg_types.pb.cc\n', 'toolkit/components/viaduct/fetch_msg_types.pb.h\n', 'toolkit/crashreporter/breakpad-client/\n', 'toolkit/crashreporter/google-breakpad/\n', 'tools/fuzzing/libfuzzer/\n', 'tools/profiler/core/vtune/\n', 'xpcom/build/mach_override.c\n', 'xpcom/build/mach_override.h\n', 'xpcom/io/crc32c.c\n', 'browser/components/newtab/logs/\n', 'browser/components/newtab/node_modules/\n', 'devtools/client/aboutdebugging/test/jest/node_modules/\n', 'devtools/client/application/test/components/node_modules/\n', 'devtools/client/debugger/node_modules/\n', 'dom/tests/ajax/jquery/\n', 'dom/tests/ajax/mochikit/\n', 'node_modules/\n', 'intl/components/src/UnicodeScriptCodes.h\n', 'intl/unicharutil/util/nsSpecialCasingData.cpp\n', 'intl/unicharutil/util/nsUnicodePropertyData.cpp\n')

I'm thinking the newline isn't being properly stripped?

Comment 3

[Ryan VanderMeulen [:RyanVM]]

Confirmed, that's the issue. If hack it to remove those newlines, things work as expected:

20:25.58 (suppressed 9 warnings in gfx/cairo)
20:25.58 (suppressed 1 warnings in gfx/graphite2)
20:25.58 (suppressed 3 warnings in gfx/harfbuzz)
20:25.58 (suppressed 23 warnings in gfx/skia)
20:25.58 (suppressed 1 warnings in intl/icu)
20:25.58 (suppressed 3 warnings in js/src/ctypes/libffi)
20:25.58 (suppressed 17 warnings in media/ffvpx)
20:25.58 (suppressed 1 warnings in media/libcubeb)
20:25.58 (suppressed 1 warnings in media/libmkv)
20:25.58 (suppressed 1 warnings in media/libnestegg)
20:25.58 (suppressed 1 warnings in media/libopus)
20:25.58 (suppressed 8 warnings in media/libsoundtouch)
20:25.59 (suppressed 1 warnings in media/libspeex_resampler)
20:25.59 (suppressed 1 warnings in media/libtheora)
20:25.59 (suppressed 1 warnings in media/libvorbis)
20:25.59 (suppressed 13 warnings in media/libvpx)
20:25.59 (suppressed 1 warnings in modules/woff2)
20:25.59 (suppressed 24 warnings in netwerk/sctp/src)
20:25.59 (suppressed 67 warnings in nsprpub)
20:25.59 (suppressed 108 warnings in security/nss)
20:25.59 (suppressed 57 warnings in third_party)

Comment 4

[Ryan VanderMeulen [:RyanVM]]

Attachment: Bug 1776344 - Properly strip newlines when reading path files. r=ahal

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a204f5fe09d7
Properly strip newlines when reading path files. r=ahal

Comment 6

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/a204f5fe09d7