Closed Bug 1738872 Opened 3 years ago Closed 3 years ago

recent WebRTC rebase broke ppc64le builds

Categories

(Core :: WebRTC, defect)

Product:
Core
Component:
WebRTC
Platform:
Other
Linux
Type:
defect

Tracking

()

Status:
RESOLVED FIXED
Milestone:
96 Branch
Tracking Flags:
Tracking Status
firefox-esr91 --- unaffected
firefox94 --- unaffected
firefox95 --- unaffected
firefox96 --- fixed

People

(Reporter: dan, Assigned: dan)

References

(Regression)

Details

(Keywords: regression)

Attachments

(1 file)

Bug 1738872 - Add preprocessor support for additional architectures
48 bytes, text/x-phabricator-request
Details | Review

The recent rebase of WebRTC broke ppc64le builds, please see below

...
third_party/libwebrtc/api/audio/aec3_factory_gn/Unified_cpp_aec3_factory_gn0.o
/usr/bin/g++ -o Unified_cpp_aec3_factory_gn0.o -c  -I/home/jenkins/workspace/Firefox-default/label/ppc64le/firefox/obj-powerpc64le-unknown-linux-gnu/dist/stl_wrappers -I/home/jenkins/workspace/Firefox-default/label/ppc64le/firefox/obj-powerpc64le-unknown-linux-gnu/dist/system_wrappers -include /home/jenkins/workspace/Firefox-default/label/ppc64le/firefox/config/gcc_hidden.h -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -DNDEBUG -DTRIMMED=1 -DABSL_ALLOCATOR_NOTHROW=1 -DRTC_ENABLE_VP9 -DWEBRTC_APM_DEBUG_DUMP=0 -DWEBRTC_ENABLE_PROTOBUF=0 -DWEBRTC_LIBRARY_IMPL -DWEBRTC_MOZILLA_BUILD -DWEBRTC_NON_STATIC_TRACE_EVENT_HANDLERS=0 -DDYNAMIC_ANNOTATIONS_ENABLED=0 -DNVALGRIND -DUSE_AURA=1 -DUSE_GLIB=1 -DUSE_NSS_CERTS=1 -DUSE_OZONE=1 -DUSE_UDEV -DUSE_X11=1 -DWEBRTC_LINUX -DWEBRTC_POSIX -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -D_LARGEFILE64_SOURCE -D_LARGEFILE_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D_FORTIFY_SOURCE=2 -DMOZ_HAS_MOZGLUE -DMOZILLA_INTERNAL_API -DIMPL_LIBXUL -DSTATIC_EXPORTABLE_JS_API -I/home/jenkins/workspace/Firefox-default/label/ppc64le/firefox/third_party/libwebrtc/api/audio/aec3_factory_gn -I/home/jenkins/workspace/Firefox-default/label/ppc64le/firefox/obj-powerpc64le-unknown-linux-gnu/third_party/libwebrtc/api/audio/aec3_factory_gn -I/home/jenkins/workspace/Firefox-default/label/ppc64le/firefox/obj-powerpc64le-unknown-linux-gnu/ipc/ipdl/_ipdlheaders -I/home/jenkins/workspace/Firefox-default/label/ppc64le/firefox/ipc/chromium/src -I/home/jenkins/workspace/Firefox-default/label/ppc64le/firefox/ipc/glue -I/home/jenkins/workspace/Firefox-default/label/ppc64le/firefox/third_party/libwebrtc -I/home/jenkins/workspace/Firefox-default/label/ppc64le/firefox/third_party/libwebrtc/third_party/abseil-cpp -I/home/jenkins/workspace/Firefox-default/label/ppc64le/firefox/tools/profiler/public -I/home/jenkins/workspace/Firefox-default/label/ppc64le/firefox/obj-powerpc64le-unknown-linux-gnu/dist/include -DMOZILLA_CLIENT -include /home/jenkins/workspace/Firefox-default/label/ppc64le/firefox/obj-powerpc64le-unknown-linux-gnu/mozilla-config.h -Wall -Wempty-body -Wignored-qualifiers -Wpointer-arith -Wsign-compare -Wtype-limits -Wunreachable-code -Wno-invalid-offsetof -Wc++2a-compat -Wduplicated-cond -Wimplicit-fallthrough -Wno-error=maybe-uninitialized -Wno-error=deprecated-declarations -Wno-error=array-bounds -Wno-error=coverage-mismatch -Wno-error=free-nonheap-object -Wno-multistatement-macros -Wno-error=class-memaccess -Wno-error=deprecated-copy -Wformat -Wformat-overflow=2 -Wno-psabi -fno-sized-deallocation -fno-aligned-new -g1 -fno-exceptions -fno-strict-aliasing -fPIC -fno-rtti -fno-exceptions -fno-math-errno -pthread -pipe -g -freorder-blocks -O2 -fno-omit-frame-pointer -funwind-tables  -MD -MP -MF .deps/Unified_cpp_aec3_factory_gn0.o.pp   Unified_cpp_aec3_factory_gn0.cpp
In file included from <command-line>:
/home/jenkins/workspace/Firefox-default/label/ppc64le/firefox/obj-powerpc64le-unknown-linux-gnu/mozilla-config.h:212: warning: "__STDC_CONSTANT_MACROS" redefined
  212 | #define __STDC_CONSTANT_MACROS
      |
<command-line>: note: this is the location of the previous definition
In file included from /home/jenkins/workspace/Firefox-default/label/ppc64le/firefox/third_party/libwebrtc/modules/audio_processing/aec3/fft_data.h:15,
                 from /home/jenkins/workspace/Firefox-default/label/ppc64le/firefox/third_party/libwebrtc/modules/audio_processing/aec3/fft_buffer.h:18,
                 from /home/jenkins/workspace/Firefox-default/label/ppc64le/firefox/third_party/libwebrtc/modules/audio_processing/aec3/render_buffer.h:22,
                 from /home/jenkins/workspace/Firefox-default/label/ppc64le/firefox/third_party/libwebrtc/modules/audio_processing/aec3/echo_remover.h:21,
                 from /home/jenkins/workspace/Firefox-default/label/ppc64le/firefox/third_party/libwebrtc/modules/audio_processing/aec3/block_processor.h:21,
                 from /home/jenkins/workspace/Firefox-default/label/ppc64le/firefox/third_party/libwebrtc/modules/audio_processing/aec3/echo_canceller3.h:25,
                 from /home/jenkins/workspace/Firefox-default/label/ppc64le/firefox/third_party/libwebrtc/api/audio/echo_canceller3_factory.cc:14,
                 from Unified_cpp_aec3_factory_gn0.cpp:2:
/home/jenkins/workspace/Firefox-default/label/ppc64le/firefox/third_party/libwebrtc/rtc_base/system/arch.h:54:2: error: #error Please add support for your architecture in rtc_base/system/arch.h
   54 | #error Please add support for your architecture in rtc_base/system/arch.h
      |  ^~~~~
/home/jenkins/workspace/Firefox-default/label/ppc64le/firefox/third_party/libwebrtc/rtc_base/system/arch.h:58:2: error: #error Define either WEBRTC_ARCH_LITTLE_ENDIAN or WEBRTC_ARCH_BIG_ENDIAN
   58 | #error Define either WEBRTC_ARCH_LITTLE_ENDIAN or WEBRTC_ARCH_BIG_ENDIAN
      |  ^~~~~
gmake[4]: *** [/home/jenkins/workspace/Firefox-default/label/ppc64le/firefox/config/rules.mk:662: Unified_cpp_aec3_factory_gn0.o] Error 1
gmake[4]: Leaving directory '/home/jenkins/workspace/Firefox-default/label/ppc64le/firefox/obj-powerpc64le-unknown-linux-gnu/third_party/libwebrtc/api/audio/aec3_factory_gn'
gmake[3]: *** [/home/jenkins/workspace/Firefox-default/label/ppc64le/firefox/config/recurse.mk:72: third_party/libwebrtc/api/audio/aec3_factory_gn/target-objects] Error 2

Dan,

Thank you for the report. ppc64le is not one of our Tier-1/2/3 builds as far as I can tell. We're happy to consider patches for the new build config. There are instructions how to generate new build files here. There is a section at the bottom of that file titled "Adding new configurations to the build" to get started.

Severity: -- → S3

Thanks for the pointer, Michael. I will take a look unless someone else will be faster :-)

I think there are actually 2 issues

  • webrtc lost support for anything except x86/arm/aarch64/mipsel in rtc_base/system/arch.h, I believe it handled other arches in the previous release
  • the buildsystem lost support for eg. ppc64, that was added in bug 1506644

Aren't the current mozconfig files all set for cross-compiling? Because only x64_* files exist in the dom/media/webrtc/third_party_build/gn-configs directory.

The fix for the first part is in https://webrtc.googlesource.com/src/+/6215ba804eb500f3e28b39088c73af3c4f4cd10a, how should we proceed here?

Set release status flags based on info from the regressing bug 1654112

status-firefox94: --- → unaffected
status-firefox95: --- → unaffected
status-firefox-esr91: --- → unaffected

Dan, can you submit the patch via phabricator? That's how the patches are merged.
https://moz-conduit.readthedocs.io/en/latest/phabricator-user.html

Flags: needinfo?(dan)

Martin, you mean patch for the change that went to the Google's webrtc repo, right? I am testing it locally now, then I will open a review request.

Flags: needinfo?(dan)

Good news, https://webrtc.googlesource.com/src/+/6215ba804eb500f3e28b39088c73af3c4f4cd10a%5E%21/ applies cleanly on trunk and FF can be built. Seems the updates to the build configurations are not needed.

  • _M_ARM is used by Microsoft 1
  • __riscv and __riscv_xlen are defined by 2
  • __sparc and sparc are documented at 3
  • MIPSEB, PPC, PPC64 are documented at 3 and used in
    Chromium's build/build_config.h 4
    Note: Chromium assumes that all PowerPC architectures are 64-bit. This
    is in fact not true.

Change-Id: I7d0176c38102e5e4cf0fcbe9b06a3520a79b0d71
Bug: webrtc:12312
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/198241
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Timothy Gu <timothygu@chromium.org>
Cr-Commit-Position: refs/heads/master@{#32897}

Assignee: nobody → dan
Status: NEW → ASSIGNED

Dan, do you need me to start a try build?

Flags: needinfo?(dan)

(In reply to Nico Grunbaum [:ng, @chew:mozilla.org] from comment #10)

Dan, do you need me to start a try build?

likely a good idea as I have tested only ppc64le build locally

Flags: needinfo?(dan)

Dan, you'll find your try build at the link below. Some of the lower tier builds (AARCH64 win) may not be scheduled for days, and aren't necessary to complete.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=fd91f621fa4800e7f3a735133fe82fc5afa3b4c3

When it looks good, let me know and I'll review the results and land the patch.

PipeWide does not build on latest trunk due to Bug 1739142.

FWIW the results look good to me :-)

Pushed by na-g@nostrum.com:
https://hg.mozilla.org/integration/autoland/rev/8bb1f58ba72b
Add preprocessor support for additional architectures r=ng

This is not enough to fix ppc64el builds:

[task 2021-12-01T08:40:23.003Z] 08:40:23     INFO -  In file included from Unified_c_pffft_pffft_gn0.c:2:
[task 2021-12-01T08:40:23.004Z] 08:40:23    ERROR -  /builds/worker/checkouts/gecko/third_party/libwebrtc/third_party/pffft/src/pffft.c:110:45: error: implicit declaration of function 'vec_lde' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
[task 2021-12-01T08:40:23.004Z] 08:40:23     INFO -  inline v4sf ld_ps1(const float *p) { v4sf v=vec_lde(0,p); return vec_splat(vec_perm(v, v, vec_lvsl(0, p)), 0); }
[task 2021-12-01T08:40:23.005Z] 08:40:23     INFO -                                              ^
[task 2021-12-01T08:40:23.006Z] 08:40:23    ERROR -  /builds/worker/checkouts/gecko/third_party/libwebrtc/third_party/pffft/src/pffft.c:110:43: error: initializing 'v4sf' (vector of 4 'float' values) with an expression of incompatible type 'int'
[task 2021-12-01T08:40:23.007Z] 08:40:23     INFO -  inline v4sf ld_ps1(const float *p) { v4sf v=vec_lde(0,p); return vec_splat(vec_perm(v, v, vec_lvsl(0, p)), 0); }
[task 2021-12-01T08:40:23.007Z] 08:40:23     INFO -                                            ^ ~~~~~~~~~~~~
[task 2021-12-01T08:40:23.008Z] 08:40:23    ERROR -  /builds/worker/checkouts/gecko/third_party/libwebrtc/third_party/pffft/src/pffft.c:110:66: error: implicit declaration of function 'vec_splat' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
[task 2021-12-01T08:40:23.009Z] 08:40:23     INFO -  inline v4sf ld_ps1(const float *p) { v4sf v=vec_lde(0,p); return vec_splat(vec_perm(v, v, vec_lvsl(0, p)), 0); }
[task 2021-12-01T08:40:23.010Z] 08:40:23     INFO -                                                                   ^
[task 2021-12-01T08:40:23.010Z] 08:40:23    ERROR -  /builds/worker/checkouts/gecko/third_party/libwebrtc/third_party/pffft/src/pffft.c:110:76: error: implicit declaration of function 'vec_perm' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
[task 2021-12-01T08:40:23.011Z] 08:40:23     INFO -  inline v4sf ld_ps1(const float *p) { v4sf v=vec_lde(0,p); return vec_splat(vec_perm(v, v, vec_lvsl(0, p)), 0); }
[task 2021-12-01T08:40:23.012Z] 08:40:23     INFO -                                                                             ^
[task 2021-12-01T08:40:23.013Z] 08:40:23    ERROR -  /builds/worker/checkouts/gecko/third_party/libwebrtc/third_party/pffft/src/pffft.c:110:91: error: implicit declaration of function 'vec_lvsl' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
[task 2021-12-01T08:40:23.013Z] 08:40:23     INFO -  inline v4sf ld_ps1(const float *p) { v4sf v=vec_lde(0,p); return vec_splat(vec_perm(v, v, vec_lvsl(0, p)), 0); }
[task 2021-12-01T08:40:23.014Z] 08:40:23     INFO -                                                                                            ^
[task 2021-12-01T08:40:23.015Z] 08:40:23    ERROR -  /builds/worker/checkouts/gecko/third_party/libwebrtc/third_party/pffft/src/pffft.c:110:66: error: returning 'int' from a function with incompatible result type 'v4sf' (vector of 4 'float' values)
[task 2021-12-01T08:40:23.016Z] 08:40:23     INFO -  inline v4sf ld_ps1(const float *p) { v4sf v=vec_lde(0,p); return vec_splat(vec_perm(v, v, vec_lvsl(0, p)), 0); }
[task 2021-12-01T08:40:23.016Z] 08:40:23     INFO -                                                                   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[task 2021-12-01T08:40:23.017Z] 08:40:23    ERROR -  /builds/worker/checkouts/gecko/third_party/libwebrtc/third_party/pffft/src/pffft.c:224:25: error: implicit declaration of function 'vec_splat_u8' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
[task 2021-12-01T08:40:23.018Z] 08:40:23     INFO -    t = a0; u = a1; t.v = VZERO();
[task 2021-12-01T08:40:23.019Z] 08:40:23     INFO -                          ^
[task 2021-12-01T08:40:23.019Z] 08:40:23     INFO -  /builds/worker/checkouts/gecko/third_party/libwebrtc/third_party/pffft/src/pffft.c:105:35: note: expanded from macro 'VZERO'
[task 2021-12-01T08:40:23.020Z] 08:40:23     INFO -  #  define VZERO() ((vector float) vec_splat_u8(0))
[task 2021-12-01T08:40:23.020Z] 08:40:23     INFO -                                    ^
[task 2021-12-01T08:40:23.020Z] 08:40:23    ERROR -  /builds/worker/checkouts/gecko/third_party/libwebrtc/third_party/pffft/src/pffft.c:226:9: error: implicit declaration of function 'vec_add' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
[task 2021-12-01T08:40:23.020Z] 08:40:23     INFO -    t.v = VADD(a1.v, a2.v);
[task 2021-12-01T08:40:23.021Z] 08:40:23     INFO -          ^
[task 2021-12-01T08:40:23.021Z] 08:40:23     INFO -  /builds/worker/checkouts/gecko/third_party/libwebrtc/third_party/pffft/src/pffft.c:107:21: note: expanded from macro 'VADD'
[task 2021-12-01T08:40:23.021Z] 08:40:23     INFO -  #  define VADD(a,b) vec_add(a,b)
[task 2021-12-01T08:40:23.021Z] 08:40:23     INFO -                      ^
[task 2021-12-01T08:40:23.021Z] 08:40:23    ERROR -  /builds/worker/checkouts/gecko/third_party/libwebrtc/third_party/pffft/src/pffft.c:226:7: error: assigning to 'v4sf' (vector of 4 'float' values) from incompatible type 'int'
[task 2021-12-01T08:40:23.021Z] 08:40:23     INFO -    t.v = VADD(a1.v, a2.v);
[task 2021-12-01T08:40:23.021Z] 08:40:23     INFO -        ^ ~~~~~~~~~~~~~~~~

Mike, what OS/compiler is that? I had no such issues on Fedora 34 with gcc 11.2.1

https://hg.mozilla.org/mozilla-central/rev/8bb1f58ba72b

Status: ASSIGNED → RESOLVED
Closed: 3 years ago
status-firefox96: affected → fixed
Resolution: --- → FIXED
Target Milestone: --- → 96 Branch

(In reply to Dan Horák from comment #17)

Mike, what OS/compiler is that? I had no such issues on Fedora 34 with gcc 11.2.1

clang 13 linux.

The difference is that GCC doesn't define __ppc__ or __ppc64__ (but rather __powerpc__ and __powerpc64__) so it doesn't try to enable the altivec code...
The pfft BUILD.gn file explicitly sets PFFFT_SIMD_DISABLE on a bunch of platforms (including ppc64), and moz.build doesn't bother to do that.

Dan, if you replace __ppc__ and __ppc64__ with __powerpc__ and __powerpc64__ in pffft.c, the same build error happens with GCC.

Flags: needinfo?(dan)

Adding #include <altivec.h> fixes the build with clang, but not GCC.

it might need an explicit "-maltivec" passed to gcc, I will look in the morning ...

Flags: needinfo?(dan)

the "ppc64" cpu in BUILD.gn might the big endian ppc64 variant, not the "modern" ppc64le ... in case the pffft vector code expects a little endian arch

and my additions are in https://bitbucket.org/jpommier/pffft/pull-requests/3

Has Regression Range: --- → yes

Is there any reson why PRs 2 and 3 from jpommier/pffft still didn't make their way into latest Firefox?
I can compile Firefox 97 without problems with gcc (Gentoo Linux ppc64le with 4K kernel) but in order to use clang I had to resort to these patches.
Even though it compiled fine with clang, now I'm getting the following error message: "The bookmarks and history system will not be functional".
I've tried to reboot and I've erased the profile, but it didn't help.
Do you apply any other patches with clang?

(In reply to Niccolò Belli from comment #27)

Is there any reson why PRs 2 and 3 from jpommier/pffft still didn't make their way into latest Firefox?
I can compile Firefox 97 without problems with gcc (Gentoo Linux ppc64le with 4K kernel) but in order to use clang I had to resort to these patches.
Even though it compiled fine with clang, now I'm getting the following error message: "The bookmarks and history system will not be functional".
I've tried to reboot and I've erased the profile, but it didn't help.
Do you apply any other patches with clang?

Do you use system sqlite3 or the bundled one? I had a similar issue that I resolved with https://svnweb.freebsd.org/ports?view=revision&revision=556308

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: