1788569 - Move abseil-cpp to the top-level third_party directory

Status: NEW

(Core :: General, task)

Description

[Ryan VanderMeulen [:RyanVM]]

Right now, we have an import of abseil-cpp that lives in third_party/libwebrtc/third_party/abseil-cpp. This makes sense for the time-being as libwebrtc is the only in-tree consumer of it at the moment. However, we have other in-tree libraries that are also moving towards requiring abseil-cpp, such as protobuf and googletest. It's likely that over time, others will follow suit.

It will make life easier for taking those future updates if abseil-cpp lives in a more generic location rather than inside the libwebrtc import.

Comment 1

[Mike Hommey [:glandium]]

This isn't going to be easy, considering abseil-cpp is defined as a dependency through webrtc's gn files.

Comment 2

[Nika Layzell [:nika] (ni? for response)]

We may also want to consider vendoring the ipcz library (bug 1795083), which also pulls a dependency on abseil-cpp and is built using gn. I believe we'd need to figure out something like this, allowing us to share abseil between webrtc and ipcz, in order to do something like that.

Comment 3

[Nika Layzell [:nika] (ni? for response)]

Attachment: WIP: Bug 1788569 - Part 1: Vendor abseil-cpp in third_party using moz.yaml

This is a direct vendoring, without any build-system integration.

The vendoring is currently set up to use an LTS version of abseil in order to
reduce the frequency of updates.

The unpatched version of abseil-cpp from github is used, rather than chromium's
patched version, as gn will not be used to build the library.

Comment 4

[Nika Layzell [:nika] (ni? for response)]

Attachment: WIP: Bug 1788569 - Part 2: Automatically generate moz.build files for building abseil

This is done by running the abseil's simple BUILD.bazel scripts in a basic
sandbox, and collecting details about which resources need to be built. This is
possible because bazel's configuration language, starlark, is a dialect of
Python and can therefore be evaluated in a python sandbox relatively
straightforwardly. This is somewhat similar to how moz.build files are
evaluated by mozillabuild.

The "minibazel" sandbox is very abseil-specific, and will not support more
complex projects, such as those with dependencies or significant
platform-specific configuration. Binary and test targets are completely
ignored.

It is somewhat likely that the build scripts will remain somewhat simple for
abseil due to its intent to support multiple build systems, though the
minibazel sandbox will certainly need to be expanded in the future.

This script is set up to be run whenever a new version of abseil is vendored.

Depends on D159533

Comment 5

[Nika Layzell [:nika] (ni? for response)]

I've attached some extremely-sketchy WIP patches which build abseil directly in third_party and should support automatic updates through ./mach vendor. This is done by evaluating the BUILD.bazel files as python files in a sandbox, and collecting the outputs into moz.build files. This doesn't do anything which would actually swap out the version of abseil which webrtc is using, but I figured I should post this thing in case it's useful for whoever actually decides to tackle this.

Comment 6

[Ryan VanderMeulen [:RyanVM]]

FYI, protobuf v22.0 is officially released now, so this bug blocks our ability to take any future updates of that library.

Comment 7

[Nico Grunbaum [:ng, @chew:mozilla.org]]

FYI, I filed [0], because we still support building with clang v5[1], which abseil does not support. Here is the link to their support matrix [2]. IIRC, until somewhat recently we were using a version of abseil that still supported being built with clang 3.4.

With libwebrtc, we are having to apply various workarounds with abseil, e.g. [3][4] , that may need to be carried forward with whatever happens here. If you need help turning them into patch files that updatebot can apply, or weeding through them, please feel free to reach out to the WebRTC team, tag #webrtc-reviewers in reviews, etc.

[0] https://github.com/abseil/abseil-cpp/issues/1309
[1] https://searchfox.org/mozilla-central/rev/66aa740e65a343659a7446b890145781f1b6a344/taskcluster/ci/build/linux-base-toolchains.yml#39
[2] https://github.com/google/oss-policies-info/blob/main/foundational-cxx-support-matrix.md
[3] https://bugzilla.mozilla.org/show_bug.cgi?id=1797157
[4] https://bugzilla.mozilla.org/show_bug.cgi?id=1795982

Comment 8

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::WebRTC' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.