1391408 - Build minidump_stackwalk binaries in toolchain tasks

Status: RESOLVED FIXED

(Firefox Build System :: General, enhancement)

Description

[(not currently active) Ted Mielczarek]

I have a script that builds the minidump_stackwalk binaries that we use in CI:
https://dxr.mozilla.org/mozilla-central/source/taskcluster/scripts/misc/minidump_stackwalk.sh

It currently builds binaries for all platforms in a specific Docker image, and I made it work in Taskcluster at some point, but we don't currently have a Taskcluster toolchain task to actually build those binaries. Instead we manually update the tooltool manifests here:
https://dxr.mozilla.org/mozilla-central/source/testing/config/tooltool-manifests

for example:
https://dxr.mozilla.org/mozilla-central/source/testing/config/tooltool-manifests/linux64/releng.manifest

Now that we have toolchain tasks as part of the task graph, we should make a toolchain task to build these binaries, and make test tasks get their minidump_stackwalk binaries from those tasks.

We could also make build tasks depend on these toolchain tasks to fix bug 1378765 in a better way. Currently build tasks get an old version of minidump_stackwalk because they pull it from the ancient build/tools repo.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Ted, given that bug 1313111 is fixed now is there something else which would block us from getting this script working in a toolchain task?

Comment 2

[(not currently active) Ted Mielczarek]

No, it just needs someone to do the work. It's a bit of a pain to build the binaries for Windows and Mac because they both need to be cross-compiled from Linux, but that should be fairly straightforward for Mac these days, and given that we have mingw32 builds of Firefox in CI, the Windows builds shouldn't be *too* hard.

Comment 3

[Mike Hommey [:glandium]]

Attachment: Bug 1391408 - Build minidump_stackwalk as a toolchain.

This produces the same executables (produced for the same platforms) as
those currently pulled from tooltool (modulo timestamps, maybe changes
since last manifest change, etc.). Unfortunately, as of currently, the
Windows variant needs to be cross-built with mingw because it doesn't
compile without some POSIX APIs that MSVC/Windows SDK don't provide.

One thing that is left out of this change is whether to be completely
accurate with the toolchain cache hash (requiring a large list of files
as resources, and making those built very frequently), whether we'd
rely on manual updates, or if we should go with completely uncached
tasks. This can be left for a followup, the tasks not being hooked up
to be actually used by other tasks yet.

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a58fad772b6f
Build minidump_stackwalk as a toolchain. r=froydnj

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/451d57ca6810
Add missing executable bit r=glandium. CLOSED TREE

Comment 6

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/a58fad772b6f
https://hg.mozilla.org/mozilla-central/rev/451d57ca6810