1442783 - minidump_stackwalk not being invoked correctly on Mac mochitest-chrome test: "MINIDUMP_STACKWALK binary not found: /usr/local/bin/linux64-minidump_stackwalk"

Status: RESOLVED FIXED

(Testing :: General, defect, P1)

Description

[Markus Stange [:mstange]]

Here's a test which crashed, but for which there is no crash stack:

https://treeherder.mozilla.org/logviewer.html#?job_id=165551225&repo=try&lineNumber=2580

Instead, it has the message "MINIDUMP_STACKWALK binary not found: /usr/local/bin/linux64-minidump_stackwalk"

This is job is running on macOS, so maybe it's looking for the wrong minidump_stackwalk program.

Comment 1

[Geoff Brown [:gbrown]]

There's a mac version of minidump_stackwalk that should be used instead. It seems to be correctly configured in 

https://dxr.mozilla.org/mozilla-central/rev/df0e967f8e8fa30a69338008b6ee296bf5e9c545/testing/mozharness/configs/unittests/mac_unittest.py#243

but since bug 1410250 landed, that's being over-ridden by the linux config in remove_executables.py.

Comment 2

[Geoff Brown [:gbrown]]

08:27:56     INFO - Run as mozharness/scripts/desktop_unittest.py --cfg mozharness/configs/unittests/mac_unittest.py --cfg mozharness/configs/remove_executables.py --mochitest-suite=chrome --no-read-buildbot-config --installer-url https://queue.taskcluster.net/v1/task/CdL09yMeSf655Hd201Njuw/artifacts/public/build/target.dmg --test-packages-url https://queue.taskcluster.net/v1/task/CdL09yMeSf655Hd201Njuw/artifacts/public/build/target.test_packages.json --download-symbols true --mochitest-suite=chrome --total-chunk=3 --this-chunk=3
...
08:27:56     INFO -  'minidump_stackwalk_path': '/usr/local/bin/linux64-minidump_stackwalk',

Comment 3

[Geoff Brown [:gbrown]]

:jmaher - Do you know if the mac_unittest.py config is correct and appropriate for mac? Do you know why remove_executables.py is being used on mac?

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

this is a good question- I believe we should clean up many things like --no-read-buildbot-config and look over our mozharness/configs scripts now that we are all on taskcluster.

I honestly never understood the use of remove_executables.py- I just knew that it was needed in migrating to taskcluster.  Happy to see any of this change.

Comment 5

[Geoff Brown [:gbrown]]

The example from comment 0,

https://treeherder.mozilla.org/logviewer.html#?job_id=165551225&repo=try&lineNumber=2580

is actually mochitest-chrome, M(c3). I'm pretty sure browser-chrome is okay.

Comment 6

[Geoff Brown [:gbrown]]

Attachment: remove mac remove_executables from configs

I'm not sure how/why/when this fault was introduced, but it seems easy to correct. With this patch, remove_executables will no longer be used for any mac test task. Tests still seem to work okay, and minidump_stackwalk consistently points to the mac version:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=ed834f93ed6ee93748a3adfa0deb41547b2d2087

https://treeherder.mozilla.org/logviewer.html#?job_id=166063986&repo=try&lineNumber=104

INFO -  'minidump_stackwalk_path': 'macosx64-minidump_stackwalk'

Comment 7

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8956217 [details] [diff] [review]
remove mac remove_executables from configs

Review of attachment 8956217 [details] [diff] [review]:
-----------------------------------------------------------------

we should be able to do this someday for our other tests as well!

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/b5ecb6a24d73
Simplify and correct mozharness configuration for some macosx64 tests; r=jmaher

Comment 9

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/b5ecb6a24d73