Closed Bug 1588538 Opened 5 years ago Closed 4 years ago

Use the new Windows dump_syms in Firefox local builds

Categories

(Toolkit :: Crash Reporting, task)

task
Not set
normal

Tracking

()

RESOLVED FIXED
mozilla74
Tracking Status
firefox74 --- fixed

People

(Reporter: gsvelto, Assigned: gsvelto)

References

(Blocks 1 open bug)

Details

Attachments

(2 files)

The new Windows-specific dump_syms is ready for action so we'd like to get it into Firefox builds ASAP but there's a couple of issues to solve for that. For starters it's not in mozilla-central and we most likely don't want it there because it depends on a bunch of Rust crates there's no point in vendoring and because ideally we'd always want to use the upstream version. We shipped the old dump_syms in mozilla-central because - among other things - we wanted it to be in-sync with breakpad and because we couldn't cross-compile it but both things are not an issue anymore.

Ideally we'd build it externally and provide it via ./mach bootstrap or only via ./mach buildsymbols if it's not available yet.

Summary: Use the new Windows dump_syms in Firefox builds → Use the new Windows dump_syms in Firefox local builds

Working on this

Assignee: nobody → gsvelto
Status: NEW → ASSIGNED
Attachment #9115738 - Attachment description: Bug 1588538 - Use the new Windows dump_syms tool to dump symbols in local builds → Bug 1588538 - Use the new Windows dump_syms tool to dump symbols in local builds r=glandium

The configure check isn't working well in automation: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=284359069&repo=try&lineNumber=1035-1045

I'll fix it before landing.

It works, you just aren't setting DUMP_SYMS in the mozconfigs.

(In reply to Mike Hommey [:glandium] from comment #4)

It works, you just aren't setting DUMP_SYMS in the mozconfigs.

Yeah, silly me. I noticed that I can also simplify the check, one more round of testing.

Almost there, only the plain builds are broken, the rest seems to be working:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=fc32ef908cab5e32be4eb3ac971c0f3305ec19f3&selectedJob=284654269

Try is green, the plain builds were using a separate mozconfig that was still missing the DUMP_SYMS definition. Time to land.

Pushed by gsvelto@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/5d0b75e5f795
Use the new Windows dump_syms tool to dump symbols in local builds r=glandium

Backed out for DUMP_SYMS bustage

backout: https://hg.mozilla.org/integration/autoland/rev/17ad5c46e7dd41dfb20cc2b08cd5b1eb14392cf1

push: https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&revision=5d0b75e5f7957adea117f88df18f8bcb7d80d505&searchStr=Windows%2C2012%2CAArch64%2Copt%2Cbuild-win64-aarch64-eme%2Fopt%2C%28Be%29&selectedJob=284827196

failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=284827196&repo=autoland&lineNumber=861

[task 2020-01-14T11:46:27.653Z] 11:46:27 INFO - Error processing command. Ignoring because optional. (optional:packages.txt:comm/build/virtualenv_packages.txt)
[task 2020-01-14T11:46:27.653Z] 11:46:27 INFO - z:/task_1578998329/build/src/obj-firefox/virtualenvs/init_py3/Scripts/python.exe (3.6.5)
[task 2020-01-14T11:46:27.653Z] 11:46:27 INFO - checking for vcs source checkout... hg
[task 2020-01-14T11:46:27.978Z] 11:46:27 INFO - checking for a shell... C:/mozilla-build/msys/bin/sh.exe
[task 2020-01-14T11:46:27.978Z] 11:46:27 INFO - checking for host system type... x86_64-pc-mingw32
[task 2020-01-14T11:46:28.037Z] 11:46:28 INFO - checking for target system type... aarch64-windows-mingw32
[task 2020-01-14T11:46:28.243Z] 11:46:28 INFO - checking whether cross compiling... yes
[task 2020-01-14T11:46:28.405Z] 11:46:28 ERROR - Traceback (most recent call last):
[task 2020-01-14T11:46:28.405Z] 11:46:28 INFO - File "z:/task_1578998329/build/src/configure.py", line 170, in <module>
[task 2020-01-14T11:46:28.405Z] 11:46:28 INFO - sys.exit(main(sys.argv))
[task 2020-01-14T11:46:28.405Z] 11:46:28 INFO - File "z:/task_1578998329/build/src/configure.py", line 46, in main
[task 2020-01-14T11:46:28.405Z] 11:46:28 INFO - sandbox.run(os.path.join(os.path.dirname(file), 'moz.configure'))
[task 2020-01-14T11:46:28.405Z] 11:46:28 INFO - File "z:\task_1578998329\build\src\python\mozbuild\mozbuild\configure_init
.py", line 454, in run
[task 2020-01-14T11:46:28.405Z] 11:46:28 INFO - self.value_for(option)
[task 2020-01-14T11:46:28.405Z] 11:46:28 INFO - File "z:\task_1578998329\build\src\python\mozbuild\mozbuild\configure_init
.py", line 544, in _value_for
[task 2020-01-14T11:46:28.405Z] 11:46:28 INFO - return self.value_for_option(obj)
[task 2020-01-14T11:46:28.405Z] 11:46:28 INFO - File "z:\task_1578998329\build\src\python\mozbuild\mozbuild\util.py", line 1017, in method_call
[task 2020-01-14T11:46:28.405Z] 11:46:28 INFO - cache[args] = self.func(instance, *args)
[task 2020-01-14T11:46:28.405Z] 11:46:28 INFO - File "z:\task_1578998329\build\src\python\mozbuild\mozbuild\configure_init
.py", line 607, in _value_for_option
[task 2020-01-14T11:46:28.405Z] 11:46:28 INFO - % option_string.split('=', 1)[0])
[task 2020-01-14T11:46:28.406Z] 11:46:28 INFO - mozbuild.configure.options.InvalidOptionError: DUMP_SYMS is not available in this configuration
[task 2020-01-14T11:46:28.424Z] 11:46:28 INFO - *** Fix above errors and then restart with
[task 2020-01-14T11:46:28.424Z] 11:46:28 INFO - "./mach build"
[task 2020-01-14T11:46:28.424Z] 11:46:28 INFO - client.mk:111: recipe for target 'configure' failed
[task 2020-01-14T11:46:28.424Z] 11:46:28 INFO - mozmake.EXE: *** [configure] Error 1
[task 2020-01-14T11:46:28.491Z] 11:46:28 ERROR - Return code: 2
[task 2020-01-14T11:46:28.491Z] 11:46:28 WARNING - setting return code to 2
[task 2020-01-14T11:46:28.491Z] 11:46:28 FATAL - 'mach build -v' did not run successfully. Please check log for errors.
[task 2020-01-14T11:46:28.491Z] 11:46:28 FATAL - Running post_fatal callback...
[task 2020-01-14T11:46:28.491Z] 11:46:28 FATAL - Exiting -1

Flags: needinfo?(gsvelto)

Ouch, I forgot about AArch64/Windows, will look into this now.

Flags: needinfo?(gsvelto)

Needs an unset DUMP_SYMS in build/mozconfig.no-compile.

Thanks, I'll do that.

Awww, I didn't rebase my fixup, that try run doesn't have the main patch applied. Nevermind, sorry for the noise, back to testing.

Pushed by gsvelto@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/893e9fda4bf3
Use the new Windows dump_syms tool to dump symbols in local builds r=glandium
Status: ASSIGNED → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla74

./mach bootstrap isn't working anymore with this change:

Error running mach:

    ['bootstrap']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

ImportError: cannot import name dump_syms

  File "d:\mozilla-unified\python/mozboot/mozboot/mach_commands.py", line 44, in bootstrap
    bootstrapper.bootstrap()
  File "d:\mozilla-unified\python/mozboot\mozboot\bootstrap.py", line 537, in bootstrap
    checkout_root)
  File "d:\mozilla-unified\python/mozboot\mozboot\bootstrap.py", line 403, in maybe_install_private_packages_or_exit
    self.instance.ensure_dump_syms_packages(state_dir, checkout_root)
  File "d:\mozilla-unified\python/mozboot\mozboot\mozillabuild.py", line 140, in ensure_dump_syms_packages
    from mozboot import dump_syms

per glandium, there's a file missing in the checkin - dump_syms.py. It should have this content in it:

# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.

from __future__ import absolute_import, print_function, unicode_literals

WIN64_DUMP_SYMS = 'win64-dump-syms'

Sorry for the breakage everybody and thanks for helping out Mike. I don't know why but ./mach bootstrap magically worked on my machine so I hadn't noticed the issue :-/

Regressions: 1614152
Blocks: 1751810
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: