Closed Bug 1578090 Opened 5 years ago Closed 5 years ago

Perma OS X ccov subprocess.CalledProcessError: Command '[u'/builds/worker/workspace/build/src/obj-firefox/_virtualenvs/init/bin/python', u'/builds/worker/workspace/build/src/toolkit/crashreporter/tools/symbolstore.py', '-c',

Categories

(Firefox Build System :: General, defect, P5)

Product:
Firefox Build System
Component:
General
Type:
defect

Tracking

(firefox-esr60 unaffected, firefox-esr68 unaffected, firefox69 unaffected, firefox70 fixed)

Status:
RESOLVED FIXED
Milestone:
mozilla70
Tracking Flags:
Tracking Status
firefox-esr60 --- unaffected
firefox-esr68 --- unaffected
firefox69 --- unaffected
firefox70 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: froydnj)

References

(Regression)

Details

(Keywords: intermittent-failure, regression)

Filed by: aiakab [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=264449591&repo=mozilla-central
Full log: https://queue.taskcluster.net/v1/task/EiW93UWDSVGC-6l4z_Y5Ug/runs/0/artifacts/public/logs/live_backing.log

[task 2019-09-01T10:32:18.891Z] 10:32:18 INFO - Starting Mac pre-processing on file: /builds/worker/workspace/build/src/obj-firefox/toolkit/components/telemetry/tests/libmodules-test.dylib
[task 2019-09-01T10:32:18.894Z] 10:32:18 INFO - Running Mac pre-processing on file: /builds/worker/workspace/build/src/obj-firefox/toolkit/components/telemetry/tests/libmodules-test.dylib
[task 2019-09-01T10:32:18.895Z] 10:32:18 INFO - /builds/worker/workspace/build/src/build/macosx/llvm-dsymutil --arch=x86_64 /builds/worker/workspace/build/src/obj-firefox/toolkit/components/telemetry/tests/libmodules-test.dylib
[task 2019-09-01T10:32:18.895Z] 10:32:18 INFO - Finished processing /builds/worker/workspace/build/src/obj-firefox/toolkit/components/telemetry/tests/libmodules-test.dylib in 0.01s
[task 2019-09-01T10:32:18.896Z] 10:32:18 INFO - Beginning work for file: /builds/worker/workspace/build/src/obj-firefox/toolkit/components/telemetry/tests/libmodules-test.dylib
[task 2019-09-01T10:32:18.896Z] 10:32:18 INFO - Processing file: /builds/worker/workspace/build/src/obj-firefox/toolkit/components/telemetry/tests/libmodules-test.dylib
[task 2019-09-01T10:32:18.896Z] 10:32:18 INFO - /builds/worker/workspace/build/src/obj-firefox/dist/host/bin/dump_syms -a x86_64 -g /builds/worker/workspace/build/src/obj-firefox/toolkit/components/telemetry/tests/libmodules-test.dylib.dSYM /builds/worker/workspace/build/src/obj-firefox/toolkit/components/telemetry/tests/libmodules-test.dylib
[task 2019-09-01T10:32:18.896Z] 10:32:18 INFO - Unexpected error: dump_syms failed to produce the expected output
[task 2019-09-01T10:32:18.896Z] 10:32:18 INFO - ----------
[task 2019-09-01T10:32:18.896Z] 10:32:18 INFO - return code: -6
[task 2019-09-01T10:32:18.896Z] 10:32:18 INFO - ----------
[task 2019-09-01T10:32:18.897Z] 10:32:18 INFO - first line of output:
[task 2019-09-01T10:32:18.897Z] 10:32:18 INFO - ----------
[task 2019-09-01T10:32:18.897Z] 10:32:18 INFO - stderr: dump_syms: /builds/worker/workspace/build/src/toolkit/crashreporter/google-breakpad/src/common/mac/dump_syms.cc:443: bool google_breakpad::DumpSymbols::ReadDwarf(google_breakpad::Module *, const mach_o::Reader &, const mach_o::SectionMap &, bool) const: Assertion debug_info_entry != file_context.section_map().end()' failed. [task 2019-09-01T10:32:18.897Z] 10:32:18 ERROR - Traceback (most recent call last): [task 2019-09-01T10:32:18.897Z] 10:32:18 INFO - File "/builds/worker/workspace/build/src/toolkit/crashreporter/tools/symbolstore.py", line 983, in <module> [task 2019-09-01T10:32:18.897Z] 10:32:18 INFO - main() [task 2019-09-01T10:32:18.898Z] 10:32:18 INFO - File "/builds/worker/workspace/build/src/toolkit/crashreporter/tools/symbolstore.py", line 979, in main [task 2019-09-01T10:32:18.898Z] 10:32:18 INFO - dumper.Process(args[2], options.count_ctors) [task 2019-09-01T10:32:18.898Z] 10:32:18 INFO - File "/builds/worker/workspace/build/src/toolkit/crashreporter/tools/symbolstore.py", line 492, in Process [task 2019-09-01T10:32:18.901Z] 10:32:18 INFO - self.ProcessFile(file_to_process, count_ctors=count_ctors) [task 2019-09-01T10:32:18.901Z] 10:32:18 INFO - File "/builds/worker/workspace/build/src/toolkit/crashreporter/tools/symbolstore.py", line 843, in ProcessFile [task 2019-09-01T10:32:18.902Z] 10:32:18 INFO - count_ctors=count_ctors) [task 2019-09-01T10:32:18.902Z] 10:32:18 INFO - File "/builds/worker/workspace/build/src/toolkit/crashreporter/tools/symbolstore.py", line 507, in ProcessFile [task 2019-09-01T10:32:18.902Z] 10:32:18 INFO - count_ctors=count_ctors) [task 2019-09-01T10:32:18.902Z] 10:32:18 INFO - File "/builds/worker/workspace/build/src/toolkit/crashreporter/tools/symbolstore.py", line 636, in ProcessFileWork [task 2019-09-01T10:32:18.902Z] 10:32:18 INFO - raise RuntimeError('\n----------\n'.join(message)) [task 2019-09-01T10:32:18.902Z] 10:32:18 INFO - RuntimeError: dump_syms failed to produce the expected output [task 2019-09-01T10:32:18.903Z] 10:32:18 INFO - ---------- [task 2019-09-01T10:32:18.903Z] 10:32:18 INFO - return code: -6 [task 2019-09-01T10:32:18.903Z] 10:32:18 INFO - ---------- [task 2019-09-01T10:32:18.903Z] 10:32:18 INFO - first line of output: [task 2019-09-01T10:32:18.903Z] 10:32:18 INFO - ---------- [task 2019-09-01T10:32:18.904Z] 10:32:18 INFO - stderr: dump_syms: /builds/worker/workspace/build/src/toolkit/crashreporter/google-breakpad/src/common/mac/dump_syms.cc:443: bool google_breakpad::DumpSymbols::ReadDwarf(google_breakpad::Module *, const mach_o::Reader &, const mach_o::SectionMap &, bool) const: Assertiondebug_info_entry != file_context.section_map().end()' failed.
[task 2019-09-01T10:32:18.904Z] 10:32:18 ERROR - Traceback (most recent call last):
[task 2019-09-01T10:32:18.904Z] 10:32:18 INFO - File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
[task 2019-09-01T10:32:18.904Z] 10:32:18 INFO - "main", fname, loader, pkg_name)
[task 2019-09-01T10:32:18.904Z] 10:32:18 INFO - File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
[task 2019-09-01T10:32:18.905Z] 10:32:18 INFO - exec code in run_globals
[task 2019-09-01T10:32:18.905Z] 10:32:18 INFO - File "/builds/worker/workspace/build/src/python/mozbuild/mozbuild/action/dumpsymbols.py", line 96, in <module>
[task 2019-09-01T10:32:18.905Z] 10:32:18 INFO - sys.exit(main(sys.argv[1:]))
[task 2019-09-01T10:32:18.905Z] 10:32:18 INFO - File "/builds/worker/workspace/build/src/python/mozbuild/mozbuild/action/dumpsymbols.py", line 92, in main
[task 2019-09-01T10:32:18.905Z] 10:32:18 INFO - args.count_ctors)
[task 2019-09-01T10:32:18.905Z] 10:32:18 INFO - File "/builds/worker/workspace/build/src/python/mozbuild/mozbuild/action/dumpsymbols.py", line 73, in dump_symbols
[task 2019-09-01T10:32:18.905Z] 10:32:18 INFO - out_files = subprocess.check_output(args)
[task 2019-09-01T10:32:18.905Z] 10:32:18 INFO - File "/usr/lib/python2.7/subprocess.py", line 573, in check_output
[task 2019-09-01T10:32:18.906Z] 10:32:18 INFO - raise CalledProcessError(retcode, cmd, output=output)
[task 2019-09-01T10:32:18.906Z] 10:32:18 INFO - subprocess.CalledProcessError: Command '[u'/builds/worker/workspace/build/src/obj-firefox/_virtualenvs/init/bin/python', u'/builds/worker/workspace/build/src/toolkit/crashreporter/tools/symbolstore.py', '-c', '-a', u'x86_64', '--vcs-info', u'--install-manifest=/builds/worker/workspace/build/src/obj-firefox/_build_manifests/install/dist_include,/builds/worker/workspace/build/src/obj-firefox/dist/include', '-s', u'/builds/worker/workspace/build/src', u'/builds/worker/workspace/build/src/obj-firefox/dist/host/bin/dump_syms', u'/builds/worker/workspace/build/src/obj-firefox/dist/crashreporter-symbols', '/builds/worker/workspace/build/src/obj-firefox/toolkit/components/telemetry/tests/libmodules-test.dylib']' returned non-zero exit status 1
[task 2019-09-01T10:32:18.906Z] 10:32:18 INFO - /builds/worker/workspace/build/src/config/rules.mk:775: recipe for target 'libmodules-test.dylib_syms.track' failed
[task 2019-09-01T10:32:18.906Z] 10:32:18 ERROR - make[4]: *** [libmodules-test.dylib_syms.track] Error 1
[task 2019-09-01T10:32:18.907Z] 10:32:18 INFO - make[4]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/toolkit/components/telemetry/tests'
[task 2019-09-01T10:32:18.907Z] 10:32:18 INFO - /builds/worker/workspace/build/src/config/recurse.mk:74: recipe for target 'toolkit/components/telemetry/tests/syms' failed
[task 2019-09-01T10:32:18.907Z] 10:32:18 ERROR - make[3]: *** [toolkit/components/telemetry/tests/syms] Error 2
[task 2019-09-01T10:32:18.907Z] 10:32:18 INFO - make[3]: *** Waiting for unfinished jobs....
[task 2019-09-01T10:32:18.907Z] 10:32:18 INFO - make[4]: Entering directory '/builds/worker/workspace/build/src/obj-firefox/toolkit/components/ctypes/tests'
[task 2019-09-01T10:32:18.908Z] 10:32:18 INFO - /builds/worker/workspace/build/src/obj-firefox/_virtualenvs/init/bin/python -m mozbuild.action.dumpsymbols /builds/worker/workspace/build/src/obj-firefox/toolkit/components/ctypes/tests/libjsctypes-test.dylib /builds/worker/workspace/build/src/obj-firefox/toolkit/components/ctypes/tests/libjsctypes-test.dylib_syms.track
[task 2019-09-01T10:32:18.908Z] 10:32:18 INFO - Running: /builds/worker/workspace/build/src/obj-firefox/_virtualenvs/init/bin/python /builds/worker/workspace/build/src/toolkit/crashreporter/tools/symbolstore.py -c -a x86_64 --vcs-info --install-manifest=/builds/worker/workspace/build/src/obj-firefox/_build_manifests/install/dist_include,/builds/worker/workspace/build/src/obj-firefox/dist/include -s /builds/worker/workspace/build/src /builds/worker/workspace/build/src/obj-firefox/dist/host/bin/dump_syms /builds/worker/workspace/build/src/obj-firefox/dist/crashreporter-symbols /builds/worker/workspace/build/src/obj-firefox/toolkit/components/ctypes/tests/libjsctypes-test.dylib
[task 2019-09-01T10:32:18.908Z] 10:32:18 INFO - Starting Mac pre-processing on file: /builds/worker/workspace/build/src/obj-firefox/toolkit/components/ctypes/tests/libjsctypes-test.dylib

Nathan can you please take a look?

Flags: needinfo?(nfroyd)
Regressed by: 1576748

I don't understand how this can fail on central, but succeed on autoland. Regardless, it looks like the patch in question has been backed out.

Status: NEW → RESOLVED
Closed: 5 years ago
Flags: needinfo?(nfroyd)
Resolution: --- → FIXED
Component: Crash Reporting → General
Product: Toolkit → Firefox Build System
Summary: Intermittent subprocess.CalledProcessError: Command '[u'/builds/worker/workspace/build/src/obj-firefox/_virtualenvs/init/bin/python', u'/builds/worker/workspace/build/src/toolkit/crashreporter/tools/symbolstore.py', '-c', '-a', u'x86_64', '--vcs-info... → Perma OS X ccov subprocess.CalledProcessError: Command '[u'/builds/worker/workspace/build/src/obj-firefox/_virtualenvs/init/bin/python', u'/builds/worker/workspace/build/src/toolkit/crashreporter/tools/symbolstore.py', '-c',
Assignee: nobody → nfroyd
status-firefox69: --- → unaffected
status-firefox70: --- → fixed
status-firefox-esr60: --- → unaffected
status-firefox-esr68: --- → unaffected
Target Milestone: --- → mozilla70
Has Regression Range: --- → yes
You need to log in before you can comment on or make changes to this bug.