mozharness unittest minidump stackwalk not working

RESOLVED FIXED

Status

Release Engineering
Applications: MozharnessCore
RESOLVED FIXED
6 years ago
4 years ago

People

(Reporter: aki, Assigned: aki)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [mozharness][unittest])

Attachments

(2 attachments, 2 obsolete attachments)

(Assignee)

Description

6 years ago
17:15:57  WARNING -  TEST-UNEXPECTED-FAIL | /tests/content/html/content/test/test_fullscreen-api.html | Exited with code 1 during test run
17:15:57  WARNING -  This is a harness error.
17:15:57     INFO -  INFO | automation.py | Application ran for: 0:08:43.839605
17:15:57     INFO -  INFO | automation.py | Reading PID log: /var/folders/vs/s48lhbg13cx0j819ycft495000000w/T/tmpibmhxJpidlog
17:15:57     INFO -  Downloading symbols from: http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/cedar-macosx64/1351548997/firefox-19.0a1.en-US.mac.crashreporter-symbols.zip
17:15:59     INFO -  PROCESS-CRASH | /tests/content/html/content/test/test_fullscreen-api.html | application crashed (minidump found)
17:15:59     INFO -  Crash dump filename: /var/folders/vs/s48lhbg13cx0j819ycft495000000w/T/tmp2GVgs5/minidumps/E4651AEA-78D1-438C-9FB9-A222016BAD4F.dmp
17:15:59     INFO -  MINIDUMP_STACKWALK not set, can't process dump.
17:16:00     INFO -  WARNING | automationutils.processLeakLog() | refcount logging is off, so leaks can't be detected!
17:16:00     INFO -  INFO | runtests.py | Running tests: end.
17:16:00    ERROR - Return code: 1
17:16:00     INFO - TinderboxPrint: mochitest-plain1<br/><em class="testfail">T-FAIL</em>

https://tbpl.mozilla.org/php/getParsedLog.php?id=16567097&tree=Cedar
(Assignee)

Comment 1

6 years ago
Created attachment 676436 [details] [diff] [review]
(wip) partial config patch

I can either check in 14mb of binaries into mozharness, or depend on build/tools again.

This patch adds build/tools as a repo, and the minidump paths to the unittest config files.  Still to do: re-add hg support to desktop_unittest.py (including windows hg exe) and use the minidump* config items in env when running the tests.
(Assignee)

Comment 2

6 years ago
Created attachment 676870 [details] [diff] [review]
enable minidump on unittests

Need to test in staging.
Attachment #676436 - Attachment is obsolete: true
(Assignee)

Comment 3

6 years ago
[16:47]	<edmorley>	aki|away: is there a bug filed on making mozharness output the PROCESS-CRASH lines as an error, not INFO ? (eg https://tbpl.mozilla.org/php/getParsedLog.php?id=16604974&tree=Cedar)

This should do it:

diff --git a/scripts/desktop_unittest.py b/scripts/desktop_unittest.py
--- a/scripts/desktop_unittest.py
+++ b/scripts/desktop_unittest.py
@@ -6,25 +6,26 @@
 # ***** END LICENSE BLOCK *****
 """desktop_unittest.py
 The goal of this is to extract desktop unittestng from buildbot's factory.py
 
 author: Jordan Lund
 """
 
 import os
+import re
 import sys
 import copy
 import shutil
 
 # load modules from parent dir
 sys.path.insert(1, os.path.dirname(sys.path[0]))
 
 from mozharness.base.errors import BaseErrorList
-from mozharness.base.log import INFO
+from mozharness.base.log import INFO, ERROR
 from mozharness.base.vcs.vcsbase import MercurialScript
 from mozharness.mozilla.testing.testbase import TestingMixin, testing_config_op
 from mozharness.mozilla.testing.unittest import DesktopUnittestOutputParser
 
 SUITE_CATEGORIES = ['mochitest', 'reftest', 'xpcshell']
 
 
 # DesktopUnittest {{{1
@@ -315,19 +316,23 @@ class DesktopUnittest(TestingMixin, Merc
                 cmd = abs_base_cmd
                 replace_dict = {
                     'abs_app_dir': abs_app_dir,
                 }
                 for arg in suites[suite]:
                     cmd.append(arg % replace_dict)
                 suite_name = suite_category + '-' + suite
                 tbpl_status, log_level = None, None
+                error_list = BaseErrorList + [{
+                    'regex': re.compile(r'''PROCESS-CRASH.*minidump found'''),
+                    'level': ERROR,
+                }]
                 parser = DesktopUnittestOutputParser(suite_category,
                                                      config=self.config,
-                                                     error_list=BaseErrorList,
+                                                     error_list=error_list,
                                                      log_obj=self.log_obj)
                 env = {}
                 if c.get('minidump_stackwalk_path'):
                     env['MINIDUMP_STACKWALK'] = c['minidump_stackwalk_path']
                 if c.get('minidump_save_path'):
                     env['MINIDUMP_SAVE_PATH'] = c['minidump_save_path']
                 env = self.query_env(partial_env=env, log_level=INFO)
                 return_code = self.run_command(cmd, cwd=dirs['abs_work_dir'],

Comment 4

6 years ago
Thank you :-)
(Assignee)

Comment 5

6 years ago
Created attachment 678166 [details] [diff] [review]
unittest minidump + improved error list
Attachment #676870 - Attachment is obsolete: true
Attachment #678166 - Flags: review?(rail)
Attachment #678166 - Flags: review?(rail) → review+
(Assignee)

Comment 6

6 years ago
Comment on attachment 678166 [details] [diff] [review]
unittest minidump + improved error list

http://hg.mozilla.org/build/mozharness/rev/5c5cd39b1131
Attachment #678166 - Flags: checked-in+
(Assignee)

Updated

6 years ago
Status: NEW → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
(Assignee)

Comment 7

6 years ago
14:23:17     INFO -  MINIDUMP_STACKWALK binary not found: C:\talos-slave\test\build/tools/breakpad/win32/minidump_stackwalk
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(Assignee)

Comment 8

6 years ago
Created attachment 681266 [details] [diff] [review]
fix minidump stackwalk path on windows
Assignee: nobody → aki
Attachment #681266 - Flags: review?(armenzg)

Updated

6 years ago
Attachment #681266 - Flags: review?(armenzg) → review+
(Assignee)

Comment 9

6 years ago
Comment on attachment 681266 [details] [diff] [review]
fix minidump stackwalk path on windows

http://hg.mozilla.org/build/mozharness/rev/230bbacd4d0e
Attachment #681266 - Flags: checked-in+
(Assignee)

Updated

6 years ago
Status: REOPENED → RESOLVED
Last Resolved: 6 years ago6 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering

Updated

4 years ago
Component: General Automation → Mozharness
You need to log in before you can comment on or make changes to this bug.