Closed Bug 563745 Opened 10 years ago Closed 10 years ago

Make automationutils.py's minidump processing able to send minidumps to minidump-stackwalk-server

Categories

(Testing :: General, defect)

defect
Not set

Tracking

(Not tracked)

RESOLVED FIXED
mozilla1.9.3a5

People

(Reporter: ted, Assigned: ted)

References

Details

Attachments

(1 file)

Currently automationutils.py knows how to run minidump_stackwalk locally to process minidumps resulting from crashes in our test suites. bug 561754 would like to offload this processing to a different machine, so that the build machines don't have to download the symbols. bug 563678 implemented a simple CGI that accepts a minidump and symbols URL via HTTP POST, and returns the stackwalk output in the body. We'll need to hack automationutils to be be able to use this service.
This modifies automationutils.py and allows it to use stackwalk.cgi for minidump processing. I've added catlee's poster library as a zip file for convenience, since posting multipart/form-data is pretty ugly in Python otherwise (and shuffling multiple files for a Python library around our tet harnesses is clunky).

To use:
1) Set MINIDUMP_STACKWALK_CGI=http://url/to/stackwalk.cgi in the environment, instead of MINIDUMP_STACKWALK
2) Run test harnesses with --symbols-path=http://url/to/symbols.zip pointing at the symbol package for the build being tested
3) ??
4) Profit
Attachment #443668 - Flags: review?(catlee)
Comment on attachment 443668 [details] [diff] [review]
Allow automationutils.py to use stackwalk.cgi

> def checkForCrashes(dumpDir, symbolsPath, testName=None):
>   stackwalkPath = os.environ.get('MINIDUMP_STACKWALK', None)
>+  stackwalkCGI = os.environ.get('MINIDUMP_STACKWALK_CGI', None)
>   # try to get the caller's filename if no test name is given
>   if testName is None:
>     try:
>@@ -111,14 +118,33 @@
>       # eat minidump_stackwalk errors
>       subprocess.call([stackwalkPath, d, symbolsPath], stderr=nullfd)
>       nullfd.close()
>+    elif stackwalkCGI and symbolsPath and isURL(symbolsPath):
>+      try:
>+        f = open(d, "rb")
>+        sys.path.append(os.path.join(os.path.dirname(__file__), "poster.zip"))
>+        from poster.encode import multipart_encode
>+        from poster.streaminghttp import register_openers
>+        import urllib2
>+        register_openers()
>+        datagen, headers = multipart_encode({"minidump": f,
>+                                             "symbols": symbolsPath})
>+        request = urllib2.Request(stackwalkCGI, datagen, headers)
>+        print urllib2.urlopen(request).read()
>+      finally:
>+        if f:
>+          f.close()

If open(d, "rb") fails for some reason, then f won't be set, and you'll get a NameError in the finally block.  Initialize f to None before the try block?
Attachment #443668 - Flags: review?(catlee) → review-
Pushed to m-c:
http://hg.mozilla.org/mozilla-central/rev/8b482a58dd47

We should test this in RelEng staging, and then if everything works fine I'll land this on branches.
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla1.9.3a5
Urgh, I'm an idiot, I read the bugmail as "review granted" when it said "review not granted". I fixed the one thing you commented on, is there anything else you wanted fixed in this patch?
Comment on attachment 443668 [details] [diff] [review]
Allow automationutils.py to use stackwalk.cgi

The correct patch was pushed as 8b482a58dd47
Attachment #443668 - Flags: review- → review+
You need to log in before you can comment on or make changes to this bug.