563745 - Make automationutils.py's minidump processing able to send minidumps to minidump-stackwalk-server

Status: RESOLVED FIXED

(Testing :: General, defect)

Description

[(not currently active) Ted Mielczarek]

Currently automationutils.py knows how to run minidump_stackwalk locally to process minidumps resulting from crashes in our test suites. bug 561754 would like to offload this processing to a different machine, so that the build machines don't have to download the symbols. bug 563678 implemented a simple CGI that accepts a minidump and symbols URL via HTTP POST, and returns the stackwalk output in the body. We'll need to hack automationutils to be be able to use this service.

Comment 1

[(not currently active) Ted Mielczarek]

Attachment: Allow automationutils.py to use stackwalk.cgi

This modifies automationutils.py and allows it to use stackwalk.cgi for minidump processing. I've added catlee's poster library as a zip file for convenience, since posting multipart/form-data is pretty ugly in Python otherwise (and shuffling multiple files for a Python library around our tet harnesses is clunky).

To use:
1) Set MINIDUMP_STACKWALK_CGI=http://url/to/stackwalk.cgi in the environment, instead of MINIDUMP_STACKWALK
2) Run test harnesses with --symbols-path=http://url/to/symbols.zip pointing at the symbol package for the build being tested
3) ??
4) Profit

Comment 2

[Chris AtLee [:catlee]]

Comment on attachment 443668 [details] [diff] [review]
Allow automationutils.py to use stackwalk.cgi

> def checkForCrashes(dumpDir, symbolsPath, testName=None):
>   stackwalkPath = os.environ.get('MINIDUMP_STACKWALK', None)
>+  stackwalkCGI = os.environ.get('MINIDUMP_STACKWALK_CGI', None)
>   # try to get the caller's filename if no test name is given
>   if testName is None:
>     try:
>@@ -111,14 +118,33 @@
>       # eat minidump_stackwalk errors
>       subprocess.call([stackwalkPath, d, symbolsPath], stderr=nullfd)
>       nullfd.close()
>+    elif stackwalkCGI and symbolsPath and isURL(symbolsPath):
>+      try:
>+        f = open(d, "rb")
>+        sys.path.append(os.path.join(os.path.dirname(__file__), "poster.zip"))
>+        from poster.encode import multipart_encode
>+        from poster.streaminghttp import register_openers
>+        import urllib2
>+        register_openers()
>+        datagen, headers = multipart_encode({"minidump": f,
>+                                             "symbols": symbolsPath})
>+        request = urllib2.Request(stackwalkCGI, datagen, headers)
>+        print urllib2.urlopen(request).read()
>+      finally:
>+        if f:
>+          f.close()

If open(d, "rb") fails for some reason, then f won't be set, and you'll get a NameError in the finally block.  Initialize f to None before the try block?

Comment 3

[(not currently active) Ted Mielczarek]

Pushed to m-c:
http://hg.mozilla.org/mozilla-central/rev/8b482a58dd47

We should test this in RelEng staging, and then if everything works fine I'll land this on branches.

Comment 4

[(not currently active) Ted Mielczarek]

Urgh, I'm an idiot, I read the bugmail as "review granted" when it said "review not granted". I fixed the one thing you commented on, is there anything else you wanted fixed in this patch?

Comment 5

[Chris AtLee [:catlee]]

Comment on attachment 443668 [details] [diff] [review]
Allow automationutils.py to use stackwalk.cgi

The correct patch was pushed as 8b482a58dd47