1533732 - support json blobs in crash report payload

Status: RESOLVED FIXED

(Socorro :: Antenna, enhancement, P2)

Description

[Will Kahn-Greene [:willkg] ET needinfo? me]

Currently, the payload for a crash report is multipart/form-data where each key is either a metadata value as a string or a minidump as binary. It looks something like this:

Content-Type: multipart/form-data; boundary=------------------------c4ae5238f12b6c82

--------------------------c4ae5238f12b6c82
Content-Disposition: form-data; name="Key1"

value1
--------------------------c4ae5238f12b6c82
Content-Disposition: form-data; name="Key2"

value2
--------------------------c4ae5238f12b6c82
Content-Disposition: form-data; name="upload_file_minidump";
filename="6da3499e-f6ae-22d6-1e1fdac8-16464a16.dmp"
Content-Type: application/octet-stream

<BINARY CONTENT>
--------------------------c4ae5238f12b6c82--

The crash reporter builds this, but has problems with parts of it. It would be easier if it was able to send JSON blobs. Thus the above would change to something like this:

Content-Type: multipart/form-data; boundary=------------------------c4ae5238f12b6c82

--------------------------c4ae5238f12b6c82
Content-Disposition: form-data; name="extra" filename="extra.json"
Content-Type: application/json

{"Key1":"value1","Key2":"value2"}
--------------------------c4ae5238f12b6c82
Content-Disposition: form-data; name="upload_file_minidump";
filename="6da3499e-f6ae-22d6-1e1fdac8-16464a16.dmp"
Content-Type: application/octet-stream

<BINARY CONTENT>
--------------------------c4ae5238f12b6c82--

This bug covers fleshing out the technical details and implementing support for it in the collector.

Comment 1

[Will Kahn-Greene [:willkg] ET needinfo? me]

I'm trying to work out exactly how parsing a payload should behave and I have these questions:

1. A payload can have multiple minidumps in it. Can a payload have multiple JSON blobs?

2. Will the "name" and "filename" values in the MIME headers have intrinsic meaning that affects the raw crash metadata or can the collector ignore it? For example, does this:

--------------------------c4ae5238f12b6c82
Content-Disposition: form-data; name="extra" filename="extra.json"
Content-Type: application/json

{"Key1":"value1","Key2":"value2"}

result in different raw crash metadata than this?:

--------------------------c4ae5238f12b6c82
Content-Disposition: form-data; name="mycat" filename="mycat.json"
Content-Type: application/json

{"Key1":"value1","Key2":"value2"}

3. If a payload can have multiple JSON blobs, can the blobs have overlapping keys? If so, should later keys stomp on earlier keys?

Gabriele: What do you think? What's easier to implement in the crash reporter?

Comment 2

[Gabriele Svelto [:gsvelto] (PTO)]

AFAIK a single submission can have multiple minidumps if they're paired with the main one (e.g. when reporting a hang). In this case there should be only one set of annotations though which go along with the main minidump. So for JSON I would enforce having only one JSON blob and that the name filename values are set to "extra" and "extra.json" so that we can easily and unconditionally validate the submission.

Comment 3

[Will Kahn-Greene [:willkg] ET needinfo? me]

Given that, I think this is pretty easy. I'll toss it in my queue to do a first pass next week.

Comment 4

[Will Kahn-Greene [:willkg] ET needinfo? me]

Attachment: pr 301: fix bug 1533732: add support for JSON-encoded fields

Comment 5

[Will Kahn-Greene [:willkg] ET needinfo? me]

willkg merged PR #301: "fix bug 1533732: add support for JSON-encoded fields" in 61996af.

Comment 6

[Will Kahn-Greene [:willkg] ET needinfo? me]

I waited for stage to deploy and verified that this is working per the above.

Gabriele: Is it enough to test this on stage? If not, I can push this to production.

Comment 7

[Gabriele Svelto [:gsvelto] (PTO)]

Stage is going to be fine, especially because I've got distracted by some other urgent work and the client part will slip at least by a week.

Comment 8

[Will Kahn-Greene [:willkg] ET needinfo? me]

Attachment: pr 302: bug 1533732: fix miniposter to support JSON blobs

Comment 9

[Will Kahn-Greene [:willkg] ET needinfo? me]

willkg merged PR #302: "bug 1533732: fix miniposter to support JSON blobs" in 9defe42.

Comment 10

[Will Kahn-Greene [:willkg] ET needinfo? me]

This was pushed to prod just now. If there are changes we need to make, that's totally fine, but we'll make them in future bugs. Marking as FIXED.