708453 - [Mac] Crash Reports not getting submitted on Mac trunk

Status: VERIFIED FIXED

(Toolkit :: Crash Reporting, defect)

Description

[Marcia Knous [:marcia]]

Bug 708145 and Bug 708312 were filed earlier led us to discover that Mac crash reports are not being submitted on trunk.

I have checked the latest Aurora build on 10.6 and reports are being submitted successfully. I will check the other builds as well but for now we believe this is confined to Mac trunk.

I started noticing the numbers were low last week, so I will try to find a regression range of when this started not working.

Comment 2

[Marcia Knous [:marcia]]

I confirmed that the reports don't submit on 10.7, 10.6 and 10.5 Mac.

I have gone back to builds as far as 20111129 and the reports don't submit. Still working on a regression range.

Comment 3

[Steven Michaud [:smichaud] (Retired)]

Here's a crash report submitted yesterday for yesterday's mozilla-central nightly, running on OS X 10.6.8:

bp-6801b887-5204-4db1-8935-d7ca32111207

So *some* Mac trunk crashes are still coming through.

Comment 4

[Marcia Knous [:marcia]]

Steven: If you click on the link manually in about:crashes, then it resubmits. So that may be one way it was submitted.

(In reply to Steven Michaud from comment #3)
> Here's a crash report submitted yesterday for yesterday's mozilla-central
> nightly, running on OS X 10.6.8:
> 
> bp-6801b887-5204-4db1-8935-d7ca32111207
> 
> So *some* Mac trunk crashes are still coming through.

Comment 5

[Steven Michaud [:smichaud] (Retired)]

So the bug is that some/all of the Mac bug reports aren't being submitted "automatically" (without clicking on links in about:crashes)?

Comment 6

[Steven Michaud [:smichaud] (Retired)]

Ted has/had a "crashme" extension, which might be useful in establishing STR for this bug.

Comment 7

[Marcia Knous [:marcia]]

STR:

1. Load the crashme extension.
2. Crash.
3. Submit the report from the UI dialog.
4. Note the error at the bottom of the dialog: "There was a problem submitting your report." I get this on both 10.6 and 10.7.

Ted know about this and indicated he was working on a fix.

I was working on a regression range yesterday but it was very slow downloading old builds. As far as I could see the issue goes back to at least 20111127 but that was as far as I got yesterday.

Comment 8

[Steven Michaud [:smichaud] (Retired)]

Marcia, could you attach a copy of the crashme extension to this bug? :-)

Comment 9

[Marcia Knous [:marcia]]

Attachment: Extension

Comment 10

[(not currently active) Ted Mielczarek]

It's at http://ted.mielczarek.org/mozilla/crashme.html , for reference.

In ~/Library/Application Support/Firefox/Crash Reports/submit.log, the error I'm getting is:
[Wed Dec  7 17:51:43 2011] Crash report submission failed: could not set up POST data
which is from here:
http://mxr.mozilla.org/mozilla-central/source/toolkit/crashreporter/client/crashreporter_osx.mm#575

Comment 11

[(not currently active) Ted Mielczarek]

I ran this in a debugger. It's failing to construct the NSURL here:
http://mxr.mozilla.org/mozilla-central/source/toolkit/crashreporter/client/crashreporter_osx.mm#591

the URL in question is "https://crash-reports.mozilla.com/submit?id={ec8030f7-c20a-464f-9b0e-13a3a9e97384}&version=11.0a1&buildid=20111206134941"

I'm wondering if it's choking on the {} or something? That change came from bug 686466, back on November 22nd. Has it been broken for that long?

Comment 12

[(not currently active) Ted Mielczarek]

It's definitely choking on the curly braces, as a little test program proves. It looks like there's a simple fix here, though, I'll get a patch together.

Comment 13

[Marcia Knous [:marcia]]

regression range:

Broken: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:11.0a1) Gecko/20111123 Firefox/11.0a1

Submits: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:11.0a1) Gecko/20111122 Firefox/11.0a1

Comment 14

[Marcia Knous [:marcia]]

Ted: Can you think of a way to add a test so we could catch something like this in the future?

Comment 15

[(not currently active) Ted Mielczarek]

Attachment: escape URLs before constructing NSURL

This simple patch fixes the problem. It escapes the URL before constructing a NSURL from it, which escapes the {}. Steven: can you review this? I don't really have any crashreporter peers, and this is just straight Cocoa in this case.

Comment 16

[(not currently active) Ted Mielczarek]

Marcia: right now we don't have any tests for the actual crashreporter client. It's kind of a pain, since we don't have a test harness into which that fits nicely.

Comment 17

[Steven Michaud [:smichaud] (Retired)]

Comment on attachment 580181 [details] [diff] [review]
escape URLs before constructing NSURL

This looks fine to me.  But let me test it to make sure :-)

Comment 19

[Steven Michaud [:smichaud] (Retired)]

Comment on attachment 580181 [details] [diff] [review]
escape URLs before constructing NSURL

I just tested this and it worked perfectly:  I installed the crashme extension (from http://ted.mielczarek.org/mozilla/crashme.html), and used it to crash.  When the Breakpad dialog came up I chose "Quit Firefox", and the crash report was successfully submitted.

bp-f125c267-d12b-4322-850f-0b0f92111208

Comment 20

[Steven Michaud [:smichaud] (Retired)]

I added logging and found that the {} characters were the only ones escaped in the URL -- which is what I expected, and seems to be correct.

Comment 21

[(not currently active) Ted Mielczarek]

Attachment: escape URLs before constructing NSURL.

Updated commit message.

Comment 23

[(not currently active) Ted Mielczarek]

http://hg.mozilla.org/integration/mozilla-inbound/rev/8b622aa1c57f

Comment 24

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/8b622aa1c57f

Comment 25

[Marcia Knous [:marcia]]

Verified fixed using Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:11.0a1) Gecko/20111212 Firefox/11.0a1.