Open Bug 1835524 Opened 6 months ago Updated 2 months ago

Use BugSplat for SeaMonkey crash reporting

Categories

(SeaMonkey :: Release Engineering, task)

Tracking

(Not tracked)

ASSIGNED

People

(Reporter: iannbugzilla, Assigned: iannbugzilla)

References

(Blocks 1 open bug)

Details

Attachments

(4 files, 7 obsolete files)

We are planning to switch to BugSplat for crash reporting.
Provisionally we will use the existing breakpad as that will probably easier than putting in crashpad.

The .mozconfig file will need updating to use BugSplat:
ac_add_options --with-crashreporter-url=https://seamonkey_project.bugsplat.com/post/bp/crash/postBP.php

Still need to workout how to get all the information visible that we want in BugSplat.

Attached patch WIP-1835524-bugsplat-25317.patch (obsolete) — Splinter Review

At the moment I have:

  • Added and tested building/use of symupload from breakpad tools for Linux.
  • Added building of symupload from breakpad tools for mac/windows but these to be checked/tested.
  • Put some temporary mapping of parameters to get the crash uploads working.
  • Created a script for uploading crash reporter symbols locally but this will probably need converting into mach/make format to use on the builders.
  • Tested both a crash report and symbols upload to BugSplat successfully

Still to be done:

  • Find out from BugSplat how to get full visibility of all the parameters we upload with the mini-dump.
  • Find out why line numbers and file names are not displaying in BugSplat Overview.
Attachment #9336294 - Flags: feedback?(frgrahl)
Attached file sym-upload.sh

Temporary linux shell script for symbol uploads

Attached patch 1835524-bugsplat-mr-25318.patch (obsolete) — Splinter Review

[Approval Request Comment]
Regression caused by (bug #): n/a
User impact if declined: no crashreports uploaded
Testing completed (on m-c, etc.): 2.53.18 linux
Risk to taking this patch (and alternatives if risky): low
String changes made by this patch: none

BugSplat requires different parameters for upload and the response from the server is in a different format.

We also need to build symupload so that releases can upload the symbols to the server.

The command to upload would be:
symupload <symfilename> "https://seamonkey_project.bugsplat.com/post/bp/symbol/breakpadsymbols.php?appName=SeaMonkey&appVer=<version>"
<version> is built from PackageVersion-BuildID e.g. SeaMonkey-2.53.18b1pre-20230910134258

The .mozconfig needs to contain:
ac_add_options --with-crashreporter-url=https://seamonkey_project.bugsplat.com/post/bp/crash/postBP.php

As BugSplat doesn't allow custom fields, I have combined all the information we might need into the comments field.
At the moment the application name is set to "SeaMonkey" so all crashreports for all versions would appear in one place. Another option would be to split by release channel or by version, so the application name could be SeaMonkey-nightly / SeaMonkey-beta / SeaMonkey-release or SeaMonkey-2.53.18b1pre / SeaMonkey-2.53.18b1 / SeaMonkey-2.53.18

Attachment #9336294 - Attachment is obsolete: true
Attachment #9336294 - Flags: feedback?(frgrahl)
Attachment #9352407 - Flags: review?(frgrahl)
Attachment #9352407 - Flags: feedback?(ewong)
Attachment #9352407 - Flags: approval-comm-release?

[Approval Request Comment]
Regression caused by (bug #): n/a
User impact if declined: links in about crashes point to wrong place
Testing completed (on m-c, etc.): 2.53.18 linux
Risk to taking this patch (and alternatives if risky): low
String changes made by this patch: none

Attachment #9352408 - Flags: review?(frgrahl)
Attachment #9352408 - Flags: approval-comm-release?

The patches will need testing on Windows and macOS

Flags: needinfo?(frgrahl)

[Approval Request Comment]
Regression caused by (bug #): 1420363
User impact if declined: symbol upload code cannot be built
Testing completed (on m-c, etc.): 2.53.18preb1 on linux
Risk to taking this patch (and alternatives if risky): medium
String changes made by this patch: none

I've expanded SendRequest to take both JSON and key/value pairs along with related functions.
I've tested on linux but the changes need to be tested on Windows and macos.

Attachment #9357017 - Flags: review?(frgrahl)
Attachment #9357017 - Flags: approval-comm-release?

Updates since last version:

  • Makes use of the updated functions as BugSplat needs parameters and doesn't process the JSON
  • BugSplat returns response as JSON so parse it as such
  • Fixes submitting crash reports through CrashSubmit.jsm - may have to look at removing duplicated code in a future bug
Attachment #9352407 - Attachment is obsolete: true
Attachment #9352407 - Flags: review?(frgrahl)
Attachment #9352407 - Flags: feedback?(ewong)
Attachment #9352407 - Flags: approval-comm-release?
Attachment #9357019 - Flags: review?(frgrahl)
Attachment #9357019 - Flags: approval-comm-release?

Fix missing variable name change

Attachment #9357017 - Attachment is obsolete: true
Attachment #9357017 - Flags: review?(frgrahl)
Attachment #9357017 - Flags: approval-comm-release?
Attachment #9357060 - Flags: review?(frgrahl)
Attachment #9357060 - Flags: approval-comm-release?

Fix missing argument from GenerateRequestBody

Attachment #9357060 - Attachment is obsolete: true
Attachment #9357060 - Flags: review?(frgrahl)
Attachment #9357060 - Flags: approval-comm-release?
Attachment #9357176 - Flags: review?(frgrahl)
Attachment #9357176 - Flags: approval-comm-release?

Fix key for wstring map to be literal

Attachment #9357019 - Attachment is obsolete: true
Attachment #9357019 - Flags: review?(frgrahl)
Attachment #9357019 - Flags: approval-comm-release?
Attachment #9357222 - Flags: review?(frgrahl)
Attachment #9357222 - Flags: approval-comm-release?

v1.4 - Fix issue with assigning to wstring
v1.5 - Fix issue with clearing submitted reports

Attachment #9357222 - Attachment is obsolete: true
Attachment #9357222 - Flags: review?(frgrahl)
Attachment #9357222 - Flags: approval-comm-release?
Attachment #9357284 - Flags: review?(frgrahl)
Attachment #9357284 - Flags: approval-comm-release?

For 1.6 - Include channel in ver parameter to make it easier to find crashes in BugSplat UI

Attachment #9357284 - Attachment is obsolete: true
Attachment #9357284 - Flags: review?(frgrahl)
Attachment #9357284 - Flags: approval-comm-release?
Attachment #9357706 - Flags: review?(frgrahl)
Attachment #9357706 - Flags: approval-comm-release?

Comment on attachment 9352408 [details] [diff] [review]
1835524-bugsplat-cr-25318.patch

LGTM

Flags: needinfo?(frgrahl)
Attachment #9352408 - Flags: review?(frgrahl)
Attachment #9352408 - Flags: review+
Attachment #9352408 - Flags: approval-comm-release?
Attachment #9352408 - Flags: approval-comm-release+
Keywords: leave-open
Pushed by frgrahl@gmx.net:
https://hg.mozilla.org/comm-central/rev/d83ca98a3267
Use BugSplat for SeaMonkey crash reporting - comm part. r=frg
Keywords: leave-open

Comment on attachment 9357706 [details] [diff] [review]
1835524-bugsplat-mr-v1_6-25318.patch

LGTM r/a+ fro our branch.

toolkit/crashreporter/CrashReports.jsm:

+this.CrashReports.pendingDir = CrashReports_getDir("pending");
+this.CrashReports.reportsDir = CrashReports_getDir();
+this.CrashReports.submittedDir = CrashReports_getDir("submitted");

We could further optimize this by just using the reports dir for the other variables and adding the subdirectory to them but not hot code so just go with it. The two additional Services.dirsvc.get will not hurt anyone I think.

Attachment #9357706 - Flags: review?(frgrahl)
Attachment #9357706 - Flags: review+
Attachment #9357706 - Flags: approval-comm-release?
Attachment #9357706 - Flags: approval-comm-release+

Comment on attachment 9357176 [details] [diff] [review]
1835524-sym-upload-fix-v1_2-25318.patch

LGTM. r/a+

Is the title for the patch not misleading? It deals with sending the crash reports and also fixes the upload?

Attachment #9357176 - Flags: review?(frgrahl)
Attachment #9357176 - Flags: review+
Attachment #9357176 - Flags: approval-comm-release?
Attachment #9357176 - Flags: approval-comm-release+
Blocks: 1858551
You need to log in before you can comment on or make changes to this bug.