Closed Bug 1061371 Opened 7 years ago Closed 5 years ago

UI to download memory file from crash reports

Categories

(Socorro :: General, task)

x86_64
Windows 7
task
Not set
normal

Tracking

(Not tracked)

VERIFIED FIXED

People

(Reporter: away, Assigned: peterbe)

References

Details

Attachments

(1 file)

OOM crashes are sending a <uuid>.memory.json.gz as of bug 1007534. Socorro should offer a way to download them. (Presumably a link right next to the .dmp link?)

Bug 1026059 and possibly bug 1037512 may have helpful information.
Can you include a sample crash id? I think this is exposed as part of the processed crash in the api.
Flags: needinfo?(dmajor)
According to bug 1026059 comment #12 this is a reports that should have one: bp-a3b212d7-505f-44c7-8f62-0d41b2140831
(In reply to Benjamin Smedberg  [:bsmedberg] from comment #1)
> Can you include a sample crash id? I think this is exposed as part of the
> processed crash in the api.

I pointed to this in 1026059 comment 14 (which was good enough for the moment) but dmajor was asking in that bug if there was a way to grab the raw report too. It looks like we meant to make that available too via the API, but I can't get it to work (which would be a prereq to fixing this bug anyway)

I don't see any additional minidumps available for this particular crash, yet it has a memory report so there must be one right? I noticed that "additional_minidumps" is [] in https://crash-stats.mozilla.com/api/ProcessedCrash/?crash_id=a3b212d7-505f-44c7-8f62-0d41b2140831&datatype=processed

The "UI" could be as simple as just providing more links next to the json/dmp links under "Raw Dump" tab.
The memory report is a .json.gz file so it's probably not counted as an additional minidump (.dmp file).
Flags: needinfo?(dmajor)
Yeah, if you log in you have access to the data via https://crash-stats.mozilla.com/api/ProcessedCrash/?crash_id=a3b212d7-505f-44c7-8f62-0d41b2140831&datatype=processed (or if you use an access token tied to your account). I think that's sufficient for now; dmajor do you agree?
Flags: needinfo?(dmajor)
For the very short term yes. I don't want to do it forever though! This bug is about making it nicer in the long term.
Flags: needinfo?(dmajor)
Btw - I don't mind copy-pasting a UUID into that URL, that's fine. What is a pain to me is that it includes all the rest of the processed output, which I have to snip out by hand or with a script, before I can load the thing into the about:memory viewer.
Oh you specifically care about loading it into a viewer? Could we just add a feature to the viewer to load directly from a crash UUID?
To be clear, I was envisioning the ProcessedCrash API as the long-term solution for this.
If somebody other than me writes the viewer code, I would accept that :)

I had assumed there would be a third link right on the Raw Dump tab, next to the .dmp and .json links -- that's what makes the most sense to me from a workflow perspective. But I guess I'd be ok with anything that lets me load reports easily.
I don't mind that, but it's probably not coming this quarter (the similar request to make plugin-hang dumps downloadable has been around for a while). I think we should try to add this to the viewer code, since that should be relatively simple. Filed bug 1061960 on that.
Flags: needinfo?(n.nethercote)
I think the needinfo request in bug 1061960 subsumes this one.
Flags: needinfo?(n.nethercote)
Bug 1061960 makes the viewer able to load the memory report from the file, but it's still annoying to replace the url, download and load. I'd like to make the memory report a link like what dmajor said in comment 10. It'd be even nicer if click the link will simply open an about:memory tab with the file loaded.
(In reply to dmajor (away) from comment #10)
> I had assumed there would be a third link right on the Raw Dump tab, next to
> the .dmp and .json links -- that's what makes the most sense to me from a
> workflow perspective. But I guess I'd be ok with anything that lets me load

Can we add this link to Socorro?
Flags: needinfo?(pyang)
Hi Peter,
This bug is about adding link for memory report in crash analysis page; detail in comment 5 and comment 10
Do you think there's any chance to add it?
Flags: needinfo?(pyang) → needinfo?(peterbe)
Peter and I walked through some of this just now.

Here's a current crash report that contains a memory report: https://crash-stats.mozilla.com/report/index/92ac5591-c7ed-42b1-b65d-4a48d2160908

There's a 'ContainsMemoryReport' annotation when one of these is present. The file is submitted in a 'memory_report' form field. The collector stores all uploaded files in the raw crash storage.

This code generates the list of raw dumps for logged-in users with permission:
https://github.com/mozilla/socorro/blob/177a4d58ee5b728feabc2ef54c4e9922970a2d72/webapp-django/crashstats/crashstats/views.py#L1436

We'll need to add memory.json.gz there.

Then we also need to tweak the URL patterns for /rawdumps/ to allow memory.json.gz as an extension:
https://github.com/mozilla/socorro/blob/177a4d58ee5b728feabc2ef54c4e9922970a2d72/webapp-django/crashstats/crashstats/urls.py#L115
Note-to-self; with the help of Ted, I think I've understood what needs to be done here. 
On the "Raw Dump" tab (of report index), there are 2 links. One to the raw crash (JSON) and the raw dump (binary). 
If the raw crash contains ContainsMemoryReport=1, we can expect to find a memory_report file in S3 by the same crash ID. If possible, make that third link available. It should "proxy" the file in S3 and set the Content-Disposition filename to be ":uuid.json.gz" so that it's easy to load into about:memory.
Flags: needinfo?(peterbe)
Assignee: nobody → peterbe
There IS a way to get to the memory report MANUALLY. What this bug is about is me making this link available automatically (if possible) so you don't have to do these manual steps:

1) Find a crash that has ContainsMemoryReport=1 (e.g. https://crash-stats.mozilla.com/search/?contains_memory_report=%21__null__&product=Firefox&_sort=-date&_facets=signature&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform#crash-reports)
2) Make sure you're signed in and have the View Personal Identifyable Information permission. 
3) Click the Raw Dumps tab and at the bottom should be two links. 
4) Copy the .json link, e.g. https://crash-stats.mozilla.com/rawdumps/24e56ff6-116c-49ae-94e3-486ca2160912.json
5) Insert the string "-memory_report" before the ".json", e.g. https://crash-stats.mozilla.com/rawdumps/24e56ff6-116c-49ae-94e3-486ca2160912-memory_report.json
6) Download it and gzip so it becomes .gz, e.g. 24e56ff6-116c-49ae-94e3-486ca2160912-memory_report.json.gz)
7) about:memory and Load the file you just gzipped.
Gah! I take it back! The instructions above DON'T work. It just returns the raw JSON data. 
I fooled myself because I was testing this on a branch I was working on where it does work.
Depends on: 1262252
Commit pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/6d0a79959cea23bcb57b044537e57ba8d2e0a66d
fixes bug 1061371 - UI to download memory file from crash reports (#3468)

* fixes bug 1061371 - UI to download memory file from crash reports

* check if memory_report exists more carefully
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Thanks for making this.
I've verified this on stage and I can download it and load it into `about:memory`
http://jmp.sh/GUoUtNP

I think this is going to go live on Wednesday this week.
I verified it too. Thank you!
Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.