Closed
Bug 1037512
Opened 10 years ago
Closed 10 years ago
Ability to get all raw dumps from the API
Categories
(Socorro :: Webapp, task)
Tracking
(Not tracked)
RESOLVED
FIXED
105
People
(Reporter: benjamin, Assigned: peterbe)
References
Details
Currently to download a raw dump, you follow a link such as https://crash-stats.mozilla.com/rawdumps/c0416855-49cc-4c23-ade6-a78182140630.dmp But this only works if you're logged in and isn't an API endpoint. There is the RawCrash API that looks like it would also provide raw dump access, but it doesn't appear to work. Links such as https://crash-stats.mozilla.com/api/RawCrash/?crash_id=c0416855-49cc-4c23-ade6-a78182140630&format=raw return an ISE. In addition, there is no way to get access to the raw dumps other than the main one: so for plugin hangs where there are 1-3 additional minidumps, there is no way to download them. I propose an additional parameter to the RawCrash API dumpname=string which can be used to get access to the additional minidumps. Can somebody look up what the current ISE is and confirm that it's supposed to be working?
Flags: needinfo?(chris.lonnen)
Comment 1•10 years ago
|
||
(In reply to Benjamin Smedberg [:bsmedberg] from comment #0) > Currently to download a raw dump, you follow a link such as > https://crash-stats.mozilla.com/rawdumps/c0416855-49cc-4c23-ade6- > a78182140630.dmp > > But this only works if you're logged in and isn't an API endpoint. > > There is the RawCrash API that looks like it would also provide raw dump > access, but it doesn't appear to work. Links such as > https://crash-stats.mozilla.com/api/RawCrash/?crash_id=c0416855-49cc-4c23- > ade6-a78182140630&format=raw return an ISE. Here's the error, the API "cleaner" is trying to treat the response as JSON and redact keys - so it's just assuming the type is 'meta'. We should probably just make a dedicated endpoint for this: https://errormill.mozilla.org/webtools/socorro-stage/group/170446/ AttributeError: 'str' object has no attribute 'keys' Stacktrace (most recent call last): File "django/core/handlers/base.py", line 115, in get_response response = callback(request, *callback_args, **callback_kwargs) File "waffle/decorators.py", line 36, in _wrapped_view return view(request, *args, **kwargs) File "ratelimit/decorators.py", line 95, in _wrapped return fn(request, *args, **kw) File "crashstats/crashstats/utils.py", line 296, in wrapper response = f(request, *args, **kw) File "crashstats/crashstats/utils.py", line 53, in wrapper response = f(request, *args, **kw) File "crashstats/api/views.py", line 284, in model_wrapper cleaner.start(result) File "crashstats/api/cleaner.py", line 42, in start self._scrub(data, self.whitelist) File "crashstats/api/cleaner.py", line 49, in _scrub self._scrub_list(result, api_whitelist) File "crashstats/api/cleaner.py", line 92, in _scrub_list self._scrub_item(data, whitelist) File "crashstats/api/cleaner.py", line 73, in _scrub_item for key in data.keys():
Reporter | ||
Comment 2•10 years ago
|
||
ok, so completely separate the API like this: CrashMetadata(uuid) - returns the metadata key/value pairs CrashDump(uuid, name) - returns the minidump file, pass "" for the default or "browser" will get you "upload_file_minidump_browser" rhelmer does that sound ok?
Flags: needinfo?(chris.lonnen) → needinfo?(rhelmer)
Comment 3•10 years ago
|
||
(In reply to Benjamin Smedberg [:bsmedberg] from comment #2) > ok, so completely separate the API like this: > > CrashMetadata(uuid) - returns the metadata key/value pairs > CrashDump(uuid, name) - returns the minidump file, pass "" for the default > or "browser" will get you "upload_file_minidump_browser" > > rhelmer does that sound ok? That looks good to me - I don't think there's a need for "Raw" in the name since the processed version is called "ProcessedCrash", I think that will be clear enough for people looking at the API docs.
Flags: needinfo?(rhelmer)
Assignee | ||
Updated•10 years ago
|
Assignee: nobody → peterbe
Component: Middleware → Webapp
Assignee | ||
Comment 4•10 years ago
|
||
The ISE isn't because of the scrubber but because it incorrectly tries to return the binary content as a JSON string. I'm going to tackle that primary problem first and foremost and then we'll extend the functionality to get other dumps other than the "main one". I wasn't aware there were more than one by name.
Assignee | ||
Comment 5•10 years ago
|
||
PR https://github.com/mozilla/socorro/pull/2217 This just adds the ability to down the raw dump by /api/RawCrash/?crash_id=:crash_id&format=raw It requires the `view_pii` permission. Things that remain to do: 1. Change the link in the /report/index/:crash_id page to use the API link instead. Maybe? 2. Benjamin's feature request to be able to get other dumps other than the "main one" .
Comment 6•10 years ago
|
||
Commits pushed to master at https://github.com/mozilla/socorro https://github.com/mozilla/socorro/commit/b6da8f6ff46ffe079ed9fabcebc515d3c61279f6 bug 1037512 - ability to get raw dumps from the API, r=@AdrianGaudebert https://github.com/mozilla/socorro/commit/92296748505704c1d6c68d634181165032802e7e Merge pull request #2217 from peterbe/bug-1037512-ability-to-get-raw-dumps-from-the-api bug 1037512 - ability to get raw dumps from the API
Updated•10 years ago
|
Target Milestone: --- → 95
Assignee | ||
Comment 7•10 years ago
|
||
/me clearly forgot to put the "fixes" in the commit message.
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Reporter | ||
Comment 8•10 years ago
|
||
Peter, since mostly the point of this bug was to be able to download all of the dumps, I'm going to reopen this. It's especially important now that I've switched nightly/aurora to use crash-stats instead of the temporary skunky-server.
Reporter | ||
Updated•10 years ago
|
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Assignee | ||
Comment 9•10 years ago
|
||
We can now query for raw dumps other than the default "upload_file_minidump" https://github.com/peterbe/socorro/commit/002f1b975a50a6bd6b7832a04e4d85023bb400fa
Assignee | ||
Comment 10•10 years ago
|
||
New PR just for the new API parameter https://github.com/mozilla/socorro/pull/2366
Comment 11•10 years ago
|
||
Commits pushed to master at https://github.com/mozilla/socorro https://github.com/mozilla/socorro/commit/fc3fd4c685503fa35bc7a2b0283b678c24ac360c bug 1037512 - adding 'name' param to /api/RawCrash/ https://github.com/mozilla/socorro/commit/177a10c7c3f370923ee7b0a93840db77106e2ebb Merge pull request #2366 from peterbe/bug-1037512-adding-name-param-to-apirawcrash bug 1037512 - adding 'name' param to /api/RawCrash/
Assignee | ||
Comment 12•10 years ago
|
||
I'm going to close this because it's annoying that it's one bug for multiple different (albeit interlinked) features in one bug. The last github commit should now make it totally possible to add do: https://crash-stats.mozilla.com/api/RawCrash/?crash_id=c0416855-49cc-4c23-ade6-a78182140630&format=raw&name=upload_file_minidump_flash1 as soon as this bug gets deployed.
Status: REOPENED → RESOLVED
Closed: 10 years ago → 10 years ago
Resolution: --- → FIXED
Updated•10 years ago
|
Target Milestone: 95 → 103
Updated•10 years ago
|
Target Milestone: 103 → 104
Updated•10 years ago
|
Target Milestone: 104 → 105
You need to log in
before you can comment on or make changes to this bug.
Description
•