Include UUID in graphics report csv

RESOLVED FIXED

Status

RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: peterbe, Assigned: peterbe)

Tracking

Trunk
Points:
---

Firefox Tracking Flags

(firefox47 affected)

Details

Attachments

(2 attachments)

(Assignee)

Description

3 years ago
We used to yield a "uuid_url" [0] with a hardcoded URL. 
Instead, let's just return the crash_id in that space. 


[0] https://github.com/mozilla/socorro/blob/f1e289789510cbdc2bf184cca121b1708efac200/socorro/external/postgresql/graphics_report.py#L21
(Assignee)

Comment 1

3 years ago
Vlad, by the way, we try to call it "Crash ID" these days. (having said that, I can never decide if it's "crashID", "crash_id", "crash ID" or "crashid" :)
Can we also include 'release_channel' and 'branch'?  I can create a PR for it myself, but figure you can probably get it in this one faster :)

Comment 4

3 years ago
Commits pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/874b6d90c1221d4f3d90ec1871fbf633d104262f
fixes bug 1249412 - Include UUID in graphics report csv

https://github.com/mozilla/socorro/commit/a51a7766007078fcbb7946190a4d10f252cd278d
Merge pull request #3206 from peterbe/bug-1249412-include-uuid-in-graphics-report-csv

fixes bug 1249412 - Include UUID in graphics report csv

Updated

3 years ago
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
(Assignee)

Comment 5

3 years ago
The graphics report thing was a desperate thing. Ideally everyone should be using the SuperSearch API. It's maintained and it makes us have fewer unique snowflake solutions we have to maintain. 
The reason we put it in, was to make it really easy for the graphics team to get exactly the report they were used to without having to run it over on the ad-hoc crash-analysis box (it's a long-running goal to decommission that server). 

So, together with Adrian (the SuperSearch guru) can we figure out if we can use SuperSearch instead? If not, can we fix SuperSearch?
(it might mean inconvenience for you in that you might have to mess with API tokens and paginate the results etc.)
This is work that's related to the graphics team, but with broader scope.  I've been talking with Adrian; unfortunately, the SuperSearch API is not sufficient for processing crashes.  I need to download/process all crash reports in a given date range.  Using the graphics-report CSV API, I can fetch a day's worth of crash reports in something like 20-30 seconds.  Using the SuperSearch API, it takes about 4-5s to fetch a single "page" of 100 reports.  Fetching a day's worth of reports (~165k) would take around 2.5 hours.

I fully agree that we should be using a single API and should be improving the SuperSearch API; however, that's not something that I can do myself, and I don't know when Adrian or you can fit it in.  Fixing up the graphics report CSV generation unblocks the analysis work that I'm doing, and it can get switched over to SuperSearch results when that's fast.  (Current code can handle both inputs, so it'll be easy to test/switch.)
(Assignee)

Comment 7

3 years ago
Adrian, 
Could we, in an ad-hoc django view, loop over the pagination server-side and buffer a big monster report? Any other ideas for how to make this doable in SuperSearch? 

Vladimir,
Thank you for your open feedback. Mucho appreciated. What we're trying to do with Socorro, in 2016, is to greatly simplify it. At the moment we store crashes in ES AND in PostgreSQL. It's our goal to remove all storage of crashes in PostrgreSQL (we're going to keep it around as a general relational store for other non-crashes stuff). That's why I'm eager to find a working solution. 

I'd be happy to hack the existing SQL we have now. I can easily add release_channel. But for branch, I don't even know what column that would be. In the old code, we never returned it. https://github.com/mozilla/socorro/blob/master/socorro/external/postgresql/graphics_report.py#L28
release_channel is the more important thing; I figured branch would be handy if we had it, but I don't have good use for it.  I *think* release_channel is the last thing I need now...
(Assignee)

Comment 9

3 years ago
/me adding release_channel
Status: RESOLVED → REOPENED
Resolution: FIXED → ---

Comment 11

3 years ago
Commits pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/ad4ed7e22755ae7590d665a4e3bf428356447874
fixes bug 1249412 - include release_channel in graphics report csv

https://github.com/mozilla/socorro/commit/bad1a2c60dd42674319468089cc52111bdd900c1
Merge pull request #3214 from peterbe/bug-1249412-include-release_channel-in-graphics-report-csv

fixes bug 1249412 - include release_channel in graphics report csv

Updated

3 years ago
Status: REOPENED → RESOLVED
Last Resolved: 3 years ago3 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.