Closed Bug 915373 Opened 7 years ago Closed 6 years ago

Display topcrash ranks for all signatures of a bug

Categories

(Socorro :: Webapp, task)

task
Not set

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: kairo, Assigned: espressive)

References

Details

(Whiteboard: [QA+])

We should have a page on Socorro where we'd display all signatures for a given bug with their topcrash ranks in all the current versions of our affected major products.


What I imagine is that
* you have an input field (or part of the URL) where you state the bug,
* and then we pull out all the connected signatures that bug has,
* and display them on a page (linked to their report/list pages),
* and for everyone of those
* display a table with all affected product/versions (similar to signature summary)
* with the rank this signature has in the 7-day TCBS view for that product/version.

We probably also should have a column that states the process type and then use the rank from the "type:plugin" TCBS if that process type is "plugin" and from the default "type:browser" TCBS else.

We possibly could also add the number of crashes, and for Nightly and Aurora versions, the rank on the 3-day by-build TCBS if possible, as we are looking for that on those channels quite often.

Bug 913266 is connected to this, FWIW.
The good thing here is that all the data for this is already there in the Socorro PostgreSQL DB, AFAIK.
So ideally, this is only UI work (possibly middleware work to expose it to the UI).

Even if we have this available from the API it would be a great help. We have the need for that type of inquiry quite often in our day-to-day crash investigation work.
I prototyped this using a script and the APIs (bugzilla and socorro).

github branch: https://github.com/bsmedberg/socorro-toolbox/compare/topcrash-rank-by-bug?expand=1

Raw script that you can on any computer standalone: https://raw.github.com/bsmedberg/socorro-toolbox/topcrash-rank-by-bug/src/main/python/bug-rank.py

Kairo I think this ought to serve the immediate needs of daily/weekly triage until this can be a builtin feature.
Blocks: 918366
Blocks: 918379
Assignee: nobody → bsavage
From IRC, some thoughts.

* We want to use the bug number to get all the signatures connected to it.
* And then the ranks to those signatures.
* It should be keyed by bug

One other thought we should have is where this goes in the app.

* It should be a separate page to not clutter TCBS
* Let's put it in the drop down

Do we have something in the DB already connecting signatures to bugzilla ids?

* Yes, have a look at bug-related (bug_associations?) tables
* We do have the bug<->signature mapping in a table, we are using that for e.g. in TCBS UI to show bugs for every signature


We have to work together on this one Brandon, as we have done very successfully before:) I will think about the UI and do some mocks and then we just have to stay in sync.
FWIW, I think we might even have the topcrash ranks in the database as they're the basis for topchangers.
Brandon: what's the status here?
I started working on a query to return topcrash ranks for all signatures of a particular bug, but since we don't pre-calculate "ranking" we'd need to redo this part of the query underlying the TCBS model:

https://github.com/mozilla/socorro/blob/65e2bccd3fbd5b6ffdff189aeaa66448b60a4c6e/socorro/external/postgresql/tcbs.py#L100

I think we should consider pre-calculating rank for 3/7/14/28 days (via a stored procedure + matview), but we'd need to do it for all product/versions if we want to replace the current https://crash-stats.mozilla.com/topchangers/products/Firefox so we should measure if it is worth it first (if we left that report alone, then we'd have different ways of calculating rank in different parts of the system, which is just asking for trouble)

Anyway, TCBS already is a matview, and the query it does is a bit hairy but EXPLAIN doesn't look *too* bad (we use it for /topchangers now after all) and we do cache the response (via the Django model) so I think we could use it, at least initially.

So, I think this feature could be implemented now with the current TCBS mware service+django model, perhaps with a new argument to only return data for the requested signature (we still need to query over the entire date range to calculate the rank relative to other signatures, but it'd be way less JSON to pass around if we filtered the result).

We'd also need a SignaturesByBug model+new mware service, the query for this would be very simple:

SELECT status, resolution, short_desc, signature
FROM bugs 
JOIN bug_associations ON (bug_associations.bug_id = bugs.id)
WHERE bug_id = '219107';

From an API point of view, one would first use the bug # to get the list of signatures:

https://crash-stats.mozilla.com/api/SignaturesByBug/?bug_id=219107

{
  "hits": [
  {
    "id": 219107, 
    "signature": "nsVoidArray::Count()"
  }
], 
"total": 1
}

Then, get the rank for those signature(s):

https://crash-stats.mozilla.com/api/TCBS/?product=Firefox&version=28.0a1&signature=nsVoidArray%3A%3ACount()&limit=300

crash-stats would use the same underlying SignaturesByBug and TCBS models.

This is basically the same as bsmedberg's prototype in comment 2 except that it uses Socorro's bug_associations table instead of querying bugzilla live. However, I remember there being an issue where we wanted to re-check the status of bugs before displaying them, in case they have been closed as security-sensitive since Socorro picked them up.

So we may end up needing to have the mware service for SignaturesByBug query Bugzilla for the requested bug number to make sure it is still public, before giving up any metadata such as summary or associated signatures.
Depends on: 949280
Depends on: 949282
(In reply to Robert Helmer [:rhelmer] from comment #6)
> I started working on a query to return topcrash ranks for all signatures of
> a particular bug, but since we don't pre-calculate "ranking"

Ah, I thought we actually would do that for topchangers to be easy. Might make sense to do over the long run.


Your description of what to do for this bug sounds good to me!


> So we may end up needing to have the mware service for SignaturesByBug query
> Bugzilla for the requested bug number to make sure it is still public,
> before giving up any metadata such as summary or associated signatures.

We probably will want to display the bug summary and status anyhow to give the page a title and a bit of context, so we need to query Bugzilla for that anyhow.
(In reply to Schalk Neethling [:espressive] from comment #8)
> UI for this is in https://github.com/mozilla/socorro/pull/1744

We are making progress on the UI for this, all necessary service/model changes have landed. These are available from the API (on stage right now, shipping with 70), the implementation we're working on for django requires the SignaturesByBugs, SignatureSummary and TCBS models.

For each bug #, we pull the signatures out of SignaturesByBugs, pull the SignatureSummary for each and pass the product and version to TCBS. I think it makes conceptual sense to be able to compose the models like this, the only unfortunate thing is that TCBS is somewhat expensive since it does the ranking on the fly.

Pre-calculating ranking (e.g. matview) is a bit tricky (since it's done per-product+version) but the good news is that we can do it totally transparently from the API point of view, if it becomes necessary. We do cache all the models involved, so it might not end up being a big deal.
Assignee: bsavage → schalk.neethling.bugs
Target Milestone: --- → 70
Commits pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/d36cf4d6c70a6cb2eb54723a14aa53e90b814e53
Fix Bug 915373, UI for display tc ranks for all signatures of a bug

https://github.com/mozilla/socorro/commit/6345f789f0c2f3fe15f01b88e45c3784b78822c1
Merge pull request #1755 from rhelmer/bug915373-display-tc-ranks-for-all-sig-of-bug

Fix Bug 915373 UI, Display topcrash ranks for all signatures of a bug
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Note that there are a few issues as landed:

1) not sure if multiple-bugs-per-signature is working right in the backend
2) rankings above 300 are not displayed (per TCBS_RESULT_COUNTS default setting in Django)
3) minor UI issue with twisty menus (at least for me on Beta and Nightly)

Since the UI is unlinked I think it's worth getting it out there so people can try it with production data - it should be live on stage shortly at:

https://crash-stats.allizom.org/topcrasher_ranks_bybug
qa+, but per comment 12 I'd rather not block unless there's something catastrophic - I would like to get this out there so we can get feedback on whether the feature is working for crashkill users
Whiteboard: [QA+]
(In reply to Robert Helmer [:rhelmer] from comment #12)
> 1) not sure if multiple-bugs-per-signature is working right in the backend

We'll need to see about that in production and by using it, I guess.

> 2) rankings above 300 are not displayed (per TCBS_RESULT_COUNTS default
> setting in Django)

That's fine and expected from my POV.
(In reply to Robert Helmer [:rhelmer] from comment #12)
> https://crash-stats.allizom.org/topcrasher_ranks_bybug

This is up on staging now ^
I just looked at https://crash-stats.allizom.org/topcrasher_ranks_bybug/?bug_number=930735 and 1) I'd prefer rows where we do not have any hit to not be displayed at all, and 2) I wonder what a rank of "0" means...
(In reply to Robert Kaiser (:kairo@mozilla.com) from comment #16)
> I just looked at
> https://crash-stats.allizom.org/topcrasher_ranks_bybug/?bug_number=930735
> and 1) I'd prefer rows where we do not have any hit to not be displayed at

OK that's easy to do

> all, and 2) I wonder what a rank of "0" means...

Looks like TCBS rank counter is 0-based, I can increment this by 1 if you like.
Commits pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/f0cf61f86d8f8fb4fb26b47541c68f244be47ed7
bug 915373 - filter out unranked crashes, and increment 0-based ranking

https://github.com/mozilla/socorro/commit/b19e6588c7453a3353f584a5fde1a8651d1c4804
Merge pull request #1757 from rhelmer/bug915373-filter-out-unranked-and-increment-0-based-ranking

bug 915373 - filter out unranked crashes, and increment 0-based ranking
kairo, how much do you care about non-featured versions? It'd improve performance quite a bit if we filtered out non-featured versions, I think... these tend to correlate with low rankings (in my limited testing so far), and now that we are filtering those out seems like a waste.
Flags: needinfo?(kairo)
(In reply to [github robot] from comment #18)
> Commits pushed to master at https://github.com/mozilla/socorro
> 
> https://github.com/mozilla/socorro/commit/
> f0cf61f86d8f8fb4fb26b47541c68f244be47ed7
> bug 915373 - filter out unranked crashes, and increment 0-based ranking
> 
> https://github.com/mozilla/socorro/commit/
> b19e6588c7453a3353f584a5fde1a8651d1c4804
> Merge pull request #1757 from
> rhelmer/bug915373-filter-out-unranked-and-increment-0-based-ranking
> 
> bug 915373 - filter out unranked crashes, and increment 0-based ranking

This is on stage now.
(In reply to Robert Helmer [:rhelmer] from comment #17)
> (In reply to Robert Kaiser (:kairo@mozilla.com) from comment #16)
> > I just looked at
> > https://crash-stats.allizom.org/topcrasher_ranks_bybug/?bug_number=930735
> > and 1) I'd prefer rows where we do not have any hit to not be displayed at
> 
> OK that's easy to do

Thanks.

> > all, and 2) I wonder what a rank of "0" means...
> 
> Looks like TCBS rank counter is 0-based, I can increment this by 1 if you
> like.

If that's the difference to the topcrash pages, then yes, sounds good.


(In reply to Robert Helmer [:rhelmer] from comment #19)
> kairo, how much do you care about non-featured versions? It'd improve
> performance quite a bit if we filtered out non-featured versions, I think...
> these tend to correlate with low rankings (in my limited testing so far),
> and now that we are filtering those out seems like a waste.

We often want non-featured versions that are new enough or want to see at a glance if we're significantly different than on the release/version before - but we definitely do not need non-active versions to be looked at any time, and right now it looks to me like that would happen.
Flags: needinfo?(kairo)
:rhelmer, :kairo So, at the moment I am just getting all nightlies but we can easily filter as we did on crash_trends.
(In reply to Schalk Neethling [:espressive] from comment #22)
> :rhelmer, :kairo So, at the moment I am just getting all nightlies but we
> can easily filter as we did on crash_trends.

You are mixing up this one with bug 915317, I think.


(In reply to Robert Helmer [:rhelmer] from comment #20)
> (In reply to [github robot] from comment #18)
> > bug 915373 - filter out unranked crashes, and increment 0-based ranking
> 
> This is on stage now.

This looks good now, thanks. It takes a long time to load, but I'm not sure if that's just because it's stage.

OTOH, one that takes a really really long time to load when it does at all (usually requires multiple tries) is https://crash-stats.allizom.org/topcrasher_ranks_bybug/?bug_number=711568 and I see some non-active versions in the list there.
Also, it looks like it does not handle the different signatures correctly, I see the same numbers for both, which cannot be right.
:kairo, want us to reopen this and retarget it?
Flags: needinfo?(kairo)
(In reply to Stephen Donner [:stephend] from comment #24)
> :kairo, want us to reopen this and retarget it?

I think a lot of work has been done here and we have a first version that can ship, but it still has some issues, namely the slowness and not working correctly with multiple signatures. I think that should probably go into followups unless Rob disagrees.
Flags: needinfo?(kairo)
(In reply to Robert Kaiser (:kairo@mozilla.com) from comment #23)
> (In reply to Robert Helmer [:rhelmer] from comment #20)
> > (In reply to [github robot] from comment #18)
> > > bug 915373 - filter out unranked crashes, and increment 0-based ranking
> > 
> > This is on stage now.
> 
> This looks good now, thanks. It takes a long time to load, but I'm not sure
> if that's just because it's stage.


This is expected unfortunately - the ranking is not pre-calculated, so we need to calculate this for each product/version/signature calculation. We can start pre-calculating it behind the scenes and it will make things better (avoiding non-featured versions would also help short-term which is why I mentioned it before).

A signature that appears in many different products and versions will take longer than one that does not for example, and currently it's simply to get that "rank" number. We could also forsake that and simply link to the topcrasher report for that product/version instead?


> OTOH, one that takes a really really long time to load when it does at all
> (usually requires multiple tries) is
> https://crash-stats.allizom.org/topcrasher_ranks_bybug/?bug_number=711568
> and I see some non-active versions in the list there.
> Also, it looks like it does not handle the different signatures correctly, I
> see the same numbers for both, which cannot be right.

I'll take a look when it loads :) I suspect that it's a signature that appears in many products and versions, so TCBS is being called for each to get the relative ranking.
(In reply to Robert Kaiser (:kairo@mozilla.com) from comment #25)
> (In reply to Stephen Donner [:stephend] from comment #24)
> > :kairo, want us to reopen this and retarget it?
> 
> I think a lot of work has been done here and we have a first version that
> can ship, but it still has some issues, namely the slowness and not working
> correctly with multiple signatures. I think that should probably go into
> followups unless Rob disagrees.

I agree, there's more work to do here, but we should be able to test and get some usefulness out of it in the meantime. We should leave it not linked in the UI for the time being too.
Commits pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/ae116db7c951f382bf15ceedde8ad7f268323d33
Fixes minor glitches with ui of bug 915373

https://github.com/mozilla/socorro/commit/0180cc90f29bb087288c4e90ede49ec571ba6664
Merge pull request #1759 from ossreleasefeed/nobug-small-tweak-to-search-bar-915373

Fixes minor glitches with ui of bug 915373
Filed followup to fix performance issues in bug 952199
See Also: → 952199
(In reply to Robert Helmer [:rhelmer] from comment #26)
> This is expected unfortunately - the ranking is not pre-calculated, so we
> need to calculate this for each product/version/signature calculation. We
> can start pre-calculating it behind the scenes and it will make things
> better (avoiding non-featured versions would also help short-term which is
> why I mentioned it before).

OK, I almost suspected that. As I mentioned, will reducing to only active versions help as well? We probably also want to sort by version.

> A signature that appears in many different products and versions will take
> longer than one that does not for example, and currently it's simply to get
> that "rank" number. We could also forsake that and simply link to the
> topcrasher report for that product/version instead?

No, that would take away what we want here: an overview of topcrash rankings for a bug in current versions.

> I'll take a look when it loads :) I suspect that it's a signature that
> appears in many products and versions, so TCBS is being called for each to
> get the relative ranking.

Yes, one of those two signatures is expected to be in the top 10 of every version we have, actually.
Blocks: 965248
Blocks: 967673
Blocks: 967675
You need to log in before you can comment on or make changes to this bug.