967673 - Sort row in topcrasher_ranks_bybug by product and version

Status: RESOLVED WONTFIX

(Socorro Graveyard :: Middleware, defect)

Description

[Robert Kaiser]

In e.g. https://crash-stats.mozilla.com/topcrasher_ranks_bybug/?bug_number=711568 we should sort the displayed rows by product (in the order of the dropdown in the UI) and versions in (probably descending, but I'm not too strong-minded about ASC/DESC) version order.

Comment 1

[Schalk Neethling [:espressive]]

Ok, so I have some digging through the code for this bug and spoke to lonnen, here is where I am now:

First question is, is there a kind of weighting used to sort the products at the DB or middleware level? I am referring to the order products are returned from say the Products (currentversions) service a.k.a product selector drop down order in the UI?

Lonnen pointed to "its created here: https://github.com/mozilla/socorro/blob/master/socorro/external/postgresql/raw_sql/views/001_product_info_view.sql"

One thing to note here is that the releases here comes from signature summary and so is not sorted the same way as currentversions. [https://github.com/mozilla/socorro/blob/master/webapp-django/crashstats/crashstats/views.py#L378]

And then filtered to only contain active here:
https://github.com/mozilla/socorro/blob/master/webapp-django/crashstats/crashstats/views.py#L378

The order the above is returned in determines the order it is displayed in, in the UI so, either before or after, I need to apply the same logic to sort the products as it is in the drop down. The other related problem is that the versions are also not sorted desc nor asc.

One thing I also noticed is that in the Products service in the middleware, it uses product_sort, version_sort in the order by clause:
https://github.com/mozilla/socorro/blob/master/socorro/external/postgresql/products.py#L45

--

From all of the above it is clear the sorting is done on the DB, as mentioned above by lonnen, but signature does not use the same sorting (from what I can tell)

So, one option is to look over the SQL and determine the logic and see if it can be applied in the bybug function in views.py or, perhaps someone more familiar with all of this (lonnen, rhelmer, peterbe) has some additional insight or suggestions in this regard.

--

On a separate note, I was wondering whether it would be an additional improvement in the UI to have each product in a separate table as apposed to all in one table. That way, I can also add sorting on the version column in the UI. Kairo, bsmedberg, would that be useful or, if the problem described here is fixed, would the latter not add much in terms of usability?

Thanks everyone!!

Comment 2

[Benjamin Smedberg]

I don't think I understand or can provide useful feedback on this question.

Comment 3

[Robert Kaiser]

(In reply to Schalk Neethling [:espressive] from comment #1)
> On a separate note, I was wondering whether it would be an additional
> improvement in the UI to have each product in a separate table as apposed to
> all in one table. That way, I can also add sorting on the version column in
> the UI. Kairo, bsmedberg, would that be useful or, if the problem described
> here is fixed, would the latter not add much in terms of usability?

I'd prefer to have them in one table per signature, but sorted by product/version combination.

Comment 4

[Schalk Neethling [:espressive]]

After some more digging I found the following:

When the report_type passed to the signature_summary service is product, which it is in this case, the results are returned ordered by report_count [https://github.com/mozilla/socorro/blob/master/socorro/external/postgresql/signature_summary.py#L140], which makes total sense for where it is used under the signature summary tab.

In this case however, I would want it ordered by product_sort and version_sort as the produts service does:
https://github.com/mozilla/socorro/blob/master/socorro/external/postgresql/products.py#L45

This leads me to believe that:

1) We need to introduce a new report type, bybug perhaps, that can be passed to the service. This will essentially be the same, perhaps return even less [just the product_name and version_string], but orders by product_sort, version_sort
2) Have a way to pass to the service which order by clause to use (probably not such a good idea)
3) Adrian mentioned/asked on IRC whether it might not just be a matter of joining the right view

Thoughts?

Comment 5

[Peter Bengtsson [:peterbe]]

(In reply to Schalk Neethling [:espressive] from comment #4)
> After some more digging I found the following:
> 
> When the report_type passed to the signature_summary service is product,
> which it is in this case, the results are returned ordered by report_count
> [https://github.com/mozilla/socorro/blob/master/socorro/external/postgresql/
> signature_summary.py#L140], which makes total sense for where it is used
> under the signature summary tab.
> 
> In this case however, I would want it ordered by product_sort and
> version_sort as the produts service does:
> https://github.com/mozilla/socorro/blob/master/socorro/external/postgresql/
> products.py#L45
> 
> This leads me to believe that:
> 
> 1) We need to introduce a new report type, bybug perhaps, that can be passed
> to the service. This will essentially be the same, perhaps return even less
> [just the product_name and version_string], but orders by product_sort,
> version_sort
> 2) Have a way to pass to the service which order by clause to use (probably
> not such a good idea)
> 3) Adrian mentioned/asked on IRC whether it might not just be a matter of
> joining the right view
> 
> Thoughts?

The SQL query used by the signature summary doesn't even use product_info. Perhaps it's still in there somehow I'm just not seeing it. 

My advice would be to create a new middleware service dedicated to this. One that does something similar to the signature summary but orders very differently. 

I'm not entirely sure where this is in the context of the UI but the signature summary is a dedicated tab on the report list and let's maybe keep it to be just that and not also used by other places in the UI. 

Do you have an idea for what the ideal SQL would be?

I'm tempting to take over this whole bug because it's nearer my domain.

Comment 6

[Schalk Neethling [:espressive]]

(In reply to Peter Bengtsson [:peterbe] from comment #5)
> (In reply to Schalk Neethling [:espressive] from comment #4) 
> 
> I'm not entirely sure where this is in the context of the UI but the
> signature summary is a dedicated tab on the report list and let's maybe keep
> it to be just that and not also used by other places in the UI. 

It seems this report is still not exposed via the UI but, you can access it as such:
https://crash-stats.mozilla.com/topcrasher_ranks_bybug/?bug_number=711568

> 
> Do you have an idea for what the ideal SQL would be?
> 

As mentioned in comment 4, I was thinking that we could introduce a separate report type, which basically mimics this one:

https://github.com/mozilla/socorro/blob/master/socorro/external/postgresql/signature_summary.py#L108

But then in it's order by clause:

https://github.com/mozilla/socorro/blob/master/socorro/external/postgresql/signature_summary.py#L140

we order by product_sort and version_sort as the Products service does:

https://github.com/mozilla/socorro/blob/master/socorro/external/postgresql/products.py#L45

With that said however, the only data used that is currently returned is the product_name and version_string so, it might in fact be a better idea to create a completely separate dedicated service as you suggested.

> I'm tempting to take over this whole bug because it's nearer my domain.

I would say, if you have free cycles, go for it as it is definitely more in your neck of the woods. Thanks Peter!

Comment 7

[Robert Kaiser]

(In reply to Schalk Neethling [:espressive] from comment #6)
> It seems this report is still not exposed via the UI but, you can access it

Yes, it's quite slow (on first run, i.e. when not cached) and still has some bugs. Also, it's a but unclear where exactly to link/expose it.
That said, it's really helpful when it works right, so us in QA really want it to work right.

Comment 8

[Schalk Neethling [:espressive]]

(In reply to Robert Kaiser (:[e-mail], slow reaction due to vacation backlog) from comment #7)
> (In reply to Schalk Neethling [:espressive] from comment #6)
> > It seems this report is still not exposed via the UI but, you can access it
> 
> Yes, it's quite slow (on first run, i.e. when not cached) and still has some
> bugs. Also, it's a but unclear where exactly to link/expose it.
> That said, it's really helpful when it works right, so us in QA really want
> it to work right.

In terms of exposing it on the front end, I was thinking that anywhere (or perhaps just on top crashers) bug numbers are listed and clickable, we can show a little context menu, on click, that offers two options:

=> View Bug on Bugzilla
=> View Signatures for Bug

What do you think? Is that something that will work or, should it just simply be exposed in the report drop down?

Comment 9

[Robert Kaiser]

(In reply to Schalk Neethling [:espressive] from comment #8)
> In terms of exposing it on the front end, I was thinking that anywhere (or
> perhaps just on top crashers) bug numbers are listed and clickable, we can
> show a little context menu, on click, that offers two options

Not sure about that, I think that could make our workflow actually worse. I'd love to see this report exposed on Bugzilla, actually, on any bug that has the "Crash Signature" field filled. But let's take this to a different bug than the sorting issue. ;-)

Comment 10

[[DEACTIVATED] Adrian Gaudebert]

We now have https://mozilla.github.io/bug-signatures-status/