1260988 - Add some filters on NewSignatures

Status: RESOLVED FIXED

(Socorro :: Webapp, task)

Description

[Calixte Denizet (:calixte)]

Add two filters: product and version, to be able to retrieve the new signatures for a specific version of Firefox.

Comment 1

[[DEACTIVATED] Adrian Gaudebert]

I've started working on this, and I'm not sure about it anymore. Here's my main question: when do you filter on what? For example, if you set product=Firefox and version=48.0a1, do you look for signatures that appeared recently on 48.0a1 and did not appear before on 48.0a1, or do you search for new signatures for 48.0a1 and that did not appear before on any version? 

I still think the product filter makes sense, notably because it is set to Firefox by default, and that should be changed by a user.

Also, once you have a signature, it's easy to get some data about it. For example, if I run it now with default values, here's what I get: 

GET https://crash-stats.mozilla.com/api/NewSignatures/?end_date=2016-04-01&start_date=2016-03-23&not_after=2016-01-01

{
  "hits": [
    "OOM | unknown | js::AutoEnterOOMUnsafeRegion::crash | js::gc::StoreBuffer::MonoTypeBuffer<T>::trace"
  ],
  "total": 1
}

So I can now use SuperSearch: 

GET https://crash-stats.mozilla.com/api/SuperSearch/?_results_number=0&signature==OOM%20|%20unknown%20|%20js::AutoEnterOOMUnsafeRegion::crash%20|%20js::gc::StoreBuffer::MonoTypeBuffer%3CT%3E::trace&_aggs.product=version

{
  "hits": [],
  "total": 4409,
  "facets": {
    "product": [
      {
        "count": 4409,
        "term": "Firefox",
        "facets": {
          "version": [
            {
              "count": 3023,
              "term": "45.0.1"
            },
            {
              "count": 789,
              "term": "46.0b5"
            },
            ...
          ]
        }
      }
    ]
  }
}

Calixte, what do you think?

Comment 2

[Calixte Denizet (:calixte)]

The same signature could appear in different channels for differents reasons (and probably different backtraces). For example, a patch has been landed in central and lead to a crash in nightly the following day and the day after the same signature appears on beta because there was a modif on www.foo.com. So this signature won't be "new" even if it can be considered as "new" in beta. So I'd prefer to be able to filter by version (which could be optional).
But you're right, it's interesting to know that this crash didn't occur only on this version: if all the crashes appear almost the same day on different versions we could be able to identify a common guilty website or an uplift which has been landed in the previous day.

Comment 3

[Sylvestre Ledru [:Sylvestre]]

Adrian, any news on it? This is blocking our progress. Merci!

Comment 4

[[DEACTIVATED] Adrian Gaudebert]

Attachment: Link to Github pull-request: https://github.com/mozilla/socorro/pull/3311#attch-to-bugzilla

Comment 5

[[DEACTIVATED] Adrian Gaudebert]

So I went with the simplest solution: all filters are applied to all sub-requests. So if you set version=Y, it will look for signatures in version Y only, and will ignore all other versions. Same for the product. I think that's what will work best, as if you need more details on a signature you can use SuperSearch. 

Does that sound good?

Comment 6

[[github robot]]

Commit pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/564783b0d10e809055717ea407d95071fcbac829
Fixes bug 1260988 - Added product and version filters to NewSignature… (#3311)

* Fixes bug 1260988 - Added product and version filters to NewSignature service.

* moved default product out of the form validation

r=peterbe