1019653 - AduBySignature API not returning expected results

Status: RESOLVED INVALID

(Socorro Graveyard :: Middleware, defect)

Description

[Benjamin Smedberg]

I was trying to use the new AduBySignature API to collect the data for bug 768395 comment 11, but I couldn't get it to return any useful results.

The queries I used were https://crash-stats.mozilla.com/api/AduBySignature/?channel=aurora&product_name=Firefox&signature=CDevice%3A%3ADriverInternalErrorCB(long)&end_date=2014-05-01&start_date=2014-03-10 and https://crash-stats.mozilla.com/api/AduBySignature/?channel=nightly&product_name=Firefox&signature=CDevice%3A%3ADriverInternalErrorCB(long)&end_date=2014-05-01&start_date=2014-03-10

I tried various versions of Firefox/firefox and Nightly/nightly in case I had the wrong casing. None seemed to return any results.

I ended up falling back to my direct-DB scripts, but I'd like to know if I did something wrong or if the API is still confused.

Comment 1

[Robert Helmer [:rhelmer]]

(In reply to Benjamin Smedberg  [:bsmedberg] from comment #0)
> I was trying to use the new AduBySignature API to collect the data for bug
> 768395 comment 11, but I couldn't get it to return any useful results.
> 
> The queries I used were
> https://crash-stats.mozilla.com/api/AduBySignature/
> ?channel=aurora&product_name=Firefox&signature=CDevice%3A%3ADriverInternalErr
> orCB(long)&end_date=2014-05-01&start_date=2014-03-10 and
> https://crash-stats.mozilla.com/api/AduBySignature/
> ?channel=nightly&product_name=Firefox&signature=CDevice%3A%3ADriverInternalEr
> rorCB(long)&end_date=2014-05-01&start_date=2014-03-10
> 
> I tried various versions of Firefox/firefox and Nightly/nightly in case I
> had the wrong casing. None seemed to return any results.
> 
> I ended up falling back to my direct-DB scripts, but I'd like to know if I
> did something wrong or if the API is still confused.

We haven't backfilled that far (starts at 2014-05-03), and there's a 60-day hardcoded limit currently (being discussed in bug 1019718 and associated PR)

I can backfill this further - how far back would be useful?

Comment 2

[Benjamin Smedberg]

Unless it's a huge cost, through 4-Feb which is the beginning of FF30 on nightly.

Comment 3

[Robert Helmer [:rhelmer]]

I don't see any problems backfilling this further, what do you think :selenamarie? We could backfill enough to answer your question in https://github.com/mozilla/socorro/pull/2076#issuecomment-44991157

Comment 4

[Robert Helmer [:rhelmer]]

(In reply to Robert Helmer [:rhelmer] from comment #3)
> I don't see any problems backfilling this further, what do you think
> :selenamarie? We could backfill enough to answer your question in
> https://github.com/mozilla/socorro/pull/2076#issuecomment-44991157

Going to go ahead and do this on stage now.

Comment 5

[Robert Helmer [:rhelmer]]

(In reply to Robert Helmer [:rhelmer] from comment #4)
> (In reply to Robert Helmer [:rhelmer] from comment #3)
> > I don't see any problems backfilling this further, what do you think
> > :selenamarie? We could backfill enough to answer your question in
> > https://github.com/mozilla/socorro/pull/2076#issuecomment-44991157
> 
> Going to go ahead and do this on stage now.

Despite saying that, I literally just started the backfill now - looks like it's taking about ~20s per day (varies depending on the day), so should be done in the next few hours.

Comment 6

[Robert Helmer [:rhelmer]]

(In reply to Benjamin Smedberg  [:bsmedberg] from comment #0)
> I was trying to use the new AduBySignature API to collect the data for bug
> 768395 comment 11, but I couldn't get it to return any useful results.
> 
> The queries I used were
> https://crash-stats.mozilla.com/api/AduBySignature/
> ?channel=aurora&product_name=Firefox&signature=CDevice%3A%3ADriverInternalErr
> orCB(long)&end_date=2014-05-01&start_date=2014-03-10 and
> https://crash-stats.mozilla.com/api/AduBySignature/
> ?channel=nightly&product_name=Firefox&signature=CDevice%3A%3ADriverInternalEr
> rorCB(long)&end_date=2014-05-01&start_date=2014-03-10
> 
> I tried various versions of Firefox/firefox and Nightly/nightly in case I
> had the wrong casing. None seemed to return any results.
> 
> I ended up falling back to my direct-DB scripts, but I'd like to know if I
> did something wrong or if the API is still confused.

OK I've backfilled on stage, so the equivalent URLs work on stage:
https://crash-stats.allizom.org/api/AduBySignature/?channel=aurora&product_name=Firefox&signature=CDevice%3A%3ADriverInternalErrorCB%28long%29&end_date=2014-05-01&start_date=2014-03-10
https://crash-stats.allizom.org/api/AduBySignature/?channel=nightly&product_name=Firefox&signature=CDevice%3A%3ADriverInternalErrorCB%28long%29&end_date=2014-05-01&start_date=2014-03-10

Please let me know if you see any problems - I took a quick look and seems reasonable.

Note that bug 1019718 is still open and the 60-day limit is still in force, pending me doing some investigation on the impact of raising this.

Comment 7

[Benjamin Smedberg]

I mocked a dashboard based off this API:

http://benjamin.smedbergs.us/tests/adubysignature.html?channel=nightly&signature=CDevice%3A%3ADriverInternalErrorCB%28long%29&startDate=2014-01-01&endDate=2014-02-28#

The data which is present appears correct. However, there is one problem: if a build experiences no crashes, it is not listed in the report at all. This makes it difficult to distinguish between actually no crashes for a build and lack of data for that build.

Comment 8

[Robert Helmer [:rhelmer]]

(In reply to Benjamin Smedberg  [:bsmedberg] from comment #7)
> I mocked a dashboard based off this API:
> 
> http://benjamin.smedbergs.us/tests/adubysignature.
> html?channel=nightly&signature=CDevice%3A%3ADriverInternalErrorCB%28long%29&s
> tartDate=2014-01-01&endDate=2014-02-28#
> 
> The data which is present appears correct. However, there is one problem: if
> a build experiences no crashes, it is not listed in the report at all. This
> makes it difficult to distinguish between actually no crashes for a build
> and lack of data for that build.

Hmm I had intended to fix that in bug 1007379 - I'll reopen.

Comment 9

[Robert Helmer [:rhelmer]]

Just tested on stage - for the most frequent signature:

 10477 | gfxContext::PushClipsToDT(mozilla::gfx::DrawTarget*)

Query over 365 days takes less than 1s, takes about 5s to finish reading all the data on localhost.

Comment 10

[Robert Helmer [:rhelmer]]

Not working on this currently - is this still broken, and if so would it still be useful to fix?

Comment 11

[Peter Bengtsson [:peterbe]]

Backfilling no longer applicable for crashes that old.