949282 - create SignaturesByBugs model

Status: RESOLVED FIXED

(Socorro :: Webapp, task)

Description

[Robert Helmer [:rhelmer]]

See bug 915373 comment 6, synopsis is that we need a django model (and underlying mware service) that takes a bug number and returns a list of associated signatures.

Something along the lines of:

SELECT status, resolution, short_desc, signature
FROM bugs 
JOIN bug_associations ON (bug_associations.bug_id = bugs.id)
WHERE bug_id = '219107';

However, I recall in other parts of the UI there was concern about exposing cached metadata for bugs that have been hidden for security reasons, so we may need to query Bugzilla on each request to ensure that the bug is still public...

Comment 1

[Robert Helmer [:rhelmer]]

(In reply to Robert Helmer [:rhelmer] from comment #0)
> However, I recall in other parts of the UI there was concern about exposing
> cached metadata for bugs that have been hidden for security reasons, so we
> may need to query Bugzilla on each request to ensure that the bug is still
> public...

lonnen, do you remember if this is the case ^?

Comment 2

[Lonnen :lonnen]

It is! It's not recommended to keep that information around longer than 10 minutes. There are also concerns about load on Bugzilla. It may be best to create a Bugzilla proxy service that uses modified-since to cache responses.

Comment 3

[Robert Helmer [:rhelmer]]

(In reply to Chris Lonnen :lonnen from comment #2)
> It is! It's not recommended to keep that information around longer than 10
> minutes. There are also concerns about load on Bugzilla. It may be best to
> create a Bugzilla proxy service that uses modified-since to cache responses.

Thanks!

So looking at the mware part of this first - I think we may want to just extend the current Bugs service, and make it able to accept sigantures or bugs as an argument, and return basically the same output in either case.

To address the Bugzilla security bug caching issue, we could have the mware query the Bugzilla API (using if-modified-since or last-modified header) and return cached info if the bug has not changed, otherwise refresh the cache. Then anything built on top of this could use it safely without having to worry about this problem (there's a requirement to expose this via the API, and we also want to use it for a new report UI)

Adrian, does this seem like getting "signatures for a given list of bugs" should be a separate service from "bugs for a given list of signatures" or does the above sound ok?

Comment 4

[Robert Helmer [:rhelmer]]

(In reply to Chris Lonnen :lonnen from comment #2)
> It is! It's not recommended to keep that information around longer than 10
> minutes. There are also concerns about load on Bugzilla. It may be best to
> create a Bugzilla proxy service that uses modified-since to cache responses.

OK after consulting w/ Bugzilla folks in #bmo, looks like the native REST API (which they recommend over the older BzAPI) supports etag but not last-modified header. So, assuming we stored the etag in our cache, something like https://gist.github.com/rhelmer/7952098 should work.

It would make the most sense if we could then update the caches (which are really the bugs and bug_assocations table), although it might be better conceptually to just continue to let the cron job do the writing and mware only read, since it runs every hour anyway.

Given the sensitivity around these, I wonder if it wouldn't be better to just switch to doing all of this in the Django app and use memcache instead? Maybe not blocking this bug, but as a refactoring after this lands...

Comment 5

[Robert Helmer [:rhelmer]]

Followup to comment 4 - alternative option to using etag is to fetch just the last_change_time https://bugzilla.mozilla.org/rest/bug/XXX?include_fields=last_change_time

We could just assume that the cache is invalid if last_change_time is within 10 minutes of "now", then do not trust the cache. This is probably the least change we can do to implement this feature, and revisit the overall caching strategy after this bug lands (per commment 4).

Comment 6

[Robert Helmer [:rhelmer]]

(In reply to Robert Helmer [:rhelmer] from comment #3)
> Adrian, does this seem like getting "signatures for a given list of bugs"
> should be a separate service from "bugs for a given list of signatures" or
> does the above sound ok?

Here is a quick&dirty implementation + docs + unit tests to show what I mean:

https://github.com/rhelmer/socorro/compare/bug949282-extend-bugs-service?expand=1

I don't see any services that do this though so maybe it'd be conceptually cleaner to implement a new mware service and have it share common code with the current "bugs" service?

Comment 7

[[DEACTIVATED] Adrian Gaudebert]

:rhelmer that looks good to me. You return the same data so it makes sense to use the same service.

Comment 8

[[github robot]]

Commits pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/d6f6ad671e0712fddc1ec912f1399bd6489972d5
fix bug 949282 extend bugs service to support query-by-bug

https://github.com/mozilla/socorro/commit/e8cf33837233909b15dcbfe5d1e41943e73116d9
Merge pull request #1746 from rhelmer/bug949282-extend-bugs-service

fix bug 949282 extend bugs service to support query-by-bug

Comment 9

[[github robot]]

Commits pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/e169eb1e60499b4108f510a8a87797fec27d6793
fixes bug 949282 - add SignaturesByBugs model

https://github.com/mozilla/socorro/commit/d299b937a356a7b9ce5bb3852a9cf9e15e3c7b6d
Merge pull request #1749 from rhelmer/bug949282-SignaturesByBugs-model

fixes bug 949282 - add SignaturesByBugs model