712079 - Fix product_info cache control

Status: VERIFIED FIXED

(Socorro :: General, task)

Description

[Laura Thomson :laura]

Bug 677588 changed the webapp from using product_dims to product_info, but it also added caching to the model.  

This has resulted in problems like bug 712049 and bug 707991.  To fix it, cache control should move from the underlying model to the controller, and be settable on a per-view basis.  In particular, relevant admin changes should be able to flush this cache on write.

Comment 1

[Lonnen :lonnen]

8 controllers, including the base controller, access the model directly. If we're moving cache control away from the model, should these be rewritten to access the info through the controller? Would it be better to rewrite the model's with write-through caching, which would also alleviate bugs like 712049 and 707991?

Comment 2

[Lonnen :lonnen]

Going to try read-through instead. Writing to the table, via the webapp at least, will invalidate the cache. On read, check the cache first and query on miss.

Because of the slow transition from directly querying to the web service model it looks like there are at least 3 independent, incomplete attempts at caching this information at different levels of the webapp. One in the branch model, one in the base model, and one in the webservice library; all are coupled to each other and/or other parts of the app.

Comment 3

[Robert Kaiser]

(In reply to Chris Lonnen :lonnen from comment #2)
> Going to try read-through instead. Writing to the table, via the webapp at
> least, will invalidate the cache. On read, check the cache first and query
> on miss.

Will this also take care of updating correctly when we add versions through FTP scraping? We recently had problems with caching in that case as well.

Comment 4

[Lonnen :lonnen]

adding key expiration should help with that, and if its not enough we'll figure out how to flush memcached from the scraper script.

Comment 5

[Lonnen :lonnen]

rhelmer, laura: r?

Tested on my local VM. The first commit contains the actual fix while the second contains some code formatting.

rationale:
The in-model cache overlapped other memcache-based caches in the Webservice library and Model base class. Since these are both read-through (check first, grab from the db if its not there) I reused them as they were. This patch removes the in-model caching, and adds a cache-flushing step to any operation writing to the database. Following a write there will always be a cache miss, but it will occur when the admin panel reloads, making this almost as efficient as write-through with less code complexity.

Steps to reproduce the problem:
1. Enter the admin panel. Edit a featured version for a product in the product selector. Unmark it as featured and hit update.
2. Check the `current versions` selector menu in the navigation to see that the product has not been removed.
3. Check the entry in the table to see that your changes have apparently not been applied
4. Restart memcached
5. Verify that the version is now missing from the selector menu and that the table accurately reflects your changes.

To verify the fix:
Follow steps 1 and then 5 above. That should be all. You could also delete / add a product to see that changes are reflected immediately.

Comment 6

[Lonnen :lonnen]

landed with r+ from rhelmer:

https://github.com/mozilla/socorro/pull/250

Comment 7

[Matt Brandt [:mbrandt]]

Verified on stage per steps in comment 5. Thanks lonnen.