729599 - Add whole-page caching to team pages

Status: RESOLVED WONTFIX

(Webtools Graveyard :: Elmo, defect, P3)

Description

[Peter Bengtsson [:peterbe]]

The team pages are highly optimized (thanks Pike) but still potentially too slow. 

We can optimize by loading the whole page into cache and thus reducing the number of queries down to 0. However, we want the page to stay fresh and not get stale when new pushes come in or relevant sign-offs are changed. Therefore, when we cache the whole page we need to know the cache key and thus invalidate it if any pushes comes in or sign-offs relevant that page.

Comment 1

[Axel Hecht [:Pike]]

Zeus apparently does a descent job with that already, should we just do ETAGs?

Not sure if they need to be as computationally involved as we're doing with the signoff view right now, maybe convert both to signals?

Comment 2

[Peter Bengtsson [:peterbe]]

Good point but I honestly don't like it. Reverse HTTP caches means we lose control of invalidation. 

To do that, we have to send out Cache-Control headers. Then we're saying to the client that this page won't change for another X hours. But it might! If we instead tightly control the caching, we can invalidate ASAP. I think that might be important in case a team member looks on the team page, goes and does some hg commit/push and then reloads and expects to see the page to have changed. 

Or, am I aiming too high?

Comment 3

[Axel Hecht [:Pike]]

That's why I was thinking of putting just an ETAG into our cache, and flush that on notification instead of putting the whole page into our cache and flush it.

For the sign-off view, I think our ETAG computation isn't all that much faster than the page itself now, so I consider computing that from scratch to be kindof of a failure, but if we just cache and invalidate them, we might be good.

Comment 4

[Peter Bengtsson [:peterbe]]

(In reply to Axel Hecht [:Pike] from comment #3)
> That's why I was thinking of putting just an ETAG into our cache, and flush
> that on notification instead of putting the whole page into our cache and
> flush it.
> 
Of course! I didn't think of that. Means we don't have to do any caching of the rendered HTML ourselves. 

> For the sign-off view, I think our ETAG computation isn't all that much
> faster than the page itself now, so I consider computing that from scratch
> to be kindof of a failure, but if we just cache and invalidate them, we
> might be good.

Thing is, I was hoping to use Model signals to notice when certain team pages should be refreshed. What about doing something like this::

 @receiver(post_save, sender=SignOff)
 def note_latest_signoff(sender, instance, **kwargs):
    locales = get_relevant_locales(instances)
    for locale in locales:
        cache_key = 'latestsignoff-%s' % locale.code
        cache.set(cache_key, instance.timestamp, 60 * 60)

Then we can look for this cache key inside the ETAG decorator. 
(obviously the code would need a lot of love and more though)

Comment 5

[Axel Hecht [:Pike]]

Yep, sounds good. Probably want to listen to shipping.Action and l10nstats.Run

Comment 6

[Axel Hecht [:Pike]]

Doesn't seem that we're having pressing pageload problems on subsequent load. This probably wouldn't help initial page load, so we're tabling this for now.

If we want to come back to it, we'll reopen.