683934 - Cache addon info on privacy page for 1 day

Status: VERIFIED FIXED

(www.mozilla.org :: General, defect)

Description

[James Long (:jlongster)]

The /privacy page which was recently rolled out curls amo for addon information. It looks like there's a total of 4 curls which is ridiculously slow. Is this really necessary? The page load for when it does that is extremely long, sometimes even timing out.

Is there any other way we can do this? We could create a service which published the content every so often.

A temporary fix is to make it cache the results for 1 day instead of just 7 minutes. That way the page load isn't extremely slow every 7 minutes, and I doubt we need to update it that frequently.

Comment 1

[James Long (:jlongster)]

I should clarify: it's already caching, but for very short period of time. We can just bump the cache limit really high. curling 4 times on a page feels weird but heavy caching should fix it.

So disregard my comment about changing the actual code.

Comment 2

[Anthony Ricaud (:rik)]

Maybe we could make it a cronjob? No user would have a slow experience that way.

Comment 3

[James Long (:jlongster)]

Mike, if you have time you can refactor it out into a php file which can be called every so often to re-publish the content.

If you don't have time to do that, can you bump the cache to a day or so?

Comment 4

[Michael Gauthier [:gauthierm]]

I will refactor it into a cron script.

In the meantime, cache lifetime was bumped to 2 days in r94637.

Comment 5

[Anthony Ricaud (:rik)]

Pushed this with r94638.

Comment 6

[Laura Forrest]

Has this gone live? If so, let's close. If not, James or Anthony -- Push!

Michael - what say you?

Comment 7

[Steven Garrity [:sgarrity]]

(In reply to Laura Forrest from comment #6)
> Has this gone live? If so, let's close. If not, James or Anthony -- Push!

Not yet - got pushed by some work on the brand toolkit. Will aim for 3.12

Comment 8

[Michael Gauthier [:gauthierm]]

Refactored into a script runnable by cron in r95582.

It should also now use Memcache if available though it's not been tested yet.

The cron script is: includes/feeds/feed-cache-cron.php

It can run at any frequency as it uses cache timeouts to know when to refresh data. I suggest every 2 minutes.

The cron script uses a config file. A config.ini.dist was committed with r95582. It should be renamed config.ini and modified as necessary.

Comment 9

[James Long (:jlongster)]

(In reply to Michael Gauthier from comment #8)
> Refactored into a script runnable by cron in r95582.

Cool! Thanks!

> It should also now use Memcache if available though it's not been tested yet.

I'm not sure he have memcache for moz.org, just Zeus caching. Is there any way you can test the memcache code? If we turn on memcache, I'd hate to break the script because there was an error in the code. Either that or we don't roll that part out yet.

> The cron script uses a config file. A config.ini.dist was committed with
> r95582. It should be renamed config.ini and modified as necessary.

How do you think we should roll this out? Just push all this code and then have IT turn the cron job on?

Comment 10

[Michael Gauthier [:gauthierm]]

(In reply to James Long (:jlongster) from comment #9)
> (In reply to Michael Gauthier from comment #8)
> > Refactored into a script runnable by cron in r95582.
> 
> Cool! Thanks!
> 
> > It should also now use Memcache if available though it's not been tested yet.
> 
> I'm not sure he have memcache for moz.org, just Zeus caching. Is there any
> way you can test the memcache code? If we turn on memcache, I'd hate to
> break the script because there was an error in the code. Either that or we
> don't roll that part out yet.
> 
I based the memcache init on includes/feeds/mobile-team.php. I'll see if we can set up memcache locally to ensure the code at least does not error out.

> > The cron script uses a config file. A config.ini.dist was committed with
> > r95582. It should be renamed config.ini and modified as necessary.
> 
> How do you think we should roll this out? Just push all this code and then
> have IT turn the cron job on?
> 
How the cron is set up somewhat depends on whether or not Memcache is available. If Memcache is available, the cron only needs to be set up on one machine as it will populate a shared cache. If memcache is not available, the cron needs to run on every server that has the code as it will use a file-cache that's not shared between servers.

Comment 11

[Michael Gauthier [:gauthierm]]

(In reply to Michael Gauthier from comment #10)
> (In reply to James Long (:jlongster) from comment #9)
> > (In reply to Michael Gauthier from comment #8)
> > > Refactored into a script runnable by cron in r95582.
> > 
> > Cool! Thanks!
> > 
> > > It should also now use Memcache if available though it's not been tested yet.
> > 
> > I'm not sure he have memcache for moz.org, just Zeus caching. Is there any
> > way you can test the memcache code? If we turn on memcache, I'd hate to
> > break the script because there was an error in the code. Either that or we
> > don't roll that part out yet.
> > 
> I based the memcache init on includes/feeds/mobile-team.php. I'll see if we
> can set up memcache locally to ensure the code at least does not error out.
> 
Ok, I tested locally and the memcache part is working as intended. If the memcache config exists on the live site it will be used.

Comment 12

[Steven Garrity [:sgarrity]]

Merged to stage in r96360 so IT can test the cron jobs over in Bug 692115.

Comment 13

[Steven Garrity [:sgarrity]]

The cron job is setup on dev/stage and seems to be working fine. James, can you push this code to production?

Comment 14

[James Long (:jlongster)]

pushed in r96766

Comment 15

[raymond [:retornam] (inactive)]

verified fixed http://www.mozilla.org/en-US/privacy/