Cache addon info on privacy page for 1 day

VERIFIED FIXED in 4.3

Status

www.mozilla.org
General
VERIFIED FIXED
7 years ago
6 years ago

People

(Reporter: jlongster, Assigned: gauthierm)

Tracking

unspecified

Firefox Tracking Flags

(Not tracked)

Details

(URL)

(Reporter)

Description

7 years ago
The /privacy page which was recently rolled out curls amo for addon information. It looks like there's a total of 4 curls which is ridiculously slow. Is this really necessary? The page load for when it does that is extremely long, sometimes even timing out.

Is there any other way we can do this? We could create a service which published the content every so often.

A temporary fix is to make it cache the results for 1 day instead of just 7 minutes. That way the page load isn't extremely slow every 7 minutes, and I doubt we need to update it that frequently.
(Reporter)

Updated

7 years ago
Assignee: nobody → steven
Target Milestone: --- → 3.9
(Reporter)

Updated

7 years ago
(Reporter)

Comment 1

7 years ago
I should clarify: it's already caching, but for very short period of time. We can just bump the cache limit really high. curling 4 times on a page feels weird but heavy caching should fix it.

So disregard my comment about changing the actual code.
Maybe we could make it a cronjob? No user would have a slow experience that way.
Assignee: steven → mike
(Reporter)

Comment 3

7 years ago
Mike, if you have time you can refactor it out into a php file which can be called every so often to re-publish the content.

If you don't have time to do that, can you bump the cache to a day or so?
(Assignee)

Comment 4

7 years ago
I will refactor it into a cron script.

In the meantime, cache lifetime was bumped to 2 days in r94637.
Whiteboard: r94637 b=trunk
Pushed this with r94638.

Updated

7 years ago
Target Milestone: 3.9 → 3.10
Target Milestone: 3.10 → 3.11

Updated

7 years ago
Target Milestone: 3.11 → 3.12

Comment 6

7 years ago
Has this gone live? If so, let's close. If not, James or Anthony -- Push!

Michael - what say you?
(In reply to Laura Forrest from comment #6)
> Has this gone live? If so, let's close. If not, James or Anthony -- Push!

Not yet - got pushed by some work on the brand toolkit. Will aim for 3.12
Target Milestone: 3.12 → 4.0
(Assignee)

Comment 8

7 years ago
Refactored into a script runnable by cron in r95582.

It should also now use Memcache if available though it's not been tested yet.

The cron script is: includes/feeds/feed-cache-cron.php

It can run at any frequency as it uses cache timeouts to know when to refresh data. I suggest every 2 minutes.

The cron script uses a config file. A config.ini.dist was committed with r95582. It should be renamed config.ini and modified as necessary.
Whiteboard: r94637 b=trunk → r94637,r95582 b=trunk
(Assignee)

Updated

7 years ago
Whiteboard: r94637,r95582 b=trunk → r94637,r95582, r95583 b=trunk
Depends on: 692115
No longer depends on: 692115
Keywords: qawanted
OS: Mac OS X → All
Hardware: x86 → All
Blocks: 692115
(Reporter)

Comment 9

7 years ago
(In reply to Michael Gauthier from comment #8)
> Refactored into a script runnable by cron in r95582.

Cool! Thanks!

> It should also now use Memcache if available though it's not been tested yet.

I'm not sure he have memcache for moz.org, just Zeus caching. Is there any way you can test the memcache code? If we turn on memcache, I'd hate to break the script because there was an error in the code. Either that or we don't roll that part out yet.

> The cron script uses a config file. A config.ini.dist was committed with
> r95582. It should be renamed config.ini and modified as necessary.

How do you think we should roll this out? Just push all this code and then have IT turn the cron job on?
(Assignee)

Comment 10

7 years ago
(In reply to James Long (:jlongster) from comment #9)
> (In reply to Michael Gauthier from comment #8)
> > Refactored into a script runnable by cron in r95582.
> 
> Cool! Thanks!
> 
> > It should also now use Memcache if available though it's not been tested yet.
> 
> I'm not sure he have memcache for moz.org, just Zeus caching. Is there any
> way you can test the memcache code? If we turn on memcache, I'd hate to
> break the script because there was an error in the code. Either that or we
> don't roll that part out yet.
> 
I based the memcache init on includes/feeds/mobile-team.php. I'll see if we can set up memcache locally to ensure the code at least does not error out.

> > The cron script uses a config file. A config.ini.dist was committed with
> > r95582. It should be renamed config.ini and modified as necessary.
> 
> How do you think we should roll this out? Just push all this code and then
> have IT turn the cron job on?
> 
How the cron is set up somewhat depends on whether or not Memcache is available. If Memcache is available, the cron only needs to be set up on one machine as it will populate a shared cache. If memcache is not available, the cron needs to run on every server that has the code as it will use a file-cache that's not shared between servers.
(Assignee)

Comment 11

7 years ago
(In reply to Michael Gauthier from comment #10)
> (In reply to James Long (:jlongster) from comment #9)
> > (In reply to Michael Gauthier from comment #8)
> > > Refactored into a script runnable by cron in r95582.
> > 
> > Cool! Thanks!
> > 
> > > It should also now use Memcache if available though it's not been tested yet.
> > 
> > I'm not sure he have memcache for moz.org, just Zeus caching. Is there any
> > way you can test the memcache code? If we turn on memcache, I'd hate to
> > break the script because there was an error in the code. Either that or we
> > don't roll that part out yet.
> > 
> I based the memcache init on includes/feeds/mobile-team.php. I'll see if we
> can set up memcache locally to ensure the code at least does not error out.
> 
Ok, I tested locally and the memcache part is working as intended. If the memcache config exists on the live site it will be used.

Updated

7 years ago
Target Milestone: 4.0 → 4.1

Updated

7 years ago
Target Milestone: 4.1 → 4.2
Merged to stage in r96360 so IT can test the cron jobs over in Bug 692115.
Whiteboard: r94637,r95582, r95583 b=trunk → r=96360 b=stage
The cron job is setup on dev/stage and seems to be working fine. James, can you push this code to production?
Status: NEW → RESOLVED
Last Resolved: 7 years ago
Resolution: --- → FIXED
Target Milestone: 4.2 → 4.3
(Reporter)

Comment 14

7 years ago
pushed in r96766
(Reporter)

Updated

7 years ago
Whiteboard: r=96360 b=stage
verified fixed http://www.mozilla.org/en-US/privacy/
Status: RESOLVED → VERIFIED
Keywords: qawanted
Component: www.mozilla.org/firefox → www.mozilla.org
Product: Websites → Websites
Component: www.mozilla.org → General
Product: Websites → www.mozilla.org
You need to log in before you can comment on or make changes to this bug.