Setup cron and config for feed imports on mozilla.org/firefox pages

RESOLVED FIXED

Status

Infrastructure & Operations
WebOps: Other
RESOLVED FIXED
6 years ago
4 years ago

People

(Reporter: sgarrity, Assigned: jakem)

Tracking

Details

(Reporter)

Description

6 years ago
A few pages that import RSS feeds on mozilla.org/firefox pages have now been cleaned up and can now use a common cron job.

The cron script is: includes/feeds/feed-cache-cron.php

It can run at any frequency as it uses cache timeouts to know when to refresh data. We suggest every 2 minutes.

The cron script uses a config file. A config.ini.dist was committed with r95582. It should be renamed config.ini and modified as necessary.
(Reporter)

Updated

6 years ago
No longer blocks: 683934
(Reporter)

Updated

6 years ago
Depends on: 683934
(Assignee)

Comment 1

6 years ago
It looks like these changes have not been pushed to tags/production, so we can't do this just yet.

Also, there are 2 other feed-related cron's for mozilla.org:

*/10 * * * * root cd /data/static/www/www.mozilla.org-merged/org/includes/; ./feed-parser.php -c ./feed-parser-config.xml
25 * * * *   root cd /data/static/www/www.mozilla.org-merged/tools; php -f refresh-press-center-feed.php

Does this obsolete either of those?
Also, this should be on stage before it goes to prod.

Comment 3

6 years ago
Jake, looks like you know what is going on here, so assigning to you.
Assignee: server-ops → nmaul
Jake, this new cron script does NOT obsolete either of those.
(Reporter)

Comment 5

6 years ago
(In reply to Shyam Mani [:fox2mike] from comment #2)
> Also, this should be on stage before it goes to prod.

Yeah, can you set this up on stage first?
(Assignee)

Comment 6

6 years ago
I still can't do this. This hasn't been pushed to tags/production.

Furthermore, if you want to test in stage, it needs to be pushed to tags/stage as well.
(Reporter)

Comment 7

6 years ago
(In reply to Jake Maul [:jakem] from comment #6)
> Furthermore, if you want to test in stage, it needs to be pushed to
> tags/stage as well.

Merged to stage in r96360.
(Assignee)

Comment 8

6 years ago
This has been completed on dev and stage. Please let me know how it looks and when this is pushed to tags/production, and I can get it done there too. Thanks!
Status: NEW → ASSIGNED
(Reporter)

Comment 9

6 years ago
(In reply to Jake Maul [:jakem] from comment #8)
> This has been completed on dev and stage. Please let me know how it looks
> and when this is pushed to tags/production, and I can get it done there too.

Thanks. Looks good on stage, and the code is now pushed to production (see Bug 683934). Please implement this cron on production and we're all set.
(Assignee)

Comment 10

6 years ago
Hmm, there's a problem with this...

[root@mradm02 feeds]# php feed-cache-cron.php
Feed cache cron config '/data/www/www.mozilla.org-merged/includes/feeds/cron-config.ini' missing.

The problem is that feed-cache-cron.php uses the "file_root" variable from includes/config.inc.php. The path in that file works on the web nodes, but not the admin node where the cron job runs.

This works on www.allizom.org and www-dev.allizom.org because those are both one node only... they don't have a separate admin node. That will change in the near future with the move to the bedrock cluster, so those will break in the future as well.

I see file_root is also referenced once in FeedCache.php.


Could this be reworked to use a relative directory and not rely on file_root ?
Reworked to use relative directories in r96837 in trunk.
(Assignee)

Comment 12

6 years ago
Excellent, this still seems to work properly on www-dev.allizom.org, and I think it'll be good on prod now, too. Can we get this pushed to tags/stage and tags/production?
(Reporter)

Comment 13

6 years ago
James, can you help us get this pushed to production?
Assignee: nmaul → jlong
pushed to production r97199
Status: ASSIGNED → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
(Reporter)

Comment 15

6 years ago
(In reply to Jake Maul [:jakem] from comment #12)
> Excellent, this still seems to work properly on www-dev.allizom.org, and I
> think it'll be good on prod now, too. Can we get this pushed to tags/stage
> and tags/production?

Jake, can you confirm that this cron is in place in production?
Assignee: jlong → nmaul
(Assignee)

Comment 16

6 years ago
No, it was not. It is now.

It took a very long time to run the first time, but seems quick now. Guessing it needed to populate some past data or something.
Component: Server Operations: Web Operations → WebOps: Other
Product: mozilla.org → Infrastructure & Operations
You need to log in before you can comment on or make changes to this bug.