PAGELIST data is not being updated on prod

VERIFIED FIXED in 1.4.1

Status

support.mozilla.org
General
P5
major
VERIFIED FIXED
9 years ago
8 years ago

People

(Reporter: cilias, Assigned: jsocol)

Tracking

unspecified
1.4.1

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: sumo_only, URL)

(Reporter)

Description

9 years ago
Go to <https://support.mozilla.com/en-US/kb/Article+list?bl=n>.

"Clear Recent History" is not listed. It should be.

Another article to look for, which isn't there, is "Backing up and restoring bookmarks".

The script seems to successfully export to <http://dm-sumotools01.mozilla.org/exports/export.txt>.

As I understand it, this also affects the l10n dashboard.
(Reporter)

Updated

9 years ago
Target Milestone: --- → 1.3
(Reporter)

Comment 1

9 years ago
I don't know if the code that needs changing is in the sumo svn code or somewhere else, so I'm setting this to 1.3 just in case.

Comment 2

9 years ago
This works locally (once I connect to the VPN):

ecooper@sisyphus:~/projects/sumo/local/trunk/scripts$ lphp fetch_sumo_lists.php http://dm-sumotools01.mozilla.org/exports/export.txt
3 of 3 lists updated!

Most likely the cron on prod has died/been removed. I can't find the bug for setting this up on prod, but bug 484809 was for testing it on stage. We should have IT check for a weekly cron that runs "/scripts/fetch_sumo_lists.php http://dm-sumotools01.mozilla.org/exports/export.txt" or something similar.
(Reporter)

Comment 3

9 years ago
Moving to IT, so someone from IT can check re comment 2.

"We should
have IT check for a weekly cron that runs "/scripts/fetch_sumo_lists.php
http://dm-sumotools01.mozilla.org/exports/export.txt" or something similar."
Assignee: nobody → server-ops
Component: Knowledge Base Software → Server Operations
Product: support.mozilla.com → mozilla.org
QA Contact: kb-software → mrz
Target Milestone: 1.3 → ---
Version: unspecified → other

Updated

9 years ago
Assignee: server-ops → jeremy.orem+bugs

Updated

9 years ago
Assignee: jeremy.orem+bugs → shyam
(Reporter)

Comment 4

9 years ago
I noticed that <http://dm-sumotools01.mozilla.org/en-US/kb/Article+list> is not updated either, if that helps.
[root@mradm02 cron.d]# cat support.mozilla.com 
# vim: syntax=crontab

# cron jobs for support.mozilla.com

# regenerate the l10n dashboard every Monday at noon
0 12 * * 1 apache cd /data/php5/www/support.mozilla.com/scripts; php fetch_sumo_lists.php http://dm-sumotools01.mozilla.org/exports/export.txt

That's actually outdated. The location has changed to /data/generic/www 

I've updated the cronjob to point to the correct location now. Reopen if it's not updated by next Tuesday or so.
Status: NEW → RESOLVED
Last Resolved: 9 years ago
Resolution: --- → FIXED
(Reporter)

Comment 6

9 years ago
It's Tuesday, and it's still not updated, so reopening. :-(
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
From /var/log/cron :

Sep 14 12:00:01 mradm02 crond[4150]: (apache) CMD (cd /data/generic/www/support.mozilla.com/scripts; php fetch_sumo_lists.php http://dm-sumotools01.mozilla.org/exports/export.txt)

Checking to see if this runs by hand
Chris,

So the issue is this : The scripts folder on mradm02 where the SUMO scripts reside, is an NFS partition...and so for the script to correctly work, the paths have to be absolute, not relative. I made the following changes for it to work :

[root@mradm02 scripts]# svn diff tiki_script_base.php 
Index: tiki_script_base.php
===================================================================
--- tiki_script_base.php        (revision 51077)
+++ tiki_script_base.php        (working copy)
@@ -7,15 +7,15 @@
  * libraries can be used if needed.
  *
  */  
-require_once('../webroot/lib/init/initlib.php');
-require_once('../webroot/lib/setup/tikisetup.class.php');
+require_once('/data/generic/www/support.mozilla.com/webroot/lib/init/initlib.php');
+require_once('/data/generic/www/support.mozilla.com/webroot/lib/setup/tikisetup.class.php');
 
-TikiSetup::appendIncludePath('../webroot');
-TikiSetup::appendIncludePath('../webroot/lib');
-TikiSetup::appendIncludePath('../webroot/lib/pear');
-TikiInit::appendIncludePath('../webroot/lib/adodb');
+TikiSetup::appendIncludePath('/data/generic/www/support.mozilla.com/webroot');
+TikiSetup::appendIncludePath('/data/generic/www/support.mozilla.com/webroot/lib');
+TikiSetup::appendIncludePath('/data/generic/www/support.mozilla.com/webroot/lib/pear');
+TikiInit::appendIncludePath('/data/generic/www/support.mozilla.com/webroot/lib/adodb');
 
 require_once('db/tiki-db.php');
 require_once('lib/tikilib.php');
 
-?>
\ No newline at end of file
+?>

So I guess you can make those changes and check in the script file (if this is the ONLY place this is being used) and the cron should work fine from next week. I can maybe change the frequency of the cron and we can check to see if it's running.
(Reporter)

Comment 10

9 years ago
Thanks Shyam.
Moving this over to sumodev, because I don't have check-in privs.
Assignee: shyam → nobody
Component: Server Operations → General
Product: mozilla.org → support.mozilla.com
QA Contact: mrz → general
Version: other → unspecified
(Reporter)

Updated

9 years ago
Target Milestone: --- → 1.4

Updated

9 years ago
Target Milestone: 1.4 → 1.4.1
(Assignee)

Comment 11

8 years ago
Shyam, do you know of any reason why PHP's realpath() function would not work, or could we use that to maintain generality?
James,

I've not looked up realpath() but I'm assuming it'd find out the path from the symlink. I'm open to giving it a try :) IANAD and so my "solutions" aren't exactly the right ones.
Shyam, did you find out why scripts is a symlink to NFS?  If it could just be normal then that would make this problem go away and avoid future push problems like we had for 1.4.  I think you said you would look into it.

If we have to have it that way, then let's it get it set up that way on stage so we have some way of testing.
(In reply to comment #13)

> Shyam, did you find out why scripts is a symlink to NFS?  

Trevor explained over email : 

Supposedly the admin interface needed access to change these:
cfb183ba76f93da6471d39ea5b8d6f25  binary_weights.txt
cfb183ba76f93da6471d39ea5b8d6f25  binary_weights.txt.dist
98595e09c2a06ce1d1acba0d549d4380  stopwords.txt
98595e09c2a06ce1d1acba0d549d4380  stopwords.txt.dist
54c5f6c7ec1af678dcad18165fe7572d  weights_weights.txt
54c5f6c7ec1af678dcad18165fe7572d  weights_weights.txt.dist
a62d9b2cae4ecd73d7eb41a616beae4a  withc_weights.txt
a62d9b2cae4ecd73d7eb41a616beae4a  withc_weights.txt.dist

This was discovered during a production push and NFS was the quick fix. But since they don't actually change, we should move it off NFS.

-Trevor

So if these files need not be accessed by the admin interface I could just remove the symlink tomm and svn switch to get the folder and then push that out.
I'm wondering if a good solution for the future is this:
If we want to allow scripts to change some files, we can write somewhere (shared place A) but _ask IT_ or _cron_ delivery of copies to all webheads. This would basically mean: have our admin pages write to a specific location, but the webheads read from their own.

Rest of team, what do you think about this?

This should work well for several of our bugs -- including, for example, the ability to have a hard-js file stored from the AAQ admin panel. It would also solve bug 518764, since we could have minify.conf be written somewhere but copied by scripts to the servers for use.

Maybe I'm missing something, but this seems to work well, even for stuff that we want to change relatively frequently?
(Assignee)

Comment 16

8 years ago
(In reply to comment #15)
> I'm wondering if a good solution for the future is this:
> If we want to allow scripts to change some files, we can write somewhere
> (shared place A) but _ask IT_ or _cron_ delivery of copies to all webheads.
> This would basically mean: have our admin pages write to a specific location,
> but the webheads read from their own.

How would we handle mid-airs?
Mid-airs of?
In my mind, though perhaps not in my phrasing, the write-to and read-from were separate locations.
(Assignee)

Updated

8 years ago
Assignee: nobody → james
Priority: -- → P5
(Assignee)

Comment 18

8 years ago
I'd like to just implement Shyam's solution in comment 14. If the files don't practically change, then it seems like NFS is just unnecessary. This would solve our path issues, remove the potential for scripts/ to get out of sync during pushes, and couldn't hurt performance (Shyam: I'm assuming IO is faster from local drives than from NFS?).

Unless there are objections, maybe we can do this during the 1.4.1 push?
(In reply to comment #18)
> during pushes, and couldn't hurt performance (Shyam: I'm assuming IO is faster
> from local drives than from NFS?).

That is correct, local drives should be a decent bit faster compared to NFS.
(Assignee)

Comment 20

8 years ago
This was fixed during the push of 1.4.1 (bug 523264).
Status: REOPENED → RESOLVED
Last Resolved: 9 years ago8 years ago
Resolution: --- → FIXED
(In reply to comment #20)
> This was fixed during the push of 1.4.1 (bug 523264).

Chris, can you verify?  Looks good to me.
(Reporter)

Comment 22

8 years ago
"Will Firefox work on Windows 7" was created on Friday and appears on the article list, so assuming the script wasn't run by hand, this if verified.
Status: RESOLVED → VERIFIED
Nope, didn't run it by hand :)

Updated

8 years ago
Whiteboard: sumo_only
You need to log in before you can comment on or make changes to this bug.