1169161 - www.seamonkey-project.org isn't being updated.

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: WebOps: Other, task)

Description

[Edmund Wong (:ewong)]

(I'm not exactly sure if this is the right product/component combination.)

After the 23rd April's cron job mail, we haven't heard from the website
updates.

Bug 1120709 migrated SeaMonkey's website from CVS to Mercurial.  This would've
required updates on the cron jobs that regularly polls the SeaMonkey website's
repo for changes and then update the website.

Has there been a change in the website update cron job?  Is it even running?

Comment 1

[Ludovic Hirlimann [:Usul]]

Where is seamonkey hosted , on the generic cluster ?

Comment 2

[Justin Wood (:Callek)]

(In reply to Ludovic Hirlimann [:Usul] from comment #1)
> Where is seamonkey hosted , on the generic cluster ?

I'm not sure top of my head, but Shyam did the migration of the code.

Let's see:

Name:    seamonkey-project.org
Address:  63.245.217.20

...

Name:    static.zlb.phx.mozilla.net
Address:  63.245.217.20

Comment 3

[Ludovic Hirlimann [:Usul]]

(In reply to Justin Wood (:Callek) from comment #2)
> (In reply to Ludovic Hirlimann [:Usul] from comment #1)
> > Where is seamonkey hosted , on the generic cluster ?
> 
> I'm not sure top of my head, but Shyam did the migration of the code.
> 
> Let's see:
> 
> Name:    seamonkey-project.org
> Address:  63.245.217.20
> 
> ...
> 
> Name:    static.zlb.phx.mozilla.net
> Address:  63.245.217.20

ok the static cluster I'll have a poke today.

Comment 4

[Ludovic Hirlimann [:Usul]]

Didn't find any thing in staticadm's crontab nor on the nodes. Found noting useful after checking puppet as described at https://mana.mozilla.org/wiki/display/websites/Static+Cluster#StaticCluster-Automation.

Over to webops.

Comment 5

[C. Liang [:cyliang]]

I think that I've manually cleared up the clog blocking the cron jobs from running properly.  Please let me know if they are now working correctly. 


According to /var/log/cron on the static admin host [1], the cron jobs that update the site are being run. [2]  It looks like there is a frequent job that runs build-exclusive.pl and less frequent one that does a build_full-exclusive.pl.  Temporarily disabling the cron jobs.  Running jobs manually, I got no obvious errors with build-exclusive.pl but did see issues with the build_full-exclusive script getting a file lock.   Looking at a similarly named lock file in /tmp and tracing through the process list, it looks like a build-exclusive.pl job got stuck back on May 14th on trying to contact hg.mozilla.org. [3]

After killing the May 14th processes, I was able to get a succesful build-exclusive.pl run.  I've re-instated the cron jobs.  



[1] cron jobs are usually run on the admin host for the cluster and can be found in /etc/cron.d
[2] e.g. May 29 08:01:01 staticadm.private.phx1.mozilla.com CROND[5669]: (root) CMD ((cd /data/static/build/seamonkeyproject-org; bin/build-exclusive.pl; /data/static/deploy -nq www.seamonkeyproject.org))
[3] $ ls -lad /tmp/sm*
-rw-r--r-- 1 root root 5 May 14 11:00 /tmp/smprojorg-build.pid 

Contents of /tmp/smprojorg-build.pid pointed at process 2666 ...

$ ps -ef | grep 2666
root      2666  2656  0 May14 ?        00:00:00 /usr/bin/perl -w bin/build-exclusive.pl
root      2705  2666  0 May14 ?        00:00:00 /bin/sh bin/build

...

root      2705  2666  0 May14 ?        00:00:00 /bin/sh bin/build
root      2707  2705  0 May14 ?        00:00:00 /usr/bin/python /usr/bin/hg pull -q -u

strace of 20707 showed that it was stuck in recv.

Comment 6

[Jens Hatlak (:InvisibleSmiley)]

(In reply to C. Liang [:cyliang] from comment #5)
> I think that I've manually cleared up the clog blocking the cron jobs from
> running properly.  Please let me know if they are now working correctly. 

I can at least confirm that the website now matches the latest update of our hg repository (http://hg.mozilla.org/SeaMonkey/seamonkey-project-org/rev/41b095f5170c). But if I understand you correctly, you initiated this update manually so I cannot tell whether the cronjobs work correctly now.

Comment 7

[C. Liang [:cyliang]]

I'm not sure how often the repository normally gets updates.  If it doesn't happen often, we can test if the cronjobs are working if someone can commit a cosmetic change (white space, comment, etc.)

Comment 8

[Jens Hatlak (:InvisibleSmiley)]

The last update that I committed (changeset 46c701397e00) resulted in an update of the website, so this seems to work now. Closing.