cvs-mirror.mozilla.org is not updating

VERIFIED FIXED

Status

P3
blocker
VERIFIED FIXED
18 years ago
4 years ago

People

(Reporter: mozilla, Assigned: harishm)

Tracking

Details

(Reporter)

Description

18 years ago
cvs-mirror.mozilla.org is not updating.  I haven't seen any new files appear in
two days now.  cvs stat shows that it thinks I am up-to-date while actually
using bonsai to compare file versions the cvs-mirror server is behind.

Comment 1

18 years ago
In the past this was caused by being out of disk space.

Comment 2

18 years ago
I can definately confirm this. Is there any way to pull from the main cvs in the
meantime?

Comment 3

18 years ago
I get an access denied from cvs.mozilla.org. Could you check the
permissions/allowed hosts, Dawn?
Status: NEW → ASSIGNED

Comment 4

18 years ago
hosts allow seems to be correct.

from the rsync log file, here is the last successful rsync from source forge
and the most recent failure. They're both from the same IP address so this
looks like a dns problem, not a config problem. /etc/rsyncd.conf was last
changed sept 26.  OTOH, nslookup seemed to behave correctly.  The sept 26
change set max connections to 1. Could that be the problem? Also while
we're at it could we fix the lines in the conf file that are causing all
the error messages? (transfer logging, use chroot, etc)

this seems like the problem is on thelizard. reassining to risto.

2000/10/22 19:19:52 [27202] rsync on mozilla from sourceforge.net (198.186.203.33)
2000/10/22 19:20:24 [27202] wrote 1142229 bytes  read 94 bytes  total size 930499981

2000/10/25 12:29:46 [2940] reverse name lookup failed
2000/10/25 12:29:46 [2940] rsync denied on module mozilla from unknown
(198.186.203.33)

root@thelizard:/etc/> nslookup 198.186.203.33
Server:  ns.netscape.com
Address:  198.95.251.10

Name:    sourceforge.net
Address:  198.186.203.33

It looks as though someone has changed the firewall config on mozilla.org's net
so that it only lets DNS packets to ns.netscape.com (198.95.251.10) through, and
not those to ns2.netscape.com, our backup server, or any other servers for that
matter.  I've worked around this by commenting out the backup server in
/etc/resolv.conf, but this is really bad, since we now have no backup server.
Please fix the firewall rules as soon as possible.
Assignee: dtype → rko
Status: ASSIGNED → NEW

Comment 6

18 years ago
Mosedale's change didn't make any difference. the attempted rsync at
15:19 failed again.

Note that this firewall horkage is causing other problems as well.
We can no longer send mail between thelizard and lounge. Since mail is
broken this is keeping people from checking in to cvs.mozilla.org
and/or keeping checkins from showing up in bonsai (bonsai.mozilla.org
on lounge.mozilla.org).

This breakage keeps CPD engineers from working. Marking severity as blocker.

Oct 25 15:22:31 thelizard.mozilla.org sendmail[9456]: e9PMLRn09456:
ruleset=check_mail, arg1=cvsmailfilter@thelizard.mozilla.org, relay=localhost
[127.0.0.1], reject=451 4.1.8 cvsmailfilter@thelizard.mozilla.org... Sender
domain must resolve


ntpdate has also been broken since oct 3
Oct 25 14:00:04 thelizard.mozilla.org ntpdate[6465]: no server suitable for
synchronization found


also it looks like weekly dumps have been failing have been failing
since sept.

USER       PID %CPU %MEM   SZ  RSS TT       S    START  TIME COMMAND
root      9842  0.0  0.1  848  704 pts/3    S 15:33:58  0:00 grep vxdump
root      2285  0.0  0.1 2080  640 ?        S   Sep 29  0:00 /usr/sbin/vxdump 6
root      2286  0.0  0.1 2080  640 ?        S   Sep 29  0:00 /usr/sbin/vxdump 6
root     22611  0.0  0.1 2080  648 ?        S   Oct 13  0:00 /usr/sbin/vxdump 6
root     22612  0.0  0.1 2080  648 ?        S   Oct 13  0:00 /usr/sbin/vxdump 6
root     24400  0.0  0.1 2080  640 ?        S   Oct 06  0:00 /usr/sbin/vxdump 6
root     24402  0.0  0.1 2080  640 ?        S   Oct 06  0:00 /usr/sbin/vxdump 6
root     27554  0.0  0.1 2080  648 ?        S   Oct 20  0:00 /usr/sbin/vxdump 6
root     27555  0.0  0.1 2080  648 ?        S   Oct 20  0:00 /usr/sbin/vxdump 6

I called the help desk and filed ticket 265177 which is being sent to
the network team.

reassigning to harishm since I think rko is out of town.
Assignee: rko → harishm
Severity: normal → blocker
this turned commercial trunk tinderbox red.  zoinks.
adding cls to the cc list.

Comment 9

18 years ago
Dumps... don't worry. If the tape is full it leaves these zombies behind. 
Usually it fills before the level 6 backups... thus all those leftover 
processes.

Comment 10

18 years ago
Seems like sourceforge's reverse DNS had been changes, thus *.sourceforge.net 
didn't match anymore and didn't let rsync. I've added cvs-mirror's IP to the 
allowed hosts and it seems to sync fine now. 

Harish, could you verify that this is fixed and if it is please close the 
ticket.

Comment 11

18 years ago
Running commands from command line seems to be syncing fine to cvs-mirror. Btw, 
I also raised limits for rsync max hosts. It was set to 1 because that was when 
we had some preformance problems with MacCVS clients loading the server and in 
elimination tried that too and forgot to put it back to what it was (3).
(Assignee)

Comment 12

18 years ago
Verified with Dawn Endico. And everything except the backup dns server is fixed.
There is another ticket alerady open for that.
Status: NEW → RESOLVED
Last Resolved: 18 years ago
Resolution: --- → FIXED
(Reporter)

Comment 13

18 years ago
Verified.  I just updated my tree and all the changes were retrieved.
Status: RESOLVED → VERIFIED
Product: mozilla.org → mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.