80587 - cvs-mirror is ignoring 'core' directories

Status: RESOLVED FIXED

(mozilla.org Graveyard :: Server Operations, task, P1)

Description

[Bradley Baetz]

Because the cvs server version on cvs.mozilla.org is old, and doesn't do
multi-dir diffs properly, I tend to diff against cvs-mirror for large diffs.

When I do this (using cvs -d) on the layout directory, I get:

cvs server: failed to create lock directory in repository
`/cvsroot/mozilla/layout/html/tests/table/core': No such file or directory
cvs server: failed to obtain dir lock in repository
`/cvsroot/mozilla/layout/html/tests/table/core'
cvs [server aborted]: read lock failed - giving up

and the diff fails. There are a couple of other core directories in the tests as
well.

imoT (on IRC) mentioned that from cvs-mirror, the directory is never checked
out, and layout/html/tests/tags is also missed.

<rko> rsync misses it if you have --cvs-ignore set and I think we do
<rko> I had to remove it from lounge copy (for bonsai) and it resolved it
<rko> Drew needs to check the mirror

Comment 1

[Risto Kotalampi]

cvs-mirror issue

Comment 2

[Tomi Leppikangas]

When doing normal cvs checkout from cvs-mirror i get these errors:

cvs server: cannot open directory
/cvsroot/mozilla/layout/html/tests/frameset/core: No such file or directory
cvs server: skipping directory mozilla/layout/html/tests/frameset/core
cvs server: cannot open directory /cvsroot/mozilla/layout/html/tests/table/core:
No such file or directory
cvs server: skipping directory mozilla/layout/html/tests/table/core
cvs server: cannot open directory /cvsroot/mozilla/layout/html/tests/tags: No
such file or directory
cvs server: skipping directory mozilla/layout/html/tests/tags

Comment 3

[Dawn Endico]

spoke to drew on the phone. He said that rko added --cvs-ignore to cvs-mirror
way back when so lock files didn't get copied over. Mirroring lock files is 
extremely painful and there is no reason to. He said he would come up with
a new way to exclude lock files and then remove --cvs-ignore.

I beleive this is the cause of the problem terry saw with bonsai/cvslog.cgi

Comment 4

[Terry Weissman]

This really is a big deal, you know.  People are starting to notice.  Is anyone
doing anything about it?  It's been almost a week since anything was mentioned
in this bug.

Comment 5

[Terry Weissman]

Every time I get another email message asking about this, I am going to whine
again in this bug.

The silence is getting oppressive.

I am also going to attach the missing mozilla/webtools/bonsai/cvslog.cgi file
here.  That way, I can tell people why it's missing and where to get a copy, all
at the same time.

Comment 6

[Terry Weissman]

Attachment: This is mozilla/webtools/bonsai/cvslog.cgi , which does not show up on the read-only CVS repository due to this bug.

Comment 7

[Terry Weissman]

I got more mail.

This is absolutely *PATHETIC*.

Fundamental mozilla.org infrastructure has been busted for a month now.  Nobody
is doing anything about it.

Why is it that nobody but me has even said anything in this bug for the last
three weeks?

Comment 8

[Dawn Endico]

i have attempted to get someone to deal with this severalmany times
over the last months, unsuccessfully.

The missing cvslog.cgi file is definatly the fault of --cvs-exclude. According
to the rsync man page, --cvs-exclude excludes (among other things) cvslog.*

I now have root access to this machine and have worked around the problem by
pulling that file alone with a separate rsync command. I did it this way
because I am reluctant to remove --cvs-exclude and replace it with equivalient
exclusions by hand since that would be fairly complicated and I don't want
to break the rest of the repository.


I pulled mozilla/webtools/bonsai from cvs-mirror and cvslog.cgi was pulled.
Marking fixed.

Comment 9

[Hixie (not reading bugmail)]

it's going to be fun when someone checks into that file and it doesn't get 
updated on the mirror again...

Comment 10

[Daniel (Leaf) Nunes]

There's a really braindead fix for this

use --exclude-from=foo instead of --cvs-exclude where foo looks like:

RCS
SCCS
CVS
CVS.adm
RCSLOG
tags
TAGS
.make.state
.nse_depinfo
*~
#*
.#*
,*
*.old
*.bak
*.BAK
*.orig
*.rej
.del-*
*.a
*.o
*.obj
*.so
*.Z
*.elc
*.ln


Which is the default cvs exclude list (minus core and cvslog.cgi). i'll get an
account on the machine and fix the overall mirroring behaviour.

Comment 11

[Tomi Leppikangas]

Maybe remove tags too?

cvs server: cannot open directory /cvsroot/mozilla/layout/html/tests/tags: No
such file or directory
cvs server: skipping directory mozilla/layout/html/tests/tags

Comment 12

[Daniel (Leaf) Nunes]

tags removed from the exlude list (but i left TAGS in, damnit)