199116 - CVS integration (patch)

Status: RESOLVED WONTFIX

(Bugzilla :: Bugzilla-General, enhancement)

Description

[David Swegen]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1) Gecko/20021130
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1) Gecko/20021130

The patches attached are used by us to achieve greater integration
between Bugzilla and CVS. It takes the form of three patches:

1. Enhancement to contrib/bugzilla_email_append.pl to allow commit
comments to be attached to multiple bugs and a script to act as glue
between bugzilla_email_append.pl and CVS. 

2. A patch to CVS to place quotes around files with spaces in the
filename in output from CVSROOT/loginfo.

3. A patch to globals.pl to autolinkify CVS commit logs so that they point
to a viewcvs installation, thus providing links to general file info,
old revision, new revision and a diff.

Patch 1 and 2 are in a reasonably complete state (though the code is grotty
- none of us here are perl coders). No doubt there are a number of
conditions that aren't handled properly (notably odd chars in filenames)

Patch 3 on the other hand requires that the URL to the viewcvs
installation be hardcoded into the globals.pl file. The obvious work
that this needs is that the URL to the installation be provided as a
parameter, which if left blank leaves the comment unchanged.

We also tried to avoid using an all-singing, all-dancing regexp in
GetCommitLinks, as our preference is to not use such things. No doubt
someone can come up with a single regexp to replace the entire function :)

For more information on setting up the commit-log append functionality
take a look at the file contrib/cvs_integration.txt which is in patch 1.


Reproducible: Always

Steps to Reproduce:
1.
2.
3.

Comment 1

[David Swegen]

Attachment: Patch 1 (CVS commit log -> bz comment)

Patch contains scripts/instructions on getting CVS commit logs into BZ
comments.

Comment 2

[David Swegen]

Attachment: Patch 2 (Extra tag for CVS loginfo)(optional)

If applied to CVS this patch enables an extra output option ({S}) for CVS that
places quotes around files. For use with patch 118405. Optional.

Comment 3

[David Swegen]

Attachment: Patch 3 (autolinkify CVS commit comments)

When used in conjunction with patch 118405 provides autolinkification of commit
messages to a viewcvs installation.

Comment 4

[Henri Sivonen (:hsivonen)]

Looks like a good enhancement. Confirming.

Comment 5

[Gervase Markham [:gerv]]

I think bbaetz would have something to say about the autolinkification; and it's
certainly true that it would need to be a lot more configurable.

This stuff may have to wait until the email interface is brought into the
mainline code. Goodness knows when that'll happen.

Gerv

Comment 6

[David Swegen]

Attachment: Patch 1 (CVS commit log -> bz comment)

Modified version of patch 1

Comment 7

[David Swegen]

Attachment: Patch 3 (autolinkify CVS commit comments)

Updated patch to cope with removed and added files.

Comment 8

[David Swegen]

Attachment: Patch 3 (autolinkify CVS commit comments)

Patch updated to allow configuration of ViewCVS URL by setting a parameter,
instead of it being hardcoded into globals.pl

Comment 9

[David Swegen]

Attachment: Patch 4: Autolinkify commit logs in ViewCVS

I've added this patch for completeness sake. It simply autolinkifies bugs
mentioned in commit logs in ViewCVS to point back to the bugs in bugzilla.

The patch is a tad rough, so if you try and use it you'll have to manually
change the URL to point to your ViewCVS installation (it's the 'url' variable
that needs to be changed).

Comment 10

[David Swegen]

Quick note: the autolinkification stuff copes poorly with spaces in filenames,
especially directorynames. I'm looking at it, but it's a really tricky one to fix.

Comment 11

[Phred Eisele]

I would like to generalize this approach to permit extension of
autolinkification by the bugzilla administrator.  For example, we use aegis
(similar to cvs) and twiki (for documentation).  I would like references to
items controlled by them to autolinkify when entered as a comment/description as
well as cvs.  
e.g. Patch 5 illustrates how the autolinks could be built from uri's.

uri:twiki:Main/JohnDoe
uri:aegis:bugzilla.1.17.4/19916
uri:cvs:bugzilla-1.1.17/19916

This approach is in keeping with the idea of the uri, inasmuch as a uri is not
intended to convey the location of the object, but only uniquely identify it.

Comment 12

[Phred Eisele]

Attachment: Autolinkification for URI identifiers

This patch was written to work with version 2.16.2

Comment 13

[David Swegen]

It's a reasonable idea, but my feeling is that it negates part of the point of
autolinkfication, which is that users don't have to learn any special
codes/strings in order for it to work (which is why people go "Wow!" when they
see it happen).

Wrt this bug, our aim was that if the admin chooses not to enable
autolinkification the plain text version will be perfectly clear (OK, OK, so the
autolink stuff was an afterthought, but it works quite nicely :)

IMHO this should go in a separate bug, as it addresses a related, but different,
issue.

Comment 14

[Harri Vartiainen]

> IMHO this should go in a separate bug, as it addresses a related, but different,
> issue.

 Autolinkfication is discussed in bug 102685.

Comment 15

[Christian Reis]

*** Bug 129765 has been marked as a duplicate of this bug. ***

Comment 16

[David Swegen]

After discussing this feature with kiko the following changes will be made:

* The functionality of the script that puts the logs into bz will be merged into
the script called by cvs loginfo. That way it simplifies the setup quite
drastically. The (potential) downside is that if CVS and bz live on different
systems the admin will have to allow a remote connection from the CVS system.
* Eventually a new user attribute would have to be added, which would allow the
user to specify their cvs login name in case it doesn't match their bz login
(which is a requirement atm). This would default to the bz login name.

Comment 17

[David Swegen]

Note to self for next version: The CVS patch will not be required in the next
version as all that is required is that in the script line in loginfo has the
oldversion and newversion being placed on either side of the filename (ie {sVv}
becomes {Vsv}).

The only things it won't cope with (I think) is whitespace at the end of
directory names, or forward slashes anywhere.

Comment 18

[David Swegen]

Attachment: Screenshot of autolinkified CVS commit comment

To save myself a lot of future typing I'm going to let the image do the
talking. And no, the "feature" mentioned in the image is not in the patches
here, it's simply a local hack.

Comment 19

[David Swegen]

Brief update, and also an RFC: The new cvs->bz script is almost done, but with a
few changes from what was planned: 

Change #1) The script is now written in python. I've done this because a) I
prefer it, and b) due to change #2 there are now no extra dependencies.

Change #2) The comments will be added via http, using process_bug.cgi. The
downside of this is that information regarding the person doing the commit will
be limited to their cvs login (and possibly any info that can be sucked out of
the GECOS field - anyone know how to do this on windows?). The comments will be
owned by a generic bz user for that purpose.

There are a few benefits: 
* Far simpler
* no need to maintain a mapping between cvs logins and bz logins
* bz deals with the db (and any changes to it that might occur)

All that remains for me to do is to code the request and result stuff.

As a sidenote I believe Gerv wants to make adding comments easier - see bug 232241

Anyone have any problems with this?

Comment 20

[Christian Reis]

Dave: be sure to check out the bugzilla-submit script ESR and I wrote -- it has
authentication and parsing code which is probably useful. I guess we could
eventually merge parts of the code into a Bugzilla.py client library.

I can't find a way to simplify the username mapping; the sad part is that having
direct access to the database would allow us to post as any user. That said, I
don't think it's all bad (it may even be a feature, since it simplifies
filtering) to have a cvs-commit bugzilla user.

Reassigning to you in the hope that you'll post a patch for reviews. Remember,
this should go into contrib/ -- perhaps contrib/cvsloginfo/ ?

Comment 21

[David Swegen]

Need to get the commenting interface done before I can polish this off.

Comment 22

[David Swegen]

Some more info about how the cvs integration patches and how related tools are
being used, plus setup instructions can be found at
http://www.einval.com/~steve/software/cvs-bugzilla/

Comment 23

[David Swegen]

I haven't tried this, but I suspect that in order for the bugzilla_email_append
script to work with 2.17.7 the patch from bug 237638 will have to be applied.

Comment 24

[David Swegen]

Attachment: Patch 3 (2.16.5 version) (autolinkify CVS commit comments)

This patch does the same as patch 127357, but for bz 2.16.5. Note that there is
no viewcvs entry in the params page, but instead the $viewcvsurl variable in
globals.pl must be set to (oddly enough) the URL of the ViewCVS installation.

Also note that autolink within the cvs comment (things like URLs, emails etc)
will not work.

Thanks to Xavier Chatelain for testing the patch.

Comment 25

[Kristis Makris]

Just a quick note that the Scmbug project attempts to address the general
problem of integrating source code version control management systems with bug
tracking systems.

More information can be found here:

http://www.public.asu.edu/~makrists/scmbug/

Comment 26

[Tyler]

Attachment: 2.18 fix for multiple bug patch for bugzilla_email_append

2.18rc2 pulls in a fix to bugzilla_email_append.pl to have it not use
processmail any more. This patch is the same as "Patch 1 (CVS commit log -> bz
comment)" but fixed for 2.18rc2 and without the extra file from the end of that
patch (which doesn't belong as part of that fix anyway).

Comment 27

[Tyler]

Attachment: Add bug 183434 fixes to multiple bug append patch

If anyone is using the multiple bug append patch here and wants to pick up the
fixes from bug 183434, this ought to do it.

Comment 28

[Carmen]

I am using the guide who appears in
http://www.einval.com/~steve/software/cvs-bugzilla/#intro with the purpose of
doing that the installation of bugzilla the 2,18 on Fedora Cores 1 communicates
with repositorio who I have, but the truth that enough I am confused, in
addition that the handling that I have of CVS very is advanced.  Good somebody
creates can collaborate to me of some form.

Comment 29

[Michiel van Everdingen]

Attachment: CVS integration without using email

I'm wondering if the dependency on bug 232241 is really needed.
Comments can be directly inserted in the Bugzilla database by using mysql -
also in case Bugzilla and CVS are on different hosts.

The attachment shows how this can be done for CVSNT. It shows the contents of
the following files for CVSNT: "loginfo", "loginfo.bat", "loginfo.sed" and
"getMRs.sed".

Furthermore, I've slightly changed the routine "GetCommitLinks" as introduced
in comment 8.

Note that I'm using "MR" (Modification Request) instead of "bug".

Comment 30

[Kristis Makris]

(In reply to comment #29)
> Furthermore, I've slightly changed the routine "GetCommitLinks" as introduced
> in comment 8.

The implementation of GetCommitLinks contains the comment:

# The routine GetCommitLinks autolinkifies a set of files committed in CVS.
# It assumes one committed file per line in the following format:
#   <root>\t<file>\t<ver1>\t<ver2>

...which specifies the parseable format.

Perhaps bug 314097 should be reviewed (and should be a dependency for this one) before applying the autolinkification portion of this integration. Inevitably, a convention for the format has to be defined, and there seems to be a more flexible proposal about the format in:

http://bugzilla.mkgnu.net/show_bug.cgi?id=266#c5

The main differences are the additional need to specify the project name (e.g. when linking with WebSVN instead of just ViewCVS) and perhaps tokens that signify a new file, modified file, a directory, etc.

Comment 31

[Max Kanat-Alexander]

This is WONTFIX--SCM integration should be a plugin, not part of Bugzilla.