102685 - Link twiki names to a twiki installation in comments

Status: RESOLVED WONTFIX

(Bugzilla :: User Interface, enhancement, P5)

Description

[Ludovic Dubost]

DESCRIPTION:
 
This patch allows to support twiki names in the bugzilla installation.. This is
quite usefull of you want to link bugs to twiki pages without including full
URLs in your bugs...
 
DETAILS:
 
twikiname.pl:
 
twikinames.pl is the perl code that does the actual substitution. The code was
extracted from the Twiki.org code base.
 
twiki.patch:
 
actual patch to defparam.pl and globals.pl.. The substitutions happens in
globals.pl.. defparam.pl allows to configure the twiki urls...
 
twiki.patch.alternative:
 
alternative version of the patch that can install on different version of
bugzilla as the one the patch was made for.. This patch does not support reverse
applying... but does the same thing.
 
The twiki installation base URL and default twiki webs are user configurable.
 
REQUIREMENTS:
 
- the patch has been based on the bugzilla post 2.14 codebase tagged with
bugzilla_mozilla_org_2001-Aug-31
 
AUTHOR(S):
 
This patch has been developed by Ludovic Dubost <[e-mail]> and Erwan
Arzur <[e-mail]> for the
bugzilla and twiki installations at Netvalue. If you want a patch for twiki that
allows to support automatic linking to bugzilla bugs in twiki please contact
Ludovic or Erwan.

Comment 1

[Ludovic Dubost]

Attachment: Perl Script interpreting twiki names

Comment 2

[Ludovic Dubost]

Attachment: Patch to intercept twiki names in comments

Comment 3

[Ludovic Dubost]

Attachment: Alternative Patch to intercept twiki names in comments (might install more easily)

Comment 4

[Gervase Markham [:gerv]]

I'd say this is too specialised; CCing Dave for his view.

Gerv

Comment 5

[Dave Miller [:justdave]]

We are currently trying to wrap up Bugzilla 2.16.  We are now close enough to
release time that anything that wasn't already ranked at P1 isn't going to make
the cut.  Thus this is being retargetted at 2.18.  If you strongly disagree with
this retargetting, please comment, however, be aware that we only have about 2
weeks left to review and test anything at this point, and we intend to devote
this time to the remaining bugs that were designated as release blockers.

Comment 6

[Dave Miller [:justdave]]

Reading Gerv's comment from earlier where he's asking for my view on it, I have
no clue what Twiki is, so I don't feel I can offer an informed opinion on this.

Comment 7

[Martin Ellison]

See http://www.twiki.org/ for TWiki(TM) - "A Web Based Collaboration 
Platform". "TWiki looks and feels like a normal Intranet or Internet web site. 
However it also has a Edit link at the bottom of every topic (web page), 
everybody can change a topic or add content by just using a browser." TWiki is 
GPLed and on SourceForge. There are other Wikis around (see http://wiki.org/) 
but TWiki seems to be the best known. In other words, it's a knowledge 
management application.

TWiki pages are referenced by TwikiNames which 
LookLikeLotsOfWordsSquashedTogether. TWiki automatically converts the 
TwikiNames into hyperlinks. The object of the patch is presumably to convert 
TwikiNames in Bugzilla comments into the same hyperlinks.

 

Comment 8

[Jouni Heikniemi]

Well, I guess we still need Dave's comment on this - maybe he's more informed
now, with the URL given and all :-)

For the record, I think we should take this patch. Wikis have more potential
than has been realized so far. A Wiki is a good way of publishing/maintaining
information about small-midsize projects (especially in intranet environments),
and having Bugzilla support integration with such a powerful documentation tool
is simply good. Besides, the patch has rather low impact on existing code.

Comment 9

[Lars Aronsson]

I agree with Jouni that the combination of Wiki and Bugzilla has a huge
potential. Many new [free software] projects (such as those on [SourceForge])
already have a [CVS] repository, a [Bugzilla], and a [Wiki]. These systems go
well together.

Both Bugzilla and Wiki are based on the user entering simple text, which the
[CGI script] converts into [HTML] with links in some more or less magic way
(e.g. the word "attachment" followed by a number in Bugzilla). Bugzilla is a bug
database with bug id as the primary entry, and Wiki is a dictionary with keyword
as the primary entry. If wiki links were recognized by Bugzilla, as the original
poster suggests, the bracketed terms above would be links to a wiki webpage that
defines that term.

Wiki links (either using CamelCase or [bracket links] - I would recommend the
latter) should be recognized in Bugzilla descriptions, and links to bugs (e.g.
the word "bug" followed by a number) should be recognized in Wiki pages. The
same text-to-HTML processing should be applied in both cases.

Comment 10

[Dave Miller [:justdave]]

The User Interface component now belongs to Gerv.  Reassigning all UNCONFIRMED
and NEW (but not ASSIGNED) bugs currently owned by Myk (the previous component
owner) to Gerv.

Comment 11

[Dave Miller [:justdave]]

Reassigning back to Myk.  That stuff about Gerv taking over the User Interface
component turned out to be short-lived.  Please pardon our confusion, and I'm
very sorry about the spam.

Comment 12

[Andreas Höfler]

Does this still work with BZ 2.17.4?

Comment 13

[Ludovic Dubost]

Attachment: Perl Script interpreting twiki Names with Multiple Twikis

New version of script working with the patch for version 2.17.4
this version now allows multiples twikis using the syntax of the 
Interwiki plugin.. See
http://twiki.org/cgi-bin/view/Plugins/InterwikiPlugin

Comment 14

[Ludovic Dubost]

Attachment: New patch for version 2.17.4

New version of the patch for version 2.17.4
this version now allows multiples twikis using the syntax of the 
Interwiki plugin.. See
http://twiki.org/cgi-bin/view/Plugins/InterwikiPlugin
Adding configuration for Interwikis needs manual editing of defparam.pl and
reruning of checksetup.. So there might be a better way to handle it..

Comment 15

[Christian Reis]

I would much rather see this code integrated as a separate function (like we
have getBugLink, etc). Do you think that would be possible, Ludovic?

Does this support non-twiki wikis?

Comment 16

[Ludovic Dubost]

Attachment: Even newer for any version

This script should work with all version as all the code is moved into
Bugzilla/Wiki.pm.. The next attachement contains the patch which is minimal.

Many improvements in this version.. The settings are now all in 'twikiurl' in
the form:

twikiurl ->
TWiki:http://www.twiki.org/bin/view/Main/?topic=,Wiki:http://www.wikipedia.org/wiki/,Bug:http://bugzilla.mozilla.org/show_bug.cgi?id=,MozBug:http://bugzilla.mozilla.org/show_bug.cgi?id=


These settings would links automatically any text like:

TWiki:Plugin.InterwikiPlugin
Wiki:Astrology
MozBug:102685

Since TWiki is the first item in the configuration, Plugin.InterwikiPlugin or
InterwikiPlugin would also link automatically...

It is interesting to note that it is not limited to twiki, but also works with
Wikis and also other applications..

The general rule would be:

Config: Name:Url will transform Name:Param -> UrlParam

Comment 17

[Ludovic Dubost]

Attachment: Newer patch for version 2.17+

Simplified patch.. All the code is now in Wiki.pm..
However because of changes in defparam.pl the patch probably only works on
recent versions..

Comment 18

[Andreas Höfler]

Does the new patch still cripple special chars in comments?
I stumbled over this when using double quotes in comments and have filed Bug
219549 for this prior finding out, that this patch is the root of the cripplement.

Comment 19

[Dave Miller [:justdave]]

Comment on attachment 131631 [details]
Even newer for any version

Bugzilla::Wiki::GetWikiLinks() does this:

     $text = &::value_quote($text);

But it's called from inside QuoteUrls(), which does this:

    $text = html_quote($text);

So it's getting escaped twice.

Comment 20

[Ludovic Dubost]

This is probably easy to fix.. can you give an example of a character that would
be crippled ?

Ludovic

Comment 21

[Ludovic Dubost]

Attachment: Wiki.pm file with fix for special characters

New Wiki.pm with fix for special chars.. I don't really know why there was the
call to value_quote.. Probably a copy paste from somewhere..

I'm removed it and my tests for Wiki Names still seem to work fine..

Comment 22

[Andreas Höfler]

from cron:

./collectstats.pl:
Reference found where even-sized list expected at Bugzilla/Wiki.pm line 47.
Reference found where even-sized list expected at Bugzilla/Wiki.pm line 47.

./whineatnews.pl
Reference found where even-sized list expected at Bugzilla/Wiki.pm line 47.

Comment 23

[Christian Reis]

Can you perhaps check on those errors, Ludovic? Thanks!

Comment 24

[Ludovic Dubost]

I'll check them out..

Comment 25

[Ludovic Dubost]

Attachment: Patch Against 2.17 Head

Newer version of the patch fixing the problem reported in comment 22.. Also the
default value for twiki.org is now correct..

Comment 26

[Ludovic Dubost]

Comment on attachment 132794 [details] [diff] [review]
Patch Against 2.17 Head

Asking for review of this patch..

Comment 27

[Bradley Baetz (:bbaetz)]

Why is this being done as a separate file rather than done as a 5 liner in
globals.pl? That way you wouldn't ahve to do the checks you do to look for
already-expanded links. All of that also makes the regexps really hard to read;
can you give an example for each, and explain why they need to be done separately.

Given where the callout is, its likely that you're still double escaping,
although I can't come up with a simple test case in a very quick glance.

Comment 28

[Ludovic Dubost]

I've separate this in the Wiki.pm file in order to minimize the new lines in
globals.pl in order to have the functionnality of wiki links as much separated
as possible..

Here are the examples for each regex:

----- The Most important one ------
+     # Handle any link of the form XXXX:YYYYY 
+     while ($text =~ s/(\s*)(\w*)\:([\w\.\#\%]*)/" ##$count##"/eo) {
+         my $item = internalLink($1,$2,"",$3,$3,"","$1$2:$4");
+         $things[$count++] = $item;
+     }

Examples: 
TWiki:WebHome -> 
<a href="http://www.twiki.org/cgi-bin/view/Main/?topic=WebHome">WebHome</a>
TWiki:Codev.WebHome -> 
<a
href="http://www.twiki.org/cgi-bin/view/Main/?topic=Codev.WebHome">Codev.WebHome</a>

--- TWiki syntax for using brackets
+     # Handle specific links with []
+     while ($text =~ s/\[\[(.*?)\]\[(.*?)\]\]/" ##$count##"/eo) {
+         my $item = specificLink("","","",$theTopic,$2,$1);
+         $things[$count++] = $item;
+     }

Example:
[wiki home][WebHome] ->
<a href="http://www.twiki.org/cgi-bin/view/Main/?topic=WebHome">wiki home</a>

This one only works for the first wiki configured in the parameters

--- Another TWiki syntax for using brackets
+     # Handle specific links with []
+     while ($text =~ s/\[\[(.*?)\]\]/" ##$count##"/eo) {
+         my $item = specificLink("","","",$theTopic,$1,$1);
+         $things[$count++] = $item;
+     }
+
Example:
[[web home]] ->
<a href="http://www.twiki.org/cgi-bin/view/Main/?topic=WebHome">web home</a>

This one only works for the first wiki configured in the parameters

--- TWiki syntax with anchors
+     # Handle links of the form Main.WebHome#anchor
+     while ($text =~
s/([\*\s][\(\-\*\s]*)([A-Z]+[a-z]*)\.([A-Z]+[a-z]+[A-Z]+[a-zA-Z0-9]*)\%(\#[a-zA-Z0-9_]*)/"
##$count##"/eo) {
+         my $item = internalLink($1,"",$2,$3,"$3$4",$4);
+         $things[$count++] = $item;
+     }
+
Example:
Codev.WebHome#anchor1 ->
<a
href="http://www.twiki.org/cgi-bin/view/Main/?topic=Codev.WebHome#anchors1">WebHome</a>

This one only works for the first wiki configured in the parameters


+     # Handle links of the form Main.WebHome
+     while ($text =~
s/([\*\s][\(\-\*\s]*)([A-Z]+[a-z]*)\.([A-Z]+[a-z]+[A-Z]+[a-zA-Z0-9]*)/"
##$count##"/eo) {
+         my $item = internalLink($1,"",$2,$3,$3,"");
+         $things[$count++] = $item;
+     }
+
Example:
Codev.WebHome ->
<a href="http://www.twiki.org/cgi-bin/view/Main/?topic=Codev.WebHome">WebHome</a>

This one only works for the first wiki configured in the parameters

+     # Handle links of the form WebHome#anchor
+     while ($text =~
s/([\*\s][\(\-\*\s]*)([A-Z]+[a-z]+[A-Z]+[a-zA-Z0-9]*)\%(\#[a-zA-Z0-9_]*)/"
##$count##"/eo) {
+         my $item = internalLink($1,"","",$2,"$2$3",$3);
+         $things[$count++] = $item;
+     }
+
Example:
WebHome#anchor1 ->
<a href="http://www.twiki.org/cgi-bin/view/Main/?topic=WebHome#anchors1">WebHome</a>

This one only works for the first wiki configured in the parameters



+     # Handle links of the form WebHome
+     while ($text =~ s/([\*\s][\(\-\*\s]*)([A-Z]+[a-z]+[A-Z]+[a-zA-Z0-9]*)/"
##$count##"/eo) {
+         my $item = internalLink($1,"","",$2,$2,"");
+         $things[$count++] = $item;
+     }

Example:
WebHome ->
<a href="http://www.twiki.org/cgi-bin/view/Main/?topic=WebHome">WebHome</a>

This one only works for the first wiki configured in the parameters.
So:

The first regex handles the links in the form TWiki:Codev.WebHome or Wiki:1969
The next 5 regex only handle the wiki names for the first wiki configured and
handles wiki names in the text like WebHome or Codev.WebHome

For intranet the default behavior or Wiki names is very interesting when you are
working with both a twiki and a bugzilla in the intranet.. It allows to link
from one to the other easily..

Comment 29

[Bradley Baetz (:bbaetz)]

Hmm. Do we really want twiki-style linkification? I can see that breaking lots
of stuff, such as code snippits.

I'll try to look at the twiki changes in more detail this weekend.

Comment 30

[Ludovic Dubost]

Note that there is a configuration option to turn it on/off.. Maybe there could
be an additional configuration option for the 5 regex twiki links without the
'TWiki:' in front of it.

Comment 31

[Christian Reis]

I personally like the linkification *very much*, though I do agree it's not for
general consumption; a configuration option for the non-prefixed forms sounds
like an acceptable compromise (in this specific case).

Reassigning to patch author.

Comment 32

[Christian Reis]

Would it make sense to have a "default wiki" which is the one we link to by
default (in the non-prefixed form)?

On the form:

  [wiki home][WebHome]

I know some other wikis (phpwiki, for instance) provide a

 [wiki home|WebHome]

syntax, is this non-standard?

Comment 33

[Gervase Markham [:gerv]]

(I'm trying not to let my extreme hatred of Wikis influence my thinking here :-)

Has this been evaluated for performance impact? The autolinkification code was
rewritten very carefully by bbaetz to be as fast as possible. 

IMO, the obvious param structure is to have a param for the URL of your Wiki; if
it's filled in, autolinkification happens. If it's not, it doesn't.

Gerv

Comment 34

[Bradley Baetz (:bbaetz)]

Wikis are fun...

Currently perf may be an issue, but correctness is more so.

the besdt way is to just add a bit after where the currently http/ftp/etc stuff is.

I'm not sure how the conf file works, though, esp with multiple wikis - how does
it pick one?

Comment 35

[Christian Reis]

I *think* the current patch does something like (haven't actually had a second
to look at the config part):

  - Enable Wiki Support
  - Wiki Servers: |                      |
                  |                      |


Enabling Wiki support turns on linkifying of explicit wiki links to servers
listed in the textarea, and the first server in the list is (in this patch) the
default server -- this is what the unprefixed WikiWords links to.

We could use  an extra 

   - Default Wiki Server: |                   |
                           Specifying a default server will activate
                           linkifying for all WikiWords, not just prefixed ones

option that turns on the more intrusive behaviour. Leaving it blank would mean
you don't want unprefixed WikiWords to be linkified. 

We could also keep the "first-is-default" behaviour and provide a 
  
  - [/] usedefaultwikiserver Use first Wiki server as a default for unprefixed
WikiWords

I prefer the second option, but only slightly. It's hard to do a decent admin
interface on a webpage without using JS, unfortunately :-/

Comment 36

[Ludovic Dubost]

This is exactly what this patch is doing.. Except the the config is not a text
area but only one line.. Quite hard to read by the way..

Currently the first server is the default one.. I will add the option to turn
on/off the default server..

I think this would reduce also the overload as there would be only one regular
expression when the default server is turned of except off 6 when turned on.

Comment 37

[Gervase Markham [:gerv]]

It's my view that, in general, the number of options in editparams.cgi for a
feature should not be out of proportion to the use of that feature. (This is why
I dislike LDAP; it's got lots of params which clutter up the list, and not many
people use it.)

IMO, we need to think very carefully about how we can avoid this feature
requiring lots of parameters. A boolean here, a text string there, and there's
no end to it... ;-)

Gerv

Comment 38

[Bradley Baetz (:bbaetz)]

We really need to rewrite editparams, but thats a separate issue.

As I said, I'll look at it this weekend. Won't have time before then.

Comment 39

[Christian Reis]

While in principle I agree with Gerv, I don't see how we can make this use a
single configuration option. I invite attempts, of course.

Bradley is long-term-correct, at any rate: editparams needs a UI whack. But
looking at the codebase, what doesn't? I'll write up something interesting on
reviewers later today on this topic.

Comment 40

[Ludovic Dubost]

If you think about rewriting editparams at the same time maybe some improvement
to defparams would be a way to reduce the conflict between patches.. If you add
some params to defparams at the end of the file two different patches will
necessarly conflict... Maybe we should find a way to not have this conflict.. It
could be separate defparams files per feature for example ? Maybe rewamping the
params things is a separate bug..

Comment 41

[Christian Reis]

I'm sorry, I may have suggested that editparams needs to be hacked in the wrong
forum -- that is *definitely* a separate bug, and *this* bug isn't blocked by it
in any way.

Comment 42

[Christian Reis]

Comment on attachment 132794 [details] [diff] [review]
Patch Against 2.17 Head

>+++ web.wiki/defparams.pl	2003-10-07 14:23:56.813945000 +0200
>+   name    => 'twikiurl',
>+   desc    => 'Base url of the twiki installation to link to.',
>+   type    => 't',
>+   default => 'TWiki:http://www.twiki.org/cgi-bin/view/Main/?topic=,Wiki:http://www.wikipedia.org/wiki/'
>+  },

How hard is it to use a textarea for this part? 

We should rename this to twikiurls, since it's (potentially) a list.

We're missing an extra boolean option here, as I suggested in comment 35.

>+++ web.wiki/globals.pl	2003-10-07 14:09:09.193948000 +0200
>@@ -916,6 +917,8 @@
>               ~GetAttachmentLink($2, $1)
>               ~egmx;
> 
>+    $text = Bugzilla::Wiki::getWikiLinks($text);
>+

Is there a reason not to do the Wiki linkification last here (just before
removing the encoding hacks)?

>+++ web.wiki/Bugzilla/Wiki.pm	2003-10-07 14:24:31.113940000 +0200
>@@ -0,0 +1,250 @@
>+# The Original Code is the Bugzilla Bug Tracking System.
>+#
>+# The Initial Developer of the Original Code is Netscape Communications
>+# Corporation. Portions created by Netscape are
>+# Copyright (C) 1998 Netscape Communications Corporation. All
>+# Rights Reserved.

This file has nothing to do with Netscape -- it's your work, so I suggest you
assign copyright to yourself (I've emailed [e-mail] to clarify).

>+# Contributor(s): Ludovic Dubost <[e-mail]>
>+#                 Code integrated from twiki.org under GPL 

Whoops. Is there actual twiki.org code in here? If so, and if it's GPLed, we
can't integrate it with Bugzilla without explicit permission from the original
copyright owners.

The other option is rewriting this code, which might not be very enjoyable
work, but saves us trouble in the future.

>+sub ReadWikiUrls
>+{
>+ foreach my $param (split(',', &::Param('twikiurl'))) {
>+   $_ = $param;
>+   my ($wiki, $wikiurl) = /^(.+?):(.+?)$/;
>+   $WikiUrls{&::trim($wiki)} = &::trim($wikiurl); 
>+   $defaultWiki = &::trim($wiki) unless $defaultWiki;

This is where the flag I suggested would be used -- we might not set a default
wiki, in which case we'd avoid any unexpected (IOW, unprefixed) linkification.

I skipped reviewing the internal functions this time around.

>+sub getWikiLinks {
>+     my ($text) = @_; 
>+     my @things;
>+     my $count = 0;
>+
>+     return $text unless &::Param('usetwiki');
>+     ReadWikiUrls();
>+     my $theTopic = "WebHome";
>+
>+      
>+     # quoting of "#" characters.
>+     $text =~ s/#/%#/g;
>+     $text = " " . $text;

Hmm, what's this last line for?

>+     # Handle any link of the form XXXX:YYYYY 
>+     while ($text =~ s/(\s*)(\w*)\:([\w\.\#\%]*)/" ##$count##"/eo) {
>+         my $item = internalLink($1,$2,"",$3,$3,"","$1$2:$4");
>+         $things[$count++] = $item;
>+     }

You can probably return early here if there is no default Wiki URL.

The rest of the code looks good at a first glance; I'll be testing the next
patches locally, too.

Comment 43

[Dave Miller [:justdave]]

Bugzilla 2.20 feature set is now frozen as of 15 Sept 2004.  Anything flagged
enhancement that hasn't already landed is being pushed out.  If this bug is
otherwise ready to land, we'll handle it on a case-by-case basis, please set the
blocking2.20 flag to '?' if you think it qualifies.

Comment 44

[Max Kanat-Alexander]

This is plugin material--not really something we'd take upstream.

Comment 46

[Frédéric Buclin]

(In reply to Max Kanat-Alexander from comment #44)
> This is plugin material--not really something we'd take upstream.

I agree! We have all the required hooks to implement this as an extension.