400256 - Link / URL autodetection chokes in international symbols

Status: RESOLVED FIXED

(Bugzilla :: Creating/Changing Bugs, defect)

Description

[techtonik]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9a9pre) Gecko/2007101605 Minefield/3.0a9pre
Build Identifier: 

BugZilla can't properly detect the end of a link with internationalized characters, see -

http://ru.wikipedia.org/wiki/Линейная_алгебра

It is the same link as

http://ru.wikipedia.org/wiki/%D0%9B%D0%B8%D0%BD%D0%B5%D0%B9%D0%BD%D0%B0%D1%8F_%D0%B0%D0%BB%D0%B3%D0%B5%D0%B1%D1%80%D0%B0

Reproducible: Always

Steps to Reproduce:
1.
2.
3.

Comment 1

[Max Kanat-Alexander]

I can see it here.

The weird part is how it linkifies the first part correctly, but stops working after the underscore.

I suspect this will be fixed by bug 363153, though I'm not certain.

Comment 2

[techtonik]

I suspect it could be the case. I've seen this behaviour in some PHP applications like Drupal and the symptoms are the same - links are cut at a second underscore before internationalized characters.

Mailings are coming in different form from Bugzilla - i18n chars are not linked at all.

Comment 3

[Marc Schumann [:Wurblzap]]

I'm with Max in suspecting that bug 363153 will fix this (but I'm not certain either).

This is what we do for autolinkification:

    $text =~ s~\b(${protocol_re}:  # The protocol:
                  [^\s<>\"]+       # Any non-whitespace
                  [\w\/])          # so that we end in \w or /

Note the \w, which I think is the culprit. The regex picks up as much non-whitespace as it can, while still stopping at a "word" character (\w: alphanumeric plus "_"). Since we're still working in ASCII mode, the last "word" character we encounter before hitting whitespace is the underscore.

Comment 4

[techtonik]

In PHP I would add /u to regex. I do not know Perl, but perhaps "use utf-8" can help..

http://perldoc.perl.org/perlunicode.html#DESCRIPTION

Comment 5

[techtonik]

"use utf8" actually

Comment 6

[Marc Schumann [:Wurblzap]]

It seems bug 363153 has indeed fixed this. Try your comment at http://landfill.bugzilla.org/bugzilla-tip/page.cgi?id=linkify.html, and you'll see it correctly linkified.