5351 - URL linkifying code should cross linebreaks [libmime][mozTXTToHTMLConv]

Status: NEW

(MailNews Core :: MIME, enhancement)

Description

[Phil Peterson]

Rich, look at this msg in Nova to see how linebreaks fool the linkifying code.
Can we do anything about this?

     Subject: URL preservation on linewrap?
        Date: Wed, 21 Apr 1999 12:09:52 +0200
        From: Gavin Brelstaff <[e-mail]>
Organization: CRS4 - http://www.crs4.it/Areas/bma.html
  Newsgroups: netscape.public.mozilla.mail-news

Guys
  Pls confirm URLs  that get line wrapped in mozilla will still be
clickable.

Jon Udell at news:news.devx.com/joncom was overheard to say:

> Gnus works even in the case where linewrapping introduced
> leading whitespace? Here's the part of RFC1738 I meant:

Examples:

    Yes, Jim, I found it under <URL:ftp://info.cern.ch/pub/www/doc;
    type=d> but you can probably pick it up from <URL:ftp://ds.in
    ternic.net/rfc>.  Note the warning in <URL:http://ds.internic.
    net/instructions/overview.html#WARNING>.

> I guess it wouldn't be hard to piece these URLs together,
> come to
> think of it, but the NS and MS readers don't.
> Jon Udell | <http://udell.roninhouse.com/> | 603-355-8980

Comment 1

[rhp (gone)]

Not really sure how to attack this, but I will give it a shot.

- rhp

Comment 2

[rhp (gone)]

The biggest challenge with this issue is the fact that the code that does this
linkifying of URL's, etc.. is all line based and has no information for
previous or subsequent lines. I would have to try to impart some sort of state
information for these calls to make this work. I will think about it, but it
won't be done for M6.

- rhp

Comment 3

[Phil Peterson]

That's cool. I think this is pretty low priority, compared to your mountain of
composition backend work.

Comment 4

[sol]

I don't think this is a must have for PR1.

Comment 5

[rhp (gone)]

Unfortunately, this is non-trivial. Bumping way out.

- rhp

Comment 6

[rhp (gone)]

With the rewrite that Ben Bucksch is doing, we will be able to detect across
linebreaks. Reassigning.

- rhp

Comment 7

[Ben Bucksch (:BenB)]

I think, I was a bit too fast with accepting.
I wrote the new scanner with linebreaks in URLs in mind, so the scanner (which
is my part) shouldn't be the problem. Of course, I can't really test it, because
it is never fed with multiple line at once.

This is up to the caller, which are other parts of libmime. I'm discussing with
rhp, how to change/cheat libmime, but that's all I can do for that bug.

However, it doesn't look good.
BTW: Feeding the scanner with multiple line would make ASCII-art/ASCII-table
recognition possible, too.

Comment 8

[Warren Harris]

I added comments on this subject to bug 19251.

Comment 9

[[e-mail] (gone - use [e-mail] instead)]

mime=ducarroz

Comment 10

[Ben Bucksch (:BenB)]

*** Bug 95822 has been marked as a duplicate of this bug. ***

Comment 11

[Ben Bucksch (:BenB)]

*** Bug 123781 has been marked as a duplicate of this bug. ***

Comment 12

[Ralph Hartley]

*** Bug 137583 has been marked as a duplicate of this bug. ***

Comment 13

[Ben Bucksch (:BenB)]

*** Bug 88753 has been marked as a duplicate of this bug. ***

Comment 14

[Phil "clef" Sweeney]

Is this still on the radar for anyone?  It seems like a fairly important fix for
mail/news, considering most people use Outlook which wraps long URLs.  Nothing
has been said on this bug since January 2001.

There are more duplicates than on this bug, as duplicates of other duplicates
have been made.

I understand Entourage on Mac unwraps long URLs.

Comment 15

[Toylet]

well, it';s more an Outlook/OE problem than a mozilla issue. To be exact,this is
a feature request - some extra coding to cater for the mess left behind by M$
bugs. Should M$ fix it for their users first and then fix all the copies of
damaged messages out there in the universe? :)

Anyway, preserving long URL during quoting of message would be nice., though.

Comment 16

[Sascha Claus]

This is not an Outlook/OE problem. RfC 1738, Appendix (Page 21) says 

   In some cases, extra whitespace (spaces, linebreaks, tabs, etc.) may
   need to be added to break long URLs across lines.  The whitespace
   should be ignored when extracting the URL.

cf. the example at the description.

Comment 17

[Ben Bucksch (:BenB)]

Confirmed with successor RFC 2396.

Nobody is working on this. It would either need a rewrite of libmime or some
hack, e.g.
- buffer until we have an empty line.
- if we found an URL, do not output that line, but buffer it, then when
processing the next line, check, if there is something in the buffer, and if so,
process it together with the current line.

Both of it would have to be done independantly in the plaintext and f=f converters.

Comment 18

[Ben Bucksch (:BenB)]

s/buffer until we have an empty line./buffer until we have an empty line (i.e. a
whole paragraph) OR/

Comment 19

[Ron Thigpen]

see comments in bug #185032

any chance of getting this back on the front burner?

Comment 20

[bugzilla]

In a discussion over long URL's and the handling of these in news clients, I
came on following information in rfc2396 round page 33,34

  | In some cases, extra whitespace (spaces, linebreaks, tabs, etc.) may
  | need to be added to break long URI across lines. The whitespace
  | should be ignored when extracting the URI.

somewhat lower

  | Using <> angle brackets around each URI is especially recommended as
  | a delimiting style for URI that contain whitespace.

So I understand Mozilla isn't complying with the RFC.

When can a solution be expected?

Bernard

Comment 21

[Ben Bucksch (:BenB)]

>  | Using <> angle brackets around each URI is especially recommended as
>  | a delimiting style for URI that contain whitespace.

There is special code which specifically strips whitespace in this case. It's
just that we are not able to process URLs that cross several lines, for
technical reasons.

> When can a solution be expected?

Never, unless somebody fixes this bug. I am unlikely to do so. See above for my
thinking about how it could be fixed. None of the solutions are very tempting.

Comment 22

[Ron Thigpen]

Ben or Jean-Francois, have you considered any of the solutions proposed in Bug
185032 comment 0 <http://bugzilla.mozilla.org/show_bug.cgi?id=185032#c0>?  

Specifically: 

"... the behavior of the action triggered by selection of the "Copy
Link Location" action could be altered.  In the case that the current selected
text contained more than the extent of a recognized URL, this action could copy
the complete selected text but stripped of linefeed characters."

This would only require changes to the Copy Link Location code.

Comment 23

[Ben Bucksch (:BenB)]

copy link location is offtopic here. this bug is only about the automatic
linking of plaintext emails (not plaintext documents in the browser).

Comment 24

[Ron Thigpen]

i'm not referring to the browser.  copy link location is available in the
context menu w/i plaintext email.  if a URL has been recognized, that is.  the
failure of which is precisely on-topic here. 

the modified behavior of copy link location would serve as a workaround for the
issue presented by this bug.  

Comment 25

[Jo Hermans]

*** Bug 199850 has been marked as a duplicate of this bug. ***

Comment 26

[David G King]

Thought I'd mention that Pegasus eMail v4.02a has the ability for the user to
highlight the multi-line URI, right click, and select "Open as Hyperlink". Maybe
this is an option for this bug.

Note: the Pegasus implementation isn't perfect. It doesn't like URI's that are
indented for example.

Comment 27

[Ben Bucksch (:BenB)]

IMHO, we should not work around backend bugs like this with unfortunate UI, we
should fix the backend as suggested in comment 17. I just don't feel like doing
it myself.

Comment 28

[Ron Thigpen]

Haven't heard anything from assignee on this one.  Perhaps it should revert to
NEW w/ HelpWanted keyword?

Comment 29

[Hixie (not reading bugmail)]

.

Comment 30

[Hermann Schwab]

re: comment 26
highlight the multi-line URI, right click, and select "Open as Hyperlink" would
be fine.
I use a simple workaround: copy & paste the URL to the location bar.
To get this working you need an entry in the profile in user.js:
user_pref("editor.singleLine.pasteNewlines", 3); 
I´m using this on win98. 

I don´t understand why this pref or your suggested option isn´t default, there
are a lot of links in forums, which are not clickable, because of linebreaks, or
forum policy.

Comment 31

[David G King]

Well, my suggested option (with thanks to Pegasus eMail and the author David
Harris for implementing it and showing me how powerful it is) and the pref you
use user_pref("editor.singleLine.pasteNewlines", 3); are really workarounds to a
fundamental problem in the code.

A question for you (that is, for Hermann), what is the diff between the default
setting of the pasteNewlines of 1, and 3? For that matter, what does setting it
to 2 accomplish?

Comment 32

[Ben Bucksch (:BenB)]

Please discuss that elsewhere. Thanks.

Comment 33

[David G King]

I'm not sure why discussing a workaround to this bug, and any possible
side-effects it may have should be discussed anywhere but here. At least I'm
willing to work on the problem.

Comment 34

[Ben Bucksch (:BenB)]

Because it doesn't help to resolve this bug. This is not a user forum. Short
hints to workarounds are fine, discussion about it is distracting spam. Even
more so such meta-discussions.

Comment 35

[Joe Infla]

*** Bug 216525 has been marked as a duplicate of this bug. ***

Comment 36

[Brendan Chandler]

*** Bug 253284 has been marked as a duplicate of this bug. ***

Comment 37

[Mike Cowperthwaite]

*** Bug 299975 has been marked as a duplicate of this bug. ***

Comment 38

[Mike Cowperthwaite]

*** Bug 339961 has been marked as a duplicate of this bug. ***

Comment 39

[Ben Bucksch (:BenB)]

Bug 344288 fixes this for the format=flowed part.

Comment 41

[Miles Libbey]

(sorry for the last dupe -- searched for URL not link).  Wanted to ensure that w3c mention was recorded: 
The W3C convention 
http://www.w3.org/Addressing/URL/5.1_Wrappers.html
is to put <> around long links <http://mozilla.org>.  Thunderbird doesn't
recognize this convention, so, long links that start with < still wrap, making them un-clickable (and a royal pain for users that have to do multiple cut and
paste).

Comment 42

[Ben Bucksch (:BenB)]

> put <> around long links <http://mozilla.org>.
> Thunderbird doesn't recognize this convention

Actually, it does. There's even *special* code in mozTXTToHTMLConv for it, to cross linebreaks etc.. Read above what the code problem is.

Comment 44

[Hendrik Maryns]

Attachment: screenshot

Wow, bug number 5351 and still unresolved.

A screenshot showing that this is still a problem, maybe someone wakes up now?

Comment 47

[Thomas D. (:thomas8)]

Ben, that's great news that you assigned yourself to fixing this bug. I'm sure you can do it, because you're an experienced coder with deep insight on the matter from your own contributions. You'll make history when you address some of these long-standing issues. Looking forward to your ideas and patches on this bug. :)