97980 - linefeeds from konqueror ignored

Status: RESOLVED FIXED

(Bugzilla :: Creating/Changing Bugs, defect, P2)

Description

[Chris Abbey]

for some odd reason, since b.m.o upgraded to 2.15 all
bug reports I've submitted

using Konqueror show up
without any of the line feeds I put into them. 97979 shows

this for example, as should this bug. I've

filled it with a number of line feeds, which
history shows won't show up.

Comment 1

[Chris Abbey]

hrm. they came through correctly... very odd,the only difference seems to be that this timearound I didn't have to login again... could theline feeds get lost durng that sequence??

Comment 2

[Chris Abbey]

bingo. If you don't have cookies and have to login then the bug text looses it's 
linefeeds.

Comment 3

[Chris Abbey]

and does the

choice of

browser matter?

This is netscape 4.77

Comment 4

[Chris Abbey]

yes. I had logged out of bugzilla when submiting that, looks like this maybe
only knoqueror. :(

Comment 5

[Myk Melez [:myk] [@mykmelez]]

Does

Mozilla

suffer

from 

this

problem?

Comment 6

[Myk Melez [:myk] [@mykmelez]]

Nope.  Resolving invalid.

Comment 7

[Chris Abbey]

FOUL! This is the *bugzilla* module, not the mozilla module. It's supposed to be
browser agnostic.

Comment 8

[Matthew Tuck [:CodeMachine]]

I think the point was this is a Konqueror bug.

Comment 9

[Myk Melez [:myk] [@mykmelez]]

The problem here is that when Konqueror submits a form, it replaces occurrences
of the character entity "
" with spaces in form field values.

Interestingly, Konqueror's behavior appears to be congruent with the W3C's HTML
4.01 specification, because the value of an INPUT tag is CDATA, and user agents
are expected to replace character entities with their equivalent characters and
carriage returns with spaces within CDATA:

http://www.w3.org/TR/html4/types.html#type-cdata

If this behavior is indeed part of the standard, it is quite impractical, which
is probably why Mozilla and Netscape 4.x deviate from the standard. 
Nevertheless, we should figure out if there is a standards-based way to do this,
and the long-term solution is to fix bug 11901.

Comment 10

[Myk Melez [:myk] [@mykmelez]]

From conversation in #mozwebtools:

<myksc> Hixie_: i found what i think is a mistake in the HTML 4.01 specification
<myksc> and i wanted to bounce it off you and see if you agree
<myksc> the value of the "value" attribute of the "input" element is CDATA
<Hixie_> yes
<myksc> user agents are expected to transform CDATA in three ways (except inside
SCRIPT and STYLE elements):
<myksc> 1. replace character entities with characters
<myksc> 2. ignore line feeds
<myksc> 3. replace each carriage return or tab with a single space
<Hixie_> careful
<Hixie_> CDATA handling in attributes and elements is different
<Hixie_> but yes, it's something like that
<myksc> (yes, i am quoting from the HTML 4.01 spec on how user agents should
interpret attribute values)
<Hixie_> this is why bugzilla does all that mess with the description when there
is a midair collision
<myksc> the problem is that in an INPUT element of TYPE="hidden", i can't think
of any situation in which the server would want the browser to do steps two and
three
<Hixie_> so the server should escape it
<Hixie_> like we do
<myksc> right, but that isn't what the spec says
<Hixie_> how do you mean?
<myksc> well, the spec says the user agent should replace carriage returns in
attribute values with spaces
<Hixie_> right
<-- techbot1 has quit (Ping timeout: 180 seconds)
<Hixie_> so if servers don't want that, they'd better escape it
<myksc> mozilla and ns4.x don't do that (for the VALUE attribute of an INPUT
TYPE="hidden" element at least)
<Hixie_> that's a bug, filed and being worked on
<myksc> yes, but what i am saying is that it shouldn't be a bug.  the spec
should be changed.
<myksc> f.e.
<Hixie_> the spec won't be changed :-)
<-- timeless has quit (Ping timeout: 180 seconds)
<myksc> if you submit a comment to bugzilla, and you aren't logged in, bugzilla
displays a login screen with a hidden field containing your comment
<Hixie_> SGML doesn't have any other way of marking up free text attributes
<Hixie_> yes, and we are careful to escape returns and stuff
<myksc> that hidden field contains entities representing the carriage returns
(&#031;)
<Hixie_> terry and I sorted that out a long time ago
<Hixie_> (well, terry did. I pointed.)
<myksc> so, if mozilla first converts the character entities into carriage
returns, and then converts the carriage returns into spaces, doesn't that get
rid of the carriage returns?
<myksc> or should it not do double-conversions?
<Hixie_> you can argue that the carriage-return-to-space conversion comes before
the entity expansion
<Hixie_> and that is what i argue
<myksc> ok, but then the spec is ambiguous
<myksc> since it doesn't specify that.
<Hixie_> welcome to the HTML spec
<Hixie_> the most ambiguous spec in W3C history
<bbaetz> Hixie_: more than svg?
<myksc> in fact, it orders the three transformations in the order i typed them
in above (although it does not order them numerically)
<Hixie_> bbaetz: SVG is not vague, it's broken
<bbaetz> heh
<Hixie_> myksc: hmm, that seems unfortunate
<Hixie_> myksc: in that case we'd better change the way bugzilla escapes stuff
* bbaetz  wishes that he had net access at home so that he could work on bugzilla
<-- Kowalski has quit (Ping timeout: 180 seconds)
<myksc> Hixie_: yeah, or do wrapping on the server side
<Hixie_> myksc: well, we'll still lose hard returns

Comment 11

[Myk Melez [:myk] [@mykmelez]]

From conversations with Ian in the channel, I think the problem here is that
Konqueror is choosing an impractical interpretation of the HTML 4.01 standard
and doing things in a way that no CGI application would ever want.

Konqueror should convert carriage returns to spaces before converting character
entities to characters or take other steps to ensure that double-conversion of
carriage return entities (
 and whatever equivalents exist) does not occur.
The XML specification, in fact, with which the XHTML 1.0 specification is
conformant, explicitly states that double-conversion should not occur.

http://www.w3.org/TR/2000/REC-xml-20001006#AVNormalize

Since Konqueror arguably implements the standard, I am going to leave this bug
open and make it dependent on bug 11901.  In the unlikely event that Bugzilla is
declared XHTML 1.0 compliant before this bug is fixed, the bug could again be
resolved invalid, but let's shoot for server-side wrapping in the near-term to
fix this problem.

Reporter, you might also file a bug on Konqueror at http://bugs.kde.org/ asking
for a more enlightened implementation of the specification in which
double-conversion does not occur.

Bug 47078 may be relevant to this discussion.

Comment 12

[Matthew Tuck [:CodeMachine]]

I've said it before, but I think we should be using display time wrapping as
opposed to entry time wrapping, whether server side or client side.

Comment 13

[Chris Abbey]

Myk, good idea... I've opened a bug with KDE.

For whatever it's worth, you can actually push multiline text around in
hidden fields, I've done this a lot with php scripts. I do exactly what
Ian suggests in your chat session, by escaping newlines and html as such:

echo '<INPUT TYPE="HIDDEN" NAME="title" VALUE="';
echo htmlentities(stripslashes($notes));
echo '">';
where $notes holds a multiline text value.

This works with Mozilla, NS 4x, IE, Konqueror, Opera and Lynx.

Comment 14

[Chris Abbey]

It just occured to me that bug 82809 is probably related somehow. ;)

Comment 15

[Myk Melez [:myk] [@mykmelez]]

You are right, Matty, although I don't know how we could convert old comments to
their pre-formatted selves.  Is there a bug filed on this?

Comment 16

[Myk Melez [:myk] [@mykmelez]]

Now that I think about it some more, I am not so sure we should be doing
display-time wrapping.  What about sections of code?  Should we allow users to
enter some kind of tag to delineate a section that should not be wrapped?

Comment 17

[Chris Abbey]

Myk, if you rely on the user to insert special tags for line feeds, or for
preformatted sections then I predict you'll end up with a lot of reports that
look like 97979 did... a mess. Especially those that are submitted by new users,
not exactly the kind of first impression you want is it?

On the otherhand, if you preserve the client's line feeds from when they submit
it, and replace thouse with <br> tags when you render the results then you'll be
able to support both free form paragraph text (as I'm entering it here in this
dialog, only a lf at the end of each para.) AND formatted, or otherwise
deliniated text such as stacks, code snippets, patches, etc. The difficulty that
presents is the one this bug was raised for... you have to preserve the users'
input linefeeds in the process of feeding it back to them in a hidden field and
back to the server a second time (possibly by escaping it into something other
than a linefeed) then store those linefeeds in a format that can be spit back on
demand (probably by writing break tags to the database directly).
This comment for example should only have 5 line feeds in it. One at the end of
each para, and one between each pair of paras.

Comment 18

[Matthew Tuck [:CodeMachine]]

We shouldn't rewrap existing comments.

As for code, I agree, but we can't do that already in the current system.  With
display time wrapping it would be better because we could make the lines longer.

Comment 19

[Matthew Tuck [:CodeMachine]]

cabbey, no matter how we do things, we have to have some way for the user to
indicate whether they want to wrap the text.

The only way to avoid this is to require users to put their own line feeds. 
This is very user unfriendly, results in paragraphs that don't resize to the
screen, and is not an option IMO.

Comment 20

[Dave Miller [:justdave]]

breaking dependency on 82809.  I just figured out what was broke on 82809, and
it wasn't related to this at all.

Comment 21

[Dave Miller [:justdave]]

We are currently trying to wrap up Bugzilla 2.16.  We are now close enough to
release time that anything that wasn't already ranked at P1 isn't going to make
the cut.  Thus this is being retargetted at 2.18.  If you strongly disagree with
this retargetting, please comment, however, be aware that we only have about 2
weeks left to review and test anything at this point, and we intend to devote
this time to the remaining bugs that were designated as release blockers.

Comment 22

[Dave Miller [:justdave]]

This will make 2.16 because it'll be fixed by bug 11901, which is a 2.16 release
blocker.

Comment 23

[Dave Miller [:justdave]]

11901 just got bumped to 2.18, so this goes with it.

Comment 24

[Randall M! Gee]

I just wanted to point out that bug 11901 won't fix this bug.

Bug 11901 will wrap the affected comments, yes.  But you still lose the
newlines the user entered himself, and that's important.

Hard newlines are going to be used to:
  * separate paragraphs
  * delimit lists
  * copy and paste program output or source code
    + especially Python source code
    + or IRC logs
  * ASCII art
  * and much more!

Consider what this comment, or other comments on this bug, would look like
without hard newlines.

Comment 25

[Dave Miller [:justdave]]

There are two parts to this issue...  part of it will be fixed by bug 11901. 
The other part will not.  The second part is a problem with Konqueror, not Bugzilla.

See http://bugs.kde.org/show_bug.cgi?id=32306

I'll leave this open until someone notices if they've fixed it on their end
(they haven't yet, according to the bug report)

Comment 26

[Dave Miller [:justdave]]

This is now fixed in the Konqueror's CVS.  Their next release will probably
include it.

To: [e-mail]
Sender: [e-mail]
Reply-To: [e-mail]
Subject: [Bug 32306] hidden form submissions implement standard oddly         

http://bugs.kde.org/show_bug.cgi?id=32306     
[e-mail] changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|                            |FIXED



------- Additional Comments From [e-mail]  2002-12-02 02:57 -------
Subject: kdelibs/khtml/html

CVS commit by mueller: 

- fix waldo's last commit
- submit stale CR's
CCMAIL: [e-mail]


  M +24 -48    html_formimpl.cpp   1.302
  M +1 -3      html_formimpl.h   1.130