22662 - [DOGFOOD] cannot post news message longer than 936 characters

Status: VERIFIED FIXED

(SeaMonkey :: MailNews: Message Display, defect, P3)

Description

[Asa Dotzler [:asa]]

The Problem:  When I attempt to submit a news message that has 985 characters
mozilla gives the error " A News (NNTP) error occurred: line 13 too long "  this
error has an "OK" button.  I click on "OK" and a second allert box pops up that
says " Sending of message failed " and then I am returned to the unsent message.
 I tried to narrow the problem down and all I could reduce it to was that if I
try to send a post with 985 unwrapped characters it refuses but if I try to send
a post with 984 unwrapped characters it does just fine.

Steps to Reproduce: open mozilla mail-news.  go to n.test and click on "New
Msg".  in the message compose window type a message with 985 characters (i'll
include a block of text that is 985 characters long at the end of this report so
you can just copy it if you like).  attempt to send the message.

Actual Results:  the message will not be sent and allerts will be displayed.

Expected Results: the message should be sent.

Tested on mozilla win32 build from 12/23 and 12/24.

Additional information:  the same text _will_ get sent in mail, just not in
news.

testcase text:

12345678901234567890123456789012345678901234567890123456789012345678901234567890
12345678901234567890123456789012345678901234567890123456789012345678901234567890
12345678901234567890123456789012345678901234567890123456789012345678901234567890
12345678901234567890123456789012345678901234567890123456789012345678901234567890
12345678901234567890123456789012345678901234567890123456789012345678901234567890
12345678901234567890123456789012345678901234567890123456789012345678901234567890
12345678901234567890123456789012345678901234567890123456789012345678901234567890
12345678901234567890123456789012345678901234567890123456789012345678901234567890
12345678901234567890123456789012345678901234567890123456789012345678901234567890
12345678901234567890123456789012345678901234567890123456789012345678901234567890
12345678901234567890123456789012345678901234567890123456789012345678901234567890
12345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345

hope that's not too painful.

Comment 1

[(not reading, please use [e-mail] instead)]

I think this is related to another bug that akkana has.

because we aren't generating new lines, we are sending a giant one lined message
to the nntp server, which rejects it.

Comment 2

[Asa Dotzler [:asa]]

Is this bug the same as 20603 ?

------- Additional Comments From [e-mail] 1999-12-07 20:34 -------
This bug is also causing us to post to newsgroups with lines that are too long
for the news server to handle. See mscott's post,
<[e-mail]>, and notice that the server (we think) has added
'!' in two places where the lines end. I think this is a pretty serious bug.

that comment sounds like it could be the same thing.  (man, I spent hours
looking through Bugzilla to see if this one was reported and couldn't find it
until sspitzer's comment)

Comment 3

[Akkana Peck]

The bug on the editor not inserting newlines into the html source is bug 490. It
used to, but something changed in the editor and apparently took out the calls
to insert the formatting whitespace.  I have 490 marked for M13 and hope to work
on it in that timeframe, but it's been marked PDT- and I'm not sure what the
rules will be on checkins for M13, so by all means lobby for it if you think
this is important.

Comment 4

[lchiang]

marking dogfood as this can be a serious problem if we are unable to post
certain messages and can cause some data change (per sfraser's comments)

Comment 5

[Peter Trudelle]

marking PDT- per PDT team

Comment 6

[Asa Dotzler [:asa]]

This is me lobbying for it.  _Please_ fix this bug.
This bug is particularly painful in that a user will not find it until s/he has
typed a pretty long message to post in a newsgroup.  Mail-news testers (the few
people daring enough to put mail and news through the hurdles day in and day
out, and they are few - check the user agent headers in the n.p.m groups) will
stop using mail and news in mozilla and move back to communicator.  I really
want to test this product but this is a serious impediment.  If I can not post
to newsgroups with mozilla then I will not read newsgroups with mozilla which
puts me right back to using communicator for news.  And if I've gone back to
using communicator for news then there's not much chance that I'll be opening
mozilla to read and write mail.  I really want to be using/testing mail and
news.  I want them to be better faster and I spend hours each day using the
product and reporting bugs on it to make that happen.  I am willing to deal with
a slower product because I believe that testing is the way to improvement.  But
I probably won't use mozilla for mail and news until this is fixed because it
blocks me on a key issue - posting to mozilla newsgroups.  I consider this basic
functionality.  Posting to newsgroups is 1/4 of what mail news is.  Please
consider this or bug 490 if that's the real culprit for PDT+ or whatever markers
will give it some priority.

Comment 7

[lchiang]

per email to pdt team, clearing PDT- resolution.  This bug occurs with
anything you type that is over 985 characters long - it doesn't have to be one
continuous string.  For example, I typed in several paragraphs which exceed this
limit and got the error on sending.

Pls re-evaluate for dogfood

Comment 8

[Akkana Peck]

This morning, I checked in some editor code which should help ... now formatting
newlines should be added to the source when nodes are created as well as when
they're split.  This still doesn't help in the case where the message is all one
very long paragraph with no html nodes in between, and there are other problems
with formatting of source, so I've left bug 490 open, but I'm hoping that the
problem with news messages will be much less frequent now.

Comment 9

[Jim Roskind]

Just to be sure we understand... even with that akkana's fix...
If a paragraph has more than 984 characters (roughly 13 lines of 80 characters)
then the news posting fails.  Is that correct?  IF so, I think this would become
a PDT+ bug... so I just wanted to be sure we understood.  With that
understanding, I've put in PDT+ on the status whiteboard.

Comment 10

[Akkana Peck]

Okay, I'll work on the intra-paragraph formatting code for M13.  I had a
tentative fix last week, but after discussing it with Joe we weren't sure where
the formatting should be inserted (during editor insertion, in editor
postprocessing, or by the output system) and that we needed to discuss it more.
I'll bring the issue up at our editor team meeting today and get a resolution.

Phil, my understanding is that the only real bug here is the editor bug, so I'm
taking this.  If there's an additional issue, please let me know or just take it
back.

Comment 11

[Akkana Peck]

Mozilla didn't do the reassign, nor did it change the milestone.  Trying again,
changing both of those fields at once, so that I'll know for sure and can file a
bug if it doesn't work.

Comment 12

[Akkana Peck]

Naturally, it worked that time.  Accepting bug, and also adding Simon to the cc
list.

Comment 13

[Akkana Peck]

After much discussion, here's the solution Joe and I came up with last night:

In the html output sink, when a text node is encountered which has any line
lengths greater than some threshold, say, 128 chars (to try to encompass any
possible intentional line length setting; Joe suggested that perhaps it should
be made longer to account for someone who might want to do tiny-font ascii
graphics inside a <pre> tag), we will assume that that text node has been
edited, and will reformat that line to comform to the line length (typically
72).  Text nodes which do not have long lines will be assumed to be unedited or
previously formatted, and will not be reformatted.

It was tempting to reformat just the long lines, but we thought that this would
lead to ragged line lengths and eventually, after many edits, a situation with
many very short lines.

Comment 14

[Akkana Peck]

The fix is in.  Please let me know if there are any other cases we missed where
this problem occurs!

Comment 15

[Karen Huang]

Used WinNT 2000-01-18-11-M13 commercial build:
Reopen this bug since I still see an error message " A News (NNTP) error
occurred: line 13 too long "  displayed even less than 983 characters.

Comment 16

[Akkana Peck]

And what is on line 13?  Can you do a Debug->OutputHTML and let me know what the
output looks like on a message which will trigger this problem?

Comment 17

[Simon Fraser [no longer active]]

Line 13 may be a line in the headers. You'll need to see the whole outgoing
message to tell.

Comment 18

[Akkana Peck]

OutputHTML will tell us if there are any long lines in the editor's output.  I
can't check this myself right now because the account wizard is busted (blank
screen) and I can't make a news account.

Comment 19

[Karen Huang]

After investigating.....
Can only send news message's characters up to 936 characters.
If over than 936....from 937 characters, will display error message.
Updating summary to change from 984 to 936!!
Will attach the 936 text on this bug!!

Comment 20

[Karen Huang]

Attachment: 936 characters test file

Comment 21

[Karen Huang]

The 2nd error message will displayed "Sending of message failed" after the first
error. I still cannot send this message successfully if I compose news message
over than 936 characters.

Comment 22

[Akkana Peck]

I'd still like to know what Debug->OutputHTML says (or OutputText, if you're
using plaintext compose) before you send the message.

Comment 23

[Karen Huang]

I select the Debug->OutputHTML or OutputText, nothing happened
Where to check the messages? On the console? I saw some messages displayed on
the console...
And, do you mean before send or after send for checking the Debug->OutputHTML?

Comment 24

[Akkana Peck]

The output of OutputHTML or OutputText appears on the console.  You have to do
it before send, because after send the window will go away, won't it?

Comment 25

[Karen Huang]

Yes. it is.
Maybe like you said, it displayed the error header.
Another bug? or wait for verifying on tomorrow build?
I still cannot mark verified for this bug yet!!

Comment 26

[Akkana Peck]

If this bug is still happening but the number of characters has changed
slightly, then I don't see the point in closing it and opening another identical
bug.

But I still want to see what the result of OutputHTML was (on a page which fails
to send).  Pleeeeeeease?

Comment 27

[lchiang]

akkana - this is what it shows in the console when I output to HTML:

Getting HTML
<html><head></head>
<body>12345678901234567890123456789012345678901234567890123456789012345678901234
567890<br>1234567890123456789012345678901234567890123456789012345678901234567890
1234567890<br>123456789012345678901234567890123456789012345678901234567890123456
78901234567890<br>12345678901234567890123456789012345678901234567890123456789012
345678901234567890<br>1234567890123456789012345678901234567890123456789012345678
9012345678901234567890<br>123456789012345678901234567890123456789012345678901234
56789012345678901234567890<br>12345678901234567890123456789012345678901234567890
123456789012345678901234567890<br>1234567890123456789012345678901234567890123456
7890123456789012345678901234567890<br>123456789012345678901234567890123456789012
34567890123456789012345678901234567890<br>12345678901234567890123456789012345678
901234567890123456789012345678901234567890<br>1234567890123456789012345678901234
5678901234567890123456789012345678901234567890<br>123456789012345678901234567890
123456789012345678901234567<br></body>
</html>

--
I'm posting this to the netscape.test newsgroup using today's afternoon build on
Win32.  If I take out the last "7" in the text above, the posting works as huang
says.

Comment 28

[lchiang]

reopening for now while Akkana investigates.

Comment 29

[Akkana Peck]

The mail account wizard doesn't work for me, but if I go to the attachment in
this bug, select all of it, then paste it into an editor window and do
OutputHTML, that shows that there are no newlines.

The problem now isn't that text nodes aren't being broken; it's that when the
editor converts embedded newlines to breaks, it isn't inserting the formatting
that should go along with an inserted break.

Joe (who wrote that code) is sick, so I'll work on a fix.

Comment 30

[lchiang]

Akkana wanted me to try this in plain text.  No problem in plain text posting.

Comment 31

[rubydoo123]

moving this to m14, Akkana will continue to debug the problem, once the fix is
ready, Akkana will have Joe and a few others review the fix.

Comment 32

[Akkana Peck]

Turns out nsHTMLEditor::InsertFormattingForNode was only inserting formatting
for block nodes, and breaks didn't count.  I have a patch that inserts
formatting for breaks as well.  Awaiting code review and a second opinion on how
safe this is and whether we should include it in M13.

Comment 33

[rubydoo123]

moving back to m14

Comment 34

[Akkana Peck]

I have a review for my fix, but the tree was red last night.  I'd be perfectly
happy to check this in to M13 instead of waiting for M14 if someone wants to
argue for it.

Comment 35

[Phil Peterson]

Doesn't seem like an M13 stopper to me.

Comment 36

[Akkana Peck]

The fix has been checked in.  Formatting should be much better now.

Comment 37

[Akkana Peck]

Oops!  I found one more case where it fails: if you type short lines followed by
<br>, the formatting is inserted for the <br> tags, but then the characters
typed next get inserted before, rather than after, the newline.

This may be a simple question of setting the caret to after the newline, or I
might have to dive into the text insertion rules code.  Anyway, I'm looking into
it.

Comment 38

[leger]

Clearing FIXED resolution due to reopen.

Comment 39

[Akkana Peck]

Well, this last bit turned out to be hard to do.  (Well, it was easy to do, but
then it broke caret setting in tables, and fixing that was hard.)  So hard, in
fact, that after banging our heads against a wall for a few hours, Joe, Charley
and I decided we'd be better off by just redesigning the whole thing.

Just checked in the redesign.  Editor output should be formatted MUCH more
nicely now, and there shouldn't be any more problems with newlines not making it
into the source.

Comment 40

[Karen Huang]

Verified On WinNT 2000-01-28-09 WinNT commercial build
Verified On Linux 2000-01-28-08 WinNT mozilla build
Can post news message longer than 936 & 984 characters now.
I mean that there are no problems for posting news message for 937 & 985 
characters now. Marking as verified!!

Comment 41

[Asa Dotzler [:asa]]

Thank you akkana, huang, and lchiang for the time and effort you put in on this 
bug.  I really do appreciate it. :D