Open Bug 1092834 Opened 10 years ago Updated 2 years ago

Lines wrap error >79 characters on reply to posts on Usenet groups (wrap at 80 bytes is not executed by format=flowed, if long line is generted by mailnews.wraplength=large or ZERO, or by text quoting of long line upon Followup)

Categories

(MailNews Core :: Composition, defect)

x86_64
Windows 7
defect

Tracking

(Not tracked)

People

(Reporter: rick_hughes, Unassigned)

Details

(Whiteboard: [workaround: Ctrl+R(Rewrap), or mailnews.send_plaintext_flowed=false to avoid RFC violation])

Attachments

(5 files)

Attached image TB error.jpg
User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:33.0) Gecko/20100101 Firefox/33.0
Build ID: 20141027150301

Steps to reproduce:

Previously worked in 'older versions'  - clean install of 31.2   now fails.
In usenet (TB as newsreader) go to a post that someone has created with lines of >79 characters.
These lines auto resize if you resize read window, so read  'wrap' is fine.

Click on 'follow up'  and enter text.
Click send and it fails ... with "Lines longer than 79 characters"  as the originator text is >79 characters 
Here is a typical post that fails  (100%) ..

use any newsreader and subscribe to    uk.d-i-y
Post details:

>>>

X-Received: by 10.236.39.174 with SMTP id d34mr1922672yhb.54.1414172134307;
Fri, 24 Oct 2014 10:35:34 -0700 (PDT)
X-Received: by 10.140.86.179 with SMTP id p48mr18315qgd.35.1414172134220; Fri,
24 Oct 2014 10:35:34 -0700 (PDT)
Path: aioe.org!news.glorb.com!k15no561443qaq.1!news-out.google.com!u5ni10qab.1!nntp.google.com!s7no319392qap.0!postnews.google.com!glegroupsg2000goo.googlegroups.com!not-for-mail
Newsgroups: uk.d-i-y
Date: Fri, 24 Oct 2014 10:35:34 -0700 (PDT)
Complaints-To: groups-abuse@google.com
Injection-Info: glegroupsg2000goo.googlegroups.com; posting-host=109.156.173.154;
posting-account=sOwz7woAAAChzQkSPwPZ-a13ArSs2FVv
NNTP-Posting-Host: 109.156.173.154
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <129a2cc8-f5fc-41c7-ba54-3425851a5cee@googlegroups.com>
Subject: Has anyone built a lightbox?
From: robgraham <robkgraham@btinternet.com>
Injection-Date: Fri, 24 Oct 2014 17:35:34 +0000
Content-Type: text/plain; charset=ISO-8859-1
Xref: aioe.org uk.d-i-y:466913

I've been asked to build an A3 size lightbox by a lady who is doing a botanical art course and has need of such for tracing leaf shapes and the likes onto paper.

I can get what is billed as suitable opaque perspex off Ebay - A3 and 5mm thick - I'm wondering if a suitable light source would be part of one of those strings of LED's at around a fiver for 5m.

Rob


Actual results:

When you try to send you get an error (screen grab attached) that txt is longer than 70 characters  (in this case originators text not my reply)

If you cancel the error popup and carry out CTL+R  it will then send OK .... unless there is a url > 79 characters ... as reformat does then not work.





Expected results:

 When you click on follow up it should automatically format test to 79 characters then wrap the line.
Been tested by several on Mozilla support who can also  recreate the problem.

Change 'wrap' settings to 'true' in advanced options makes no difference.

This worked OK in previous versions, some on current version report it works OK ... but many seeing the same issue.
Some have reported downgrading to ver 28 then works (not tested that)
line length is determined by your option setting.
What is set by you for wrap related options? 
   Tools/options/Advnced\General. Config Editor,  Search: wrap
   mailnews.wraplength = 72?

> Content-Type: text/plain; charset=ISO-8859-1

IIRC, if iso-8859-1, format=flowed is used by Thunderbird, so line length is less than 78 bytes(excluding CRLF). 
What do you set in mailnews.send_plaintext_flowed?

> Line logner than 79 chars

Server response? or Tb's warning message before send?
If server response, why the original post of "line longer than 79" is accepted by the news server?
Can you get NNTP protocol log? (see bug 402793 comment #28)

If quoted text is intentionally made long, "line longer 80" is observed by "Send Later" in Tb 32.1.2.
  1. Followup
  2. Add many words to quoted text line
  3. Send Later
  => line for quoted text is longer than 80
        Content-Type: text/plain; charset=windows-1252; format=flowed
        On 2014/06/21 15:47, xxx@z.y.z wrote:
       > original text added-word added-word ... added-word added-word
  => In news post display at Outbox, long line is shown like next(width depends on window width).
           |  original text added-word  ... added-word |
           |  added-word  ... added-word                     |
       Long quoted line is shown in multiple lines.

When composing followup, how quoted text was shown by Tb?
In single long line longer than 80 chars?
What message source is generated by "Send Later"?
FYI.
When text line is long in original, "Long quoted text line" is not observed by Reply. "Long quoted text line" is observed only by "Forward in Inline".
"Followup" is perhaps a kind of "Inline Forward" instead of a kind of "Reply".
#1  "line length is determined by your option setting.What is set by you for wrap related options? "

All settings are default ...  I looked in options > advanced > config editir    it has this entry:
mailnews.wraplength;72

#2 "What do you set in mailnews.send_plaintext_flowed?"
set as default:  mailnews.send_plaintext_flowed;true

#3  "When composing followup, how quoted text was shown by Tb?"
I just used 'follow up' button, in the example post given ... this is what I see when I press 'follow up'
................>

On 24/10/2014 18:35, robgraham wrote:
> I've been asked to build an A3 size lightbox by a lady who is doing a botanical art course and has need of such for tracing leaf shapes and the likes onto paper.
> 
> I can get what is billed as suitable opaque perspex off Ebay - A3 and 5mm thick - I'm wondering if a suitable light source would be part of one of those strings of LED's at around a fiver for 5m.
> 
> Rob

...................

Although when I paste it in here it is wrapped, on TB it is not ... lines are full length ... first line starts with word "I've ... and ends with word "paper"
i.e 130 characters.

This Bug forum treats it as you would expect .. i.e wraps the text, and previously TB newsreader has always wrapped the text ... several are reporting the same issue.
> 


#4  "What message source is generated by "Send Later"?"
Sorry I don't understand that question, all I press is 'follow up' and never see anything related to send later.

#5 on your 2nd comment, regarding "Follow up" ..  TB was changed to add "Follow up"  this was not there in previous versions ... is it possible something changed formatting when this was implemented.


My last 'clean install' would have been about 12 months ago (at time of last OS install) .. TB had been working fine up until then, and continued working fine, with all incremental updates as they are issued.
The problem was only seen on recent clean install.

Several people on Usenet have also reported seeing same issue in ver 32 ....... one person said he downgraded to ver 28 and all was fine .......... I have not tried that.


#6  "Can you get NNTP protocol log?"
I read post #28 ... and it is not that clear, is there a dummies guide ? ..... do you for example open cmd shell and execute script there ? ........ or is this somehow 'within'  TB
Do I need to set both varibales as a one off then access newsreader ?     

I'm just a user of TB .... my knowledge is lmited - but happy to follow instructions.
(In reply to rick_hughes@btconnect.com from comment #3)
> On 24/10/2014 18:35, robgraham wrote:
> > I've been asked to build an A3 size lightbox by a lady who is doing a botanical art course and has need of such for tracing leaf shapes and the likes onto paper.
> > 
> > I can get what is billed as suitable opaque perspex off Ebay - A3 and 5mm thick - I'm wondering if a suitable light source would be part of one of those strings of LED's at around a fiver for 5m.
> > 
> > Rob
> 
> ...................

"> " at beginning of a line == quoted text in bugzill.mozilla.org, so quoted text in composed message was fortunately shown as-is.

Phenomenon is;
(a) When quoted text in text/plain message is long, Tb displays it with fitting message display area width.
(b) When line length of original post is long, quoted text line by "Follow Up"(or Inline Forward) is kept long,
      and the long line is sent as-is, regardless of mailnews.wraplength=72 and mailnews.send_plaintext_flowed=true.
(c) It looks that your news server accepts "line length > 79" in some cases but rejects it when Tb send it as "Follow Up".
      It may be difference between format=Not-Flowed and format=flowed.
          original post         : Content-Type: text/plain; charset=ISO-8859-1 => not format=flowed.
          "Follow Up" in Tb : Content-Type: text/plain; charset=xxx; format=flowed is perhaps used in your case.

> #4  "What message source is generated by "Send Later"?"
> Sorry I don't understand that question, all I press is 'follow up' and never see anything related to send later.

"Send Later" is to see message source which is generated by "Follow Up" of Tb for original post with long line.
Do you understand this?
Do next, please.
(1) "Follow Up" of the original post with long line.
(2) "Send Later"
(3) Go "Outbox" folder of "Local Folders" account.
(4) View message which is generated in Outbox by Tb.
(5) View Message Source of the message which is generated in Outbox by Tb.
If you need to show the message source to other people, save as .eml file and attach the .eml file to this bug after removing/replacing personal data from the .eml file. Please don't paste long mail data in order to keep bug readable.

If composing charset=iso-2022-jp, Tb perhaps won't set format=flowed.
Do you see your problem with composing charset=iso-2022-jp?
   "Follow Up", change character encoding to iso-2022-jp,
   then "Send Later" and check message source, finlally "Send this mail iimmediately" at Outbox.
Does your news server accept the post?

> #6  "Can you get NNTP protocol log?"
> I read post #28 ... and it is not that clear, is there a dummies guide ?

"read post #28" implies "read pointed documents in the comment well".
Phenomenon was observed with news.mozilla.org, mozillaa.test, with Tb 31.2.0, and it's not quoted text for long line" only issue. It was observed on initial news posting.

Steps to reproduce.
   - mailnews.wraplength=1024 to force long line length
     mailnews.send_plaintext_flowed=true
   - create a news post to mozilla.test with long line.
     - charset=utf-8 => sent in format=flowed, but sent as-is. Wrapping at 80 bytes, which format=flowed requests, is not done.
       If this is ctually sent as-is, the news post didn't appear at mozillaa.test. It perhaps one of next.
       - news.mozilla.org rejects it, then send error occurs, thus bug 780124 occurs and message was saved in Sent.
       - News post is successful, but news.mozilla.org ignored it because of RFC violation(long line even though format=flowed)
    - charset=iso-2022-jp => sent without format=flowed, so the news post appeared at mozilla.test.

Format=flowed is ignored?
What should be done on "lo-ng quoted text line" when format=flowed is used?

Anyway, "long line even though format=flowed" was observed, so confirmin.
Status: UNCONFIRMED → NEW
Component: Untriaged → Composition
Ever confirmed: true
Product: Thunderbird → MailNews Core
Version: 31 → 32
Summary: Lines wrap error >79 characters on reply to posts on Usenet groups → Lines wrap error >79 characters on reply to posts on Usenet groups (wrap at 80 bytes is not executed by format=flowed, if long line is generted by mailnews.wraplength=large or ZERO, or by text quoting for long line of Follow Up/Inline Forward)
As far as mailnews.wraplength=72(==default) is used by you, there is no need to send in format=flowed.
So workaround is : mailnews.send_plaintext_flowed=false.
If mail sender/news poster sent in long line, there is no problem in sending "long quoted text line" when Inline Forward/Follow Up, as far as format=flowed is not set by Thunderbird.
Whiteboard: [workaround: mailnews.send_plaintext_flowed=false]
Summary: Lines wrap error >79 characters on reply to posts on Usenet groups (wrap at 80 bytes is not executed by format=flowed, if long line is generted by mailnews.wraplength=large or ZERO, or by text quoting for long line of Follow Up/Inline Forward) → Lines wrap error >79 characters on reply to posts on Usenet groups (wrap at 80 bytes is not executed by format=flowed, if long line is generted by mailnews.wraplength=large or ZERO, or by text quoting in Follow Up/Inline Forward for long line)
FYI.
Behaviour of news.mozilla.org was different from your news server.
   news.mozilla.org accepted "long quoted line which is longer 80 bytes" with format=flowed.
Summary: Lines wrap error >79 characters on reply to posts on Usenet groups (wrap at 80 bytes is not executed by format=flowed, if long line is generted by mailnews.wraplength=large or ZERO, or by text quoting in Follow Up/Inline Forward for long line) → Lines wrap error >79 characters on reply to posts on Usenet groups (wrap at 80 bytes is not executed by format=flowed, if long line is generted by mailnews.wraplength=large or ZERO, or by text quoting of long line in Followup)
Summary: Lines wrap error >79 characters on reply to posts on Usenet groups (wrap at 80 bytes is not executed by format=flowed, if long line is generted by mailnews.wraplength=large or ZERO, or by text quoting of long line in Followup) → Lines wrap error >79 characters on reply to posts on Usenet groups (wrap at 80 bytes is not executed by format=flowed, if long line is generted by mailnews.wraplength=large or ZERO, or by text quoting of long line upon Followup)
FYI.
If originaal message is format=flowed, "wrap of quoted text lines at 80 bytes for format=flowed" was seen, even when "line longer than 80 bytes" is contained in original message.
It looks that quoting is done on text lines of original after appication of format=flowed for mail display.
Received many emails ... is there any specific information you need from me ?
(In reply to rick_hughes@btconnect.com from comment #9)
> Received many emails ... is there any specific information you need from me ?

Mails from bugzilla.mozilla.org is merely a notification of "something is changed". 
B.M.O. is not mailing list. Read thru this bug of bugzilla.mozillaa.org, please. My asking to you is comment #4.
Attached image send later.jpg
Attached image compose.jpg
As per request ...
used : follow up
       entered long line of text

It shows original mail as 'non-wrapped' and my reply long line of text as 'wrapped'   no Line breaks were added by me.
Screen shot attached as screen shot  "compose.jpeg"

Then as requested used "Send Later"
   opened up outbox and looking at file it shows both original an my text as wrapped .... which is action I would expect.
Screen shot attached as attachment "send later"

As regards character set ... all is as default install ... header information states it is:
Content-Type: text/plain; charset=windows-1252; format=flowed
Data of screen shot is not sent to your news server. Same data stream as "data generated by Send Later" will be sent to your news server.
And, Tb has cpability to display long line with fitting window width.

Problem in your case is :
   When Followup post is created by Thunderbird for a news post which has long line without format=flowed, 
   your news server rejects it.
I can guess what occurs from pasted mail data and screen shots, but "actual data in your environment where your problem occurs" is needed.
Attach message source data to this bug, please.
(0) Delete all mails in Outbox of "Local Folders".
(1) With mailnews.send_plaintext_flowed = true (default),
      do "Followup" to the news post which has long line without format=flowed, "Send Later".
      Call the generated post POST-A.
      Save POST-A as .eml file. Call EML-A.
      Long quoting line is perhaps generated with format=flowed.
(2) With mailnews.send_plaintext_flowed = false,
      do "Followup" to the news post which has long line without format=flowed, "Send Later".
      Call the generated post POST-B.
      Save POST-B as .eml file. Call EML-B.
      Long quoting line is perhaps generated without format=flowed.
(3) Edit EML-A and EML-B by Text Editor, remove/replace personal data, 
      and attach EML-A and EML-B to this bug(not paste)
(4) "Send Unsent Messages" at Outbox of "Local Folders".
      Is both posts accepted by your news server?
Attached file 20141110_A.eml
Attached file 20141110_ B.eml
I have carried out what was requested and attach 'send later' eml file ... A9SET TO TRUE) and B (set to false)
Neither of them would send, same error about 79 characters
(In reply to rick_hughes@btconnect.com from comment #17)
> Neither of them would send, same error about 79 characters

If 20141110_ B.eml is rejected by your news server(long quoting line, but no format flowed), why can the original post(long body text line, without format-flowed) be accepted by your news server?

20141110_ B.eml
> X-Mozilla-News-Host: nntp.aioe.org
Original post
> Path: aioe.org
> !news.glorb.com!k15no561443qaq.1
> !news-out.google.com
> !u5ni10qab.1
> !nntp.google.com!s7no319392qap.0
> !postnews.google.com
> !glegroupsg2000goo.googlegroups.com!not-for-mail

Posted via different news server? 
Or posted using browser? (no Mime-Version: header, no User-Agent: header, googlegroups.com is seen)
I have no idea ... I am only a user of service.
Other users report they are not having any issues.
Phenomenon itself doesn't depend on news server, so it can be observed with any news server.
See message source of following news threads of mozilla.test of news.mozilla.org, please.
>  news://news.news.news:119/FPOdnSzmM8Ndq_zJnZ2dnUU7-eWdnZ2d@mozilla.org
>      Subject: Bug 1092834 Test, Long Line, Without format=flowed
>  news://news.news.news:119/66KdnXgOv7T_rvzJnZ2dnUU7-c2dnZ2d@mozilla.org
>      Subject: Bug 1092834 Test, Long Line, format=flowed

Problem is:
(A) When following option(wraplength is large), 
          mailnews.wraplength;1024
          mailnews.send_plaintext_flowed;true
      Tb sends without wrap at 80 bytes, even though Tb sends with format=flowed
(B) If original post is format=Fixed(No format=flowed) and has long line,
      When following option(good wraplength), 
          mailnews.wraplength;72
          mailnews.send_plaintext_flowed;true
     Tb doesn't do "wrap at 72" of quoted text line upon Followup,
     and Tb sends the long quoted text line as-is,
     even though Tb sends with format=flowed(==problem A)

Workaround in your case(news server rejects line longer than 79) :
    At Followup window,
       Ctrl+A, Ctrl+X, Ctrl+V
       => quoted text line is wrapped at 72 by "Paste" operation
       CRLF between "... wrote:" and ">" is lost by "Cut&Paste",
       So, manually insert CRLF between "... wrote:" and ">".
About "Workaround" in my comment #20.

When the "Workaround" is done, Followup is sent with "line length"<80 by Tb, then "rejects by your news server" is avoided.
However, if the "Workaround" is done with mailnews.send_plaintext_flowed=true,
"quoting by > as first byte of a line" is lost, because Tb inserts space before each ">",.

Do "Workaround in my comment #20" with mailnews.send_plaintext_flowed=false, mailnews.wraplength=72, please.
(In addition to comment #21)
If you want good quoting of original body text in your Followup,
manually insert ">" at top of each line generated from long line in original post,
with the "Workaround", please.
Can't see any point in the workaround ... 
Ctrl+A, Ctrl+X, Ctrl+V  then manually insert CRLF between "... wrote:" and ">".

I have use CTL+R   and it usually does teh same thing (OK not 100% workaround)

Or do you want me to try workaround as a test for results ?


Is this seen as  'bug' to fix, or accepted way for working ?
Checked same posts that have issues with TB  with other newsreader programs and they work fine, my preference would be to use just TB ... but reading plain ascii text newsgroups should be a very simple task .... it was all setout in standards 20 years ago ... nothing new.
Sorry. CTL+R is better and sufficient and correct workaround.

> Is this seen as  'bug' to fix, or accepted way for working ?

I thought "job like Ctrl+R" is already done by Tb upon first "Followup text with quoting" display.
"Rewrap is not executed upon first Followup display by Tb" may not be bug of Tb.
However, "longer line than 80 is sent by Tb even though Tb sets format=flowed" is apparent RFC violation by Tb.
Rewrap should be called at least upon send by Tb, if format=flowed is  used by Tb.

When original post is format=flowed, "first Followup text display by Tb" is same as one which is shown by Rewrap, even if original post of format=flowed has longer line than 80.
Problem can be called:
(i)  When original is not format=flowed, Rewrap is not called before first "Followup text" display.
(ii) Even when Followup is format=flowed, Rewrap is not called before Send.
It's perhaps a regression in Tb 31.
Whiteboard: [workaround: mailnews.send_plaintext_flowed=false] → [workaround: Ctrl+R(Rewrap), or mailnews.send_plaintext_flowed=false]
You are right CTL+R, did not work in this instance.
So is this a genuine bug that needs a fix?
(In reply to rick_hughes@btconnect.com from comment #25)
> You are right CTL+R, did not work in this instance.

What case do you call by the "CTL+R, did not work in this instance"?

I couldn't observe "Manual CTL+R doesn't work when original post is not format=flowed and it has line longer than 80, with mailnews.wraplength=72"...
Maybe it wasn't on that one ... but have had this on some posts ... although CTL+R does work most times.
(In reply to rick_hughes@btconnect.com from comment #27)
> Maybe it wasn't on that one ... but have had this on some posts ... although CTL+R does work most times.

(a) If quoting was done multiple times by multiple Followup, ">" for quoting increases by each Followup.
      >>>...(80 >)...>>> Text in original post
(b) Long word is not split by wrapping. Long URL is a kind of word in Plain Text, and is not split.
In these cases, your news server perhaps rejects it, because it exceeds 79 chars.
Such case?
Possibly, but would expect a newsreader to be able to cope with this, and format based on actual outgoing line length.
Always used to work this way.
(In reply to rick_hughes@btconnect.com from comment #29)
> Possibly, but would expect a newsreader to be able to cope with this, and format based on actual outgoing line length.
> Always used to work this way.

But what can news reader do when case (a) and case (b)?
Even if Tb will call Rewrap correctly as we expect again, I think case (a) and case (b) can occur, and your news server rejects it.
IIUC, "split of continuous > for quoting" breaks quoting by ">" in Plain Text.
IIUC, "split of URL string" breaks linkify of URL in Text Plain by Tb and other mailers.
I think (a) is rare, but I believe (b) can occur in ordinal circumstances.
Are you saying "Tb should send in HTML" or "Tb should send in base64 or quoted-printable" in such cases?
Or "Tb should send without format=flowed if line length exceeds 80 due to (a)/(b)" in order to void RFC violation on format=flowed?
Please note that your news server looks to reject "line longer 80 of text/plain" even when format=flowed is not used. IIUC, there is no RFC violation by Tb in this case.
Or, in NNTP, line length has to be less than 80 always? (if so, why "line length greater than 80 without format=flowed" was held in your news server?)
Whiteboard: [workaround: Ctrl+R(Rewrap), or mailnews.send_plaintext_flowed=false] → [workaround: Ctrl+R(Rewrap), or mailnews.send_plaintext_flowed=false to avoid RFC violation]
Just as an update ... I made 15 posts in last 24 Hrs .. and kept a note ... 12 of them failed because of this error.

Something has become broken in TB ... as I had never seen this error once prior to last month.

In each case CTT+A, CTL+R   was applied and worked as a workaround - but this is a pain.
This issue cleared for a while but is now back and again 100% repeatable.  on TB ver38.2.0

To be clear ... I click on a 'Follow up' in a usenet thread,
The original text appears in body and is longer than 79 charters ... I enter my reply text, it is correctly wrapped.
When I click on send I initially get error:

http://i771.photobucket.com/albums/xx351/Tafflad/computer%20stuff/error_zpsczzqn4bs.jpg

If I click on OK I then get:

http://i771.photobucket.com/albums/xx351/Tafflad/computer%20stuff/error%202_zpsbmbyk6il.jpg

This pic is useful as it clearly show original text is > page width, and my response is formatted to correct length.

If I do a  CTL+A  follwd by CTL+R   it reformats and send OK.

This happens on multiple newsgroups ... it does not happen with other Newreaders I have tested.

This must be a bug or TB config issue ?
(In reply to rick_hughes@btconnect.com from comment #32)
> This issue cleared for a while but is now back and again 100% repeatable. on TB ver38.2.0

If so, it may be a result of changes around charset relevant features, for example, "Quirks on non-RFC2047-encoded binary of some non-ascii charset in Subject:" is not done from recent Tb release.

This bug is problem when format=flowed is used in NNTP posting. IIRC, format=flowed was internally killed if iso-2022-jp is used for long time. If format=flowed is used in new Tb for iso-2022-jp, problem of this bug may start to occur from the new Tb release.
And, "reject or not-reject" depends on news server, so different phenomenon can occur on different news server, even if absolutely same NNTP posting.
Please check message source. Even if NNTP posting is rejected by news server, you can always check message source generated by Tb by "Send Later".
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: