1837884 - Please stop using base 64 encoding when replying to a newsgroup posting

Status: NEW

(MailNews Core :: MIME, defect)

Description

[Todd]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/114.0

Steps to reproduce:

Please stop using base 64 encoding when replying to a newsgroup posting.

First original post (by Thunderbird) comes out in UTF-8, as it is suppose to.

Replying to the post with Thunderbird, gets encoded in base64.

Problem: this breaks other news reader programs. (And you get a angry response back.)

Comment 2

[Todd]

Fedora 37
thunderbird-102.12.0-1.fc37.x86_64

Original post:
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Replying to the above post:
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64

Comment 3

[Magnus Melin [:mkmelin]]

That would not normally happen. Just tried it and just normal encoding was used for me.
Is there something special with the message(s) in question?

Comment 4

[Todd]

Original message:

Path: eternal-september.org!news.eternal-september.org!news.mixmin.net!news.neodome.net!feeder1.feed.usenet.farm!feed.usenet.farm!peer02.ams4!peer.am4.highwinds-media.com!peer02.ams1!peer.ams1.xlned.com!news.xlned.com!peer02.iad!feed-me.highwinds-media.com!news.highwinds-media.com!fx12.iad.POSTED!not-for-mail
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0)
Gecko/20100101 Thunderbird/102.12.0
Newsgroups: alt.computer.workshop
Content-Language: en-GB
From: David Brooks <[e-mail]>
Subject: What do folk see here, please?
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Lines: 6
Message-ID: <k4mgM.8049$[e-mail]>
X-Complaints-To: [e-mail]
NNTP-Posting-Date: Thu, 08 Jun 2023 15:02:40 UTC
Organization: blocknews - www.blocknews.net
Date: Thu, 8 Jun 2023 16:02:40 +0100
X-Received-Bytes: 853
Xref: news.eternal-september.org alt.computer.workshop:175204

https://developer.apple.com/forums/profile/Purbeckview

TIA

--
David

My response:

Path: eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: T <[e-mail]>
Newsgroups: alt.computer.workshop
Subject: Re: What do folk see here, please?
Date: Fri, 9 Jun 2023 11:06:19 -0700
Organization: A noiseless patient Spider
Lines: 4
Message-ID: <u5vpms$1vc8e$[e-mail]>
References: <k4mgM.8049$[e-mail]>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64
Injection-Date: Fri, 9 Jun 2023 18:06:20 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="8508548e72268a263408989e09834864";
logging-data="2076942"; mail-complaints-to="[e-mail]"; posting-account="U2FsdGVkX18vxzaUJyB99PDab/Co28QeM/joDs8+Qoc="
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
Thunderbird/102.11.0
Cancel-Lock: sha1:YokjH18aV+EW2jUUnT7mVfVAUh0=
In-Reply-To: <k4mgM.8049$[e-mail]>
Content-Language: en-US
Xref: news.eternal-september.org alt.computer.workshop:175349

T24gNi84LzIzIDA4OjAyLCBEYXZpZCBCcm9va3Mgd3JvdGU6DQo+IGh0dHBzOi8vZGV2ZWxv
cGVyLmFwcGxlLmNvbS9mb3J1bXMvcHJvZmlsZS9QdXJiZWNrdmlldw0KPiANCj4gVElBDQo+
IA0KDQoiVGhlIHBhZ2UgeW914oCZcmUgbG9va2luZyBmb3IgY2Fu4oCZdCBiZSBmb3VuZCIN
Cg==

Comment 5

[Alfred Peters [:infofrommozilla]]

(In reply to Todd from comment #2)

Replying to the above post:
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64

I think for format=flowed this is intentional behavior. See: Bug 1702197

If you can live without format=flowed, you could turn it off. Then base64 should also be gone again.

Comment 6

[Todd]

Is format=flowed in the config editor. I am not finding it.

Comment 7

[Alfred Peters [:infofrommozilla]]

(In reply to Todd from comment #6)

Is format=flowed in the config editor.

Yes - see:
https://kb.mozillazine.org/Plain_text_e-mail_-_Thunderbird#Flowed_format

Comment 8

[betterbird.project+6]

There is a long story to tell here. It all started with bug 1689804 which was regressed by bug 1287126. An attempt to fix bug 1689804 caused bug 1705274. So bug 1689804 was backed out.

To avoid the original issue, bug 1702197 implemented CTE base64 for f=f messages instead of QP. This caused bug 1706809. Note that without setting mail.strictly_mime, message are still sent with CTE 8bit. Why is that bad?

What really needs to be fixed is bug 1706822. Basically the MIME processing is processing the data in the wrong sequence, see comment in bug 1689804 comment #76 (quote):
this bug is caused by incorrect ordering of the format=flowed and QP encoders/decoders
Further information in bug 1689804 comment #78 and below.

In summary: It's not possible to go back from base64 to QP without exposing the still not solved bug 1689804. You've have to solve bug 1706822 first. And that would entail reworking MIME code from 2000:
https://searchfox.org/comm-central/search?q=04%2F20%2F2000&path=mime&case=false&regexp=false
That is unlikely to happen. Magnus, are there any plans to replace the old MIME library altogether?

In the meantime, why don't you leave mail.strictly_mime at its default false and use CTE 8bit?

Is format=flowed in the config editor. I am not finding it.

mailnews.send_plaintext_flowed