1587813 - missing paragraph of text from message that have IBM437 encoding on nntp account

Status: RESOLVED WONTFIX

(Thunderbird :: Message Reader UI, defect)

Description

[Fernando Toledo]

Attachment: cap1.png

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:69.0) Gecko/20100101 Firefox/69.0

Steps to reproduce:

• Create new account on BBS (via telnet bbs.docksud.com.ar port 23) (sorry server do not accept anonymous, if you need, i can provide a test account to avoid new account proccess. Just let me know.

• Add a nntp account, use your user/pass and point to news server "bbs.docksud.com.ar"

• Select newsgroup ZONA4.GENERAL

• Read messages that have IBM437 as content-type (most of mesage have that encoding)

Actual results:

Some part of text are missing when read the message (see cap1.png) , if manual switch to unicode, the text appear but with encoding errors (see cap3.png)
the source of message are on cap2.png

Expected results:

All text of message must show correctly. and IBM437 should be detected

I'm do not skill over the TB source code, if someone can tell me where to start researching to add support for new encoding, I can try to generate a patch.

Comment 1

[Fernando Toledo]

Attachment: cap2.png

Comment 2

[Fernando Toledo]

Attachment: cap3.png

Comment 3

[Jorg K (CEST = GMT+2)]

Looks like IBM437 (https://en.wikipedia.org/wiki/Code_page_437) is not supported be the modern encoding standard:
https://encoding.spec.whatwg.org/

Henri?

Reporter, we don't need access to the news server, just save a post/message as .eml file and attach it here using "Attach New File" above.

Comment 4

[Fernando Toledo]

thanks for your reply. I'm attach the .eml file

Comment 5

[Fernando Toledo]

Attachment: ZCC.eml

Comment 6

[Jorg K (CEST = GMT+2)]

Well, as Western/windows-1252 it doesn't look so bad in TB 68.1.1, only two hiccups:

Hola Flavio!

El Mi‚rcoles 25 Septiembre 2019 a las 23:27, Flavio Bessa escribi¢ a Manuel 
Adorni:

 FB>         Aparentemente es el viejo problema del nodelist, como te
 FB> acuerdas hubo un cambio en el Z1C (Nick Andre ahora es el ZC) y puede
 FB> ser por eso que tus actualizaciones en el nodelist estan fallando.

Si, estoy solucion ndolo.

Abrazo.

­Saludos!
-=Momia=-

escribí and solucionándolo. Nothing's missing, ah yes, and Miércoles. But then the author didn't care about accents anyway, see "estan", jaja.

Comment 7

[Fernando Toledo]

I know that is a old encoding, but many BBS's systems using it, also messages netowork like Fidonet still working.
Another encoding are IBM850 (I use it for Spanish/Argentina).

It's techically possible to thunderbird to support it? Maybe can be detect as iso8859-1 / Windows-125x ?

I do not have problem if some character show weird using workaround, but if misses paragraph is not good user experience.

The final pararaph should be

"Si, estoy solucionándolo"

Thanks!

Comment 8

[Jorg K (CEST = GMT+2)]

I can see that entire paragraphs are missing in TB 60, but that's not the case in TB 68.1.2 which has just been shipped. IBM850 is also not supported, people should really be using windows-1252. I think it's fair to say that we won't fix/implement this.

Comment 9

[Henri Sivonen (:hsivonen)]

I do not have problem if some character show weird using workaround, but if misses paragraph is not good user experience.

The missing paragraph is weird but not worth pursuing given that the problem doesn't occur in newer Thunderbird.

From the Firefox point of view removal of DOS encodings other than 1) those that are the same as Windows encodings and 2) IBM866 was intentional as part of the effort to limit the encodings to what the legacy Web actually needs. ("Actually" being an approximation such that a tiny amount of breakage was accepted.) Keeping IBM866 might have been unnecessary, but at this point, removing it would be more trouble than it would be worth since the standard, Firefox and Chrome have now aligned on the set of supported encodings.

It's techically possible to thunderbird to support it?

Technically, Thunderbird could support message character encodings that Firefox doesn't support for the Web. In practice, Thunderbird does this only for UTF-7 but has consistently resisted doing this for other encodings.

It took 5 years from the removal of the DOS encodings to a Thunderbird-context bug report about them, which suggests the encodings aren't needed much.

Comment 10

[Jorg K (CEST = GMT+2)]

Thanks for the detailed comment, Henri!