1675507 - Forwarded HTML mail containing --></style> are not fully displayed in OWA (Microsoft Exchange) - when mail.html_sanitize.drop_conditional_css true

Status: RESOLVED FIXED

(MailNews Core :: MIME, defect)

Description

[Soporte Correo]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.111 Safari/537.36 OPR/72.0.3815.186

Steps to reproduce:

The steps are as follows:

• Forward an email with Thunderbird in html format to a Exchange account.
• If the message is open with Outlook or Thunderbird, the message is displayed correctly.
• But if the message is open with OWA (Exchange webmail), only it shows the latest message, not the forwarded message. It happens because Thunderbird doesn't close an html tag correctly and OWA doesn´t show it properly.

The issue occurs with the latest version. We did a test with Thunderbird 68 and the issue doesn´t happen.

Actual results:

We´ve contacted with the Microsoft support and the problem is that when you forward the message from thunderbird, it doesn't correctly close a tag in the html code.
<style><!--
/* Font Definitions */
....
{page:WordSection1;}</style><!--[if gte mso 9]><xml>

Expected results:

The html code should close the comment tag as follows:

<style><!--
/* Font Definitions */
....
{page:WordSection1;}--></style><!--[if gte mso 9]><xml>

Comment 1

[David]

I am experiencing the exact same thing, emails forwarded to Exchange accounts and opened with OWA only displays what I write and not the forwarded message.

I looked at the message source of the messages I sent and can confirm they all have the problem Soporte Correo pointed out, no closing comment tag within the style tag.

Comment 2

[Magnus Melin [:mkmelin]]

Can you attach a sample mail as .eml? I don't think Thunderbird changes the html.

Comment 3

[mkht]

I'm facing the same problem.
I send an HTML email from Outlook to Thunderbird 78 and forward it. Then some email clients do not display the source message.

I will attach two .eml files, the one I sent from Outlook and the one I forwarded it to.
(Note that I have sanitized the private information from the file.)

Carefully compare lines 86-87 of the first file with lines 122 of the second file.
In the second file, the "-->" is missing.

lines 86-87 of the first file

	{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>

lines 122 of the second file

	{page:WordSection1;}</style><!--[if gte mso 9]><xml>

Comment 4

[mkht]

Attachment: Original email sent from Outlook

Comment 5

[mkht]

Attachment: Email forwarded by Thunderbird 78

Comment 6

[Soporte Correo]

Attachment: original_message.eml

Comment 7

[Soporte Correo]

Attachment: original_message.PNG

Comment 8

[Soporte Correo]

Attachment: forwarded_message.eml

Comment 9

[Soporte Correo]

Attachment: forwarded_message.PNG

Comment 10

[Soporte Correo]

We attached EML examples and how it shows the original message and the forwarded message through OWA:

• Original message: original_message.eml, original_message.PNG
• Forwarded message: forwarded_message.eml, forwarded_message.PNG

When the html code is edited and the tag is correctly closed, for example using MFCMapi, the message is correctly shows in OWA.

Comment 11

[mkht]

I have found a workaround for this issue.

Change the value of mail.html_sanitize.drop_conditional_css in the Config Editor to false.

This will prevent Thunderbird from modifying the html tags.

Comment 12

[mkht]

Attachment: Thunderbird_AboutConfig.png

Comment 13

[Magnus Melin [:mkmelin]]

Thanks!

STR:

• open attachment 9186678 [details]
• press forwards and save as draft
• check the draft
• should contain "{page:WordSection1;}--></style>", NOT "{page:WordSection1;}</style>" (some line breaks can be different)

Comment 14

[Soporte Correo]

Thanks!! the workaround works for me. We will wait for support in case it proposes a definitive solution if it is possible.

Comment 15

[Bart Jacobs]

I am experiencing the same problem. Thunderbird Version 78.4.3 Build ID 20201110184431 on macOS 10.15.7.

Minimal repro:

1. I sent "This is a test." from OWA:

Received: from ICTS-S-EXMBX19.luna.kuleuven.be (10.112.11.50) by
 ICTS-S-EXMBX23.luna.kuleuven.be (10.112.11.58) with Microsoft SMTP Server
 (TLS) id 15.0.1497.2 via Mailbox Transport; Fri, 13 Nov 2020 20:20:59 +0100
Received: from ICTS-S-EXMBX23.luna.kuleuven.be (10.112.11.58) by
 ICTS-S-EXMBX19.luna.kuleuven.be (10.112.11.50) with Microsoft SMTP Server
 (TLS) id 15.0.1497.2; Fri, 13 Nov 2020 20:20:59 +0100
Received: from ICTS-S-EXMBX23.luna.kuleuven.be ([fe80::45cc:a3cb:6fb9:a1f5])
 by ICTS-S-EXMBX23.luna.kuleuven.be ([fe80::45cc:a3cb:6fb9:a1f5%20]) with mapi
 id 15.00.1497.006; Fri, 13 Nov 2020 20:20:59 +0100
From: Bart Jacobs <[e-mail]>
To: Bart Jacobs <[e-mail]>
Subject: Thunderbird bug test
Thread-Topic: Thunderbird bug test
Thread-Index: AQHWufIen8sfGINzTU+pedtXIO+xJg==
Date: Fri, 13 Nov 2020 20:20:59 +0100
Message-ID: <[e-mail]>
Accept-Language: en-US, nl-BE
Content-Language: en-US
X-MS-Exchange-Organization-AuthAs: Internal
X-MS-Exchange-Organization-AuthMechanism: 04
X-MS-Exchange-Organization-AuthSource: ICTS-S-EXMBX23.luna.kuleuven.be
X-MS-Has-Attach:
X-MS-Exchange-Organization-SCL: -1
X-MS-TNEF-Correlator:
Content-Type: text/html; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Diso-8859-=
1">
<style type=3D"text/css" style=3D"display:none"><!-- P { margin-top: 0px; m=
argin-bottom: 0px; }--></style>
</head>
<body dir=3D"ltr" style=3D"font-size:12pt;color:#000000;background-color:#F=
FFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<p>This is a <strong>test</strong>.<br>
</p>
</body>
</html>

2. I replied "This is the reply." from Thunderbird:

Subject: Re: Thunderbird bug test
To: Bart Jacobs <[e-mail]>
References: <[e-mail]>
From: Bart Jacobs <[e-mail]>
Message-ID: <[e-mail]>
Date: Fri, 13 Nov 2020 20:21:52 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.4.3
MIME-Version: 1.0
In-Reply-To: <[e-mail]>
Content-Type: multipart/alternative;
 boundary="------------9F33359F109EB7D8DF997DD9"
Content-Language: en-GB

This is a multi-part message in MIME format.
--------------9F33359F109EB7D8DF997DD9
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit

This is the reply.


On 13/11/2020 20:20, Bart Jacobs wrote:
>
> This is a *test*.
>

--------------9F33359F109EB7D8DF997DD9
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 7bit

<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    <p>This is the reply.</p>
    <p><br>
    </p>
    <div class="moz-cite-prefix">On 13/11/2020 20:20, Bart Jacobs wrote:<br>
    </div>
    <blockquote type="cite"
cite="mid:[e-mail]">
      <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
      <style type="text/css" style="display:none"><!-- P { margin-top: 0px; margin-bottom: 0px; }</style>
      <p>This is a <strong>test</strong>.<br>
      </p>
    </blockquote>
  </body>
</html>

--------------9F33359F109EB7D8DF997DD9--

Comment 16

[Bart Jacobs]

I confirm that setting mail.html_sanitize.drop_conditional_css to false solves the problem.

Comment 18

[mkht]

Is the Thunderbird team working on a fix for this issue?

Comment 19

[Magnus Melin [:mkmelin]]

Yes. Follow bug 1659362.

Comment 20

[Petr Hroudný]

Bug 1659362 has severity S4 i.e. cosmetic - difference in message width.

However, due to missing end of comment (-->) important parts of message are completely lost, so this looks more like S2.

The problem can be reproduced on multiple MUAs and highly confuses users, since they can't see what has been forwarded to them or what the original message was about.

Could you please prepare a fix to properly insert end of comment to <style> sections (this is one line fix) and push it to r78 as soon as possible?

Waiting for cosmetic bug 1659362 might keep user confusion for quite a long time...

Comment 22

[Kai Engert (:KaiE:)]

Bug 1659362 is about display not being as expected, such as layout.

This bug here is more serious, because it hides intended message contents.
I've filed bug 1680084 to track a potential fix in the sanitizer code.

(One could argue this bug and bug 1680084 are duplicates, but maybe it makes sense to cause and symptom separately.)

Comment 23

[Ben Bucksch (:BenB)]

Our ExQuilla users are hitting this, badly - all the more as more are upgrading to TB 78. They are seeing that messages are garbled and contain linebreaks replaced by "\A0" (as visible literals) in the message.

What TB sends out is simply invalid HTML. Microsoft is not at fault here, TB is.

Comment 24

[Kai Engert (:KaiE:)]

(In reply to Ben Bucksch (:BenB) from comment #23)

Our ExQuilla users are hitting this, badly - all the more as more are upgrading to TB 78. They are seeing that messages are garbled and contain linebreaks replaced by "\A0" (as visible literals) in the message.

What TB sends out is simply invalid HTML. Microsoft is not at fault here, TB is.

Ben, the issue you describe sounds different from the other reports in this bug.

Can you please attach a test message, which shows how Thunderbird corrupts those messages?
Is my understanding correct, you have a valid message, and when forwarding with Thunderbird, it replaces some linebreaks with \A0 ?

If that's correct, and you attach a sample original message, we should be able to reproduce the corruption by forwarding your sample message.

Comment 25

[Ben Bucksch (:BenB)]

Kai, we're hitting exactly this bug as described by the original description (comment 0). TB turns valid HTML <style> <!-- foo --> </style> into invalid HTML <style> <!-- foo </style>. The HTML comment is never closed, turning the entire rest of the document into a single HTML comment and therefore the document is technically empty.

This malformed HTML in turn is confusing the Exchange server, after ExQuilla submits this invalid mail body to Exchange for sending. The Exchange server cannot parse the invalid HTML, and apparently starts to act strange. Apparently newlines and/or Unicode characters are turned into literal \A0 by the Exchange server, but as a direct consequence of this invalid HTML that TB sends. You cannot blame Exchange for any misbehavior, because the HTML message we sent is invalid. Garbage in, garbage out.

Many of our ExQuilla users are running into this, and more so as people auto-update to TB 78.

We verified that if we close the HTML comment, i.e. re-add the --> before the </style> as required by HTML, the message is sent correctly. So, it's precisely this bug here at the root cause.

It took us a long time to find the root cause, from the symptoms that the end users reported. We were about to submit a TB bug report and supply a patch, when we found this bug here. It appears you are working on it in bug 1680084, so please give us access to bug 1680084 (by CCing me).

Comment 26

[Kai Engert (:KaiE:)]

Ben, thanks for the clarification.

Emilio has fixed this bug, it has already been landed into mozilla-esr78 and will be part of the next 78.x release (probably 78.6.0).

Comment 27

[Kai Engert (:KaiE:)]

I suggest that we wait until after the next TB release.
Once it is confirmed the issue is fixed, we can resolve this bug as a duplicate of bug 1680084.

Comment 28

[Ben Bucksch (:BenB)]

Thanks for the access, Kai. I concur with you.

Comment 29

[Magnus Melin [:mkmelin]]

Fixed by bug 1680084.

Comment 30

[Nathan Neulinger]

Given the impact of this - is there any intent to apply this fix to 78? It looks from that milestone that it's not intended to be fixed until 85?

Comment 31

[Kai Engert (:KaiE:)]

78.6.0 includes the fix

Comment 32

[Nathan Neulinger]

I think I have whiplash from the turnaround on that bug fix... only found out about the bug 20 minutes ago from Microsoft and from my perspective it's now already fixed. Nice work!

Comment 33

[Nathan Neulinger]

FYI, was asked to post the context of the comment from MS support. Not sure if they found something internally - or their support was just really good about finding this bug. We basically had a case of a user seeing no content on any of my forwarded or replied emails whether with OWA or with ActiveSync/mobile.

Thank you for the meeting yesterday and your patience while I investigated this issue.
The behavior that you're seeing in OWA appears to be due to incorrect HTML code being generated by the Thunderbird client. Specifically it seems that Thunderbird is not closing the style tag correctly in the email sample that you provided and so OWA is running into the rendering issue that you're seeing. Take a look at the following HTML code in the email that you provided:

<snipped image>

A <style><-- tag is opened but there is no corresponding --> to close it.
This is not an issue with the Microsoft 365 service or with OWA. Unfortunately it appears that this is a bug on the Thunderbird side that will need to be corrected by the Thunderbird development team.
A search of their public bug tracker reveals bug 1675507 which seems to match what we're seeing here. You can search for this bug on their public tracker (Bugzilla) however Microsoft does not take any stance towards the accuracy of this bug. I am providing it for informational purposes only.
Please let me know if you have any questions or need clarification regarding anything in this email.

Comment 34

[tom.vercauteren]

In case it helps, I can confirm the same behaviour on mac os with various versions of TB 78 since a couple of months:
https://support.mozilla.org/questions/1308170
https://support.mozilla.org/questions/1307465