1659363 - Google calendar event descriptions contain HTML tags but Thunderbird renders descriptions as plain text

Status: RESOLVED FIXED

(Calendar :: E-mail based Scheduling (iTIP/iMIP), defect, P2)

Description

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Attachment: meeting invitation display without vs with lightning.png

Thunderbird 78.1.1 on Windows 8.1

Thunderbird's Lightning replaces the content of messages from Google Calendar about event invitations/updates with its own display. Sometimes, it renders the event description with HTML as plain text (right side of screenshot, left side Thunderbird 68.x with Lightning disabled).

From a different email with the issue: The .ics has the description field starting like this:
DESCRIPTION:<br>Password to join the meeting: <span>

Keeping this as a separate bug which is only about not displaying raw HTML.

Comment 1

[Magnus Melin [:mkmelin]]

IIRC the standards do not allow HTML in the description.
Anyway, this causes major problems with Google calendar invites, so we should find a solution.

For bug 1607834, there's also X-ALT-DESC to handle - but at least that's standards compliant.

Comment 2

[Paul Morris [:pmorris]]

Apparently DESCRIPTION is a plain text property in the icalendar spec:
https://icalendar.org/iCalendar-RFC-5545/3-8-1-5-description.html

An HTML version of a description would go in the X-ALT-DESC property (with a FMTTYPE declaration of text/html):
https://stackoverflow.com/questions/854036/html-in-ical-attachment

From discussion with Magnus today, the plan is to leave the content of the property as-is but when displaying it, display it as plain text with any html tags removed. For example, by using parserUtils.convertToPlainText like here: https://searchfox.org/comm-central/rev/568e955cf231f8265491df1ada8800c3d0daaecd/mail/base/content/mailWindow.js#160

Comment 3

[Paul Morris [:pmorris]]

Attachment: Bug 1659363 - Don't show HTML tags in calendar invite event descriptions. r=darktrojan

Google calendar puts HTML tags into calendar event descriptions,
for formatting etc., but that's not following the iCalendar spec
where DESCRIPTION is plain text. Don't show HTML tags in
descriptions in invitations.

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/c2e7b598699a
Don't show HTML tags in calendar invite event descriptions. r=darktrojan DONTBUILD

Comment 5

[Geoff Lankow (:darktrojan)]

Backed out for breaking tests:
https://hg.mozilla.org/comm-central/rev/61ba4ec62a13ffebb8c4b12e408cff6ff4f51389

Comment 6

[Magnus Melin [:mkmelin]]

Re sanitizing, there is a sanitize - https://searchfox.org/mozilla-central/source/parser/html/nsIParserUtils.idl#108

Comment 7

[Paul Morris [:pmorris]]

Attachment: Bug 1659363 - Fix some typos in comments in chat code. r=clokep

Comment 8

[Paul Morris [:pmorris]]

Attachment: Bug 1659363 - Render HTML in calendar item descriptions. r=darktrojan

Google puts HTML tags in calendar event/task DESCRIPTION
properties, so where we display descriptions render them
as HTML rather than just showing the raw plain text.
This commit does this in these places:

• summary dialog (and ICS import dialog)
• invitation overlays in emails
• tooltips shown when mousing over an item.

Depends on D88365

Comment 9

[Paul Morris [:pmorris]]

Going in a different direction. Rather than removing the HTML tags, sanitize and render descriptions as HTML. That way links are preserved and clickable. We need to do this in several places as noted in the commit message for the patch. Plain text URLs are turned into links (as we previously did for the invitation view in an email message). A small behavior change is that "file" urls are no longer linkified (see modified test), but that seems like a worthwhile trade off. I didn't see a way to prevent this from happening.

Comment 10

[Paul Morris [:pmorris]]

Comment on attachment 9172305 [details]
Bug 1659363 - Render HTML in calendar item descriptions. r=darktrojan

Richard, could you take a look at the CSS for theme-ability? Namely the white background of the description and the blue link color, which I think will need some adjustment.

Comment 12

[Richard Marti (:Paenglab)]

Comment on attachment 9172305 [details]
Bug 1659363 - Render HTML in calendar item descriptions. r=darktrojan

Looks good. Yes, white as background isn't so optimal with a dark theme. I'll attach a fixed CSS file you can then update on Phabricator.

Comment 13

[Richard Marti (:Paenglab)]

Attachment: calendar-event-dialog.css

Updated CSS file with better colours.

Comment 14

[Paul Morris [:pmorris]]

Thanks Richard! I've added your CSS changes to the latest revision of the patch.

Comment 15

[Paul Morris [:pmorris]]

Here's a try run. Failures look unrelated. https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=2680c40155e920f273e1e20a3ad18a7d39c31398&selectedTaskRun=NkkYLAvUSDqMCG6jOeTejg.0

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/f78b8f5ae8b1
Fix some typos in comments in chat code. r=clokep
https://hg.mozilla.org/comm-central/rev/d1bccebf5b42
Render HTML in calendar item descriptions. r=darktrojan

Comment 18

[TBirdFanLondon]

I tried a nightly build and I can't await the day of it appearing in the release channel!!!
THANKS!

Comment 19

[Magnus Melin [:mkmelin]]

We should take this for 78, but let's polish some more first (bug 1663947 at least).

Comment 21

[Sisim Biva]

Hello, I understand that these commits will allow clicking on links in the description of an event, right? It is indeed more and more useful with the increase of online meetings and link to Jitsi, Zoom, Teams, Google meet and other platforms... Really appreciated, thanks!
When do you plan to release it?

Comment 22

[Rob Lemley [:rjl]]

Temporary backout

Thunderbird 83.0b2:
https://hg.mozilla.org/releases/comm-beta/rev/7c2924f35b3e

Comment 23

[Rob Lemley [:rjl]]

NI :wsmwk since we need to remember to reapply this on beta once bug 1673164 is fixed.

Comment 24

[TBirdFanLondon]

Can this now be brought back, as bug1673164 appears as fixed

Comment 25

[Rob Lemley [:rjl]]

Yes, it's lined up for the next beta.

Restored in Thunderbird 83.0b3:
https://hg.mozilla.org/releases/comm-beta/rev/1bfa718a7cd3

Comment 28

[Igor R.]

We should take this for 78, but let's polish some more first (bug 1663947 at least).

Will this ever be brought into 78?

Comment 29

[Paul Kallnbach]

Hello. Thanks for working on this. I'm running 86.0b1 and unfortunately the commit above doesn't seem to work for Microsoft Teams invitations. Teams saves links in the DESCRIPTION field sourrounded by left and right angle brackets (<) and (>). The introduced 'textToHtmlDocumentFragment' function fails to recognize this (assumably because it thinks a html tag is about to start) and removes the link instead, which is worse than at least having the text version. If you remove those brackets manually, the link is recognized correctly.

Interestingly the main email window of Thunderbird already handles that scenario correctly. Whether your email contains <www.google.com> or www.google.com it always recognizes the link and just ignores the brackets.

...
DESCRIPTION;LANGUAGE=en-US:\n\n______________
__________________________________________________________________\nMicros
oft Teams-Meeting\nJoin on your computer or mobile app\nClick here\,
to join the meeting<https://teams.microsoft.com/l/meetup-j
oin/19%3ameeting_ODFmOGFmMGItZjA3OC66MGVjLTllMzItNDNmM2QxNzViOTg1%40thread
.v2/0?context=%7b%22Tid%22%3a%222f1bf2a7-730e-4228-82d5-440fbfcb419a%22%2c
%22Oid%22%3a%229a3157f6-b6ff-4f1b-894d-baf81555bfb0%22%7d>\n ...

Comment 30

[Magnus Melin [:mkmelin]]

The teams problem is bug 1666296.

Comment 31

[Paul Kallnbach]

Hi Magnus, yes I've seen that. But since this commit's function was to find and convert links in plain text and it doesn't do that in this corner case (that may or may not be Teams exclusive), I thought this would be the right place. I will report it in bug 1666296 too. Thanks.