Open Bug 1830941 Opened 2 years ago Updated 2 months ago

Filelink adds html part

Categories

(Thunderbird :: Add-Ons: Extensions API, defect)

Product:
Thunderbird
Component:
Add-Ons: Extensions API
Version:
Thunderbird 102
Type:
defect

Tracking

(Not tracked)

Status:
UNCONFIRMED

People

(Reporter: je, Unassigned)

References

Details

Steps to reproduce:

  1. Create a new text only message
  2. Add a FileLink attachment
  3. Send the message

Actual results:

Thunderbird adds an html attachment, containing the link that is already in the message body.

Users of *cloud reported that they cannot send messages, because their organization's outbound mail server blocks html mails.

The problem was introduced by the solution of bug 1670791

Expected results:

Thunderbird should not add parts to the message behind the user's back.

At least this unexpected and breaking behavior should be configurable.

because their organization's outbound mail server blocks html mails.

That's an highly unusual mail server :)

Component: Untriaged → FileLink

Changing the .html extension that is added to something else or omitting the added extension altogether makes sense to me but I vote strongly to keep the attachment file. As a user of *cloud, I think it is a strength and to me is not "behind the user's back".

The added zero length indicator file (to me) gives emails the attachment that allows emails with file payloads to be easily recognised when sorting emails (e.g. to re-send a file to another client). My clients in government departments also have commented on the same - ie the recipients need to sort their email to find the ones I have sent that have file payloads - be it a containerised link as provided by *cloud or a real attachment.

Looking at my emails the extension tag file is called filename.extension.html matching the filename and extension of the *cloud loaded file. The .html seems unnecessary if the tag can be named with the file... but perhaps it is there to defang bad actors?

(In reply to S.Brooks from comment #2)

Changing the .html extension that is added to something else or omitting the added extension altogether makes sense to me but I vote strongly to keep the attachment file. As a user of *cloud, I think it is a strength and to me is not "behind the user's back".

The added zero length indicator file (to me) gives emails the attachment that allows emails with file payloads to be easily recognised when sorting emails (e.g. to re-send a file to another client). My clients in government departments also have commented on the same - ie the recipients need to sort their email to find the ones I have sent that have file payloads - be it a containerised link as provided by *cloud or a real attachment.

Current versions of Thunderbird don't add a zero length indicator but an actual html part with the download link and some additional text.
Apart from the problems this seems to cause with some antivirus solutions, it adds text to my message that I can't read, check or edit before it is sent in my name. Before several users of *cloud complained I didn't even know it was there. What's worse: The text I didn't see or even know about is digitally signed as originating from me.

I don't argue against adding an indicator attachment (any longer ;-) ). But the current solution takes away control of the message I'm sending. I deeply believe that an email client shouldn't do that.

I will try if I can make it optional.

I can add the following. It doesn't matter what kind of message you create (HTML or only text). Anyway Thunderbird create HTML attachment containing the link that is already in the message body and if text in this attachment is non-ASCII it looks like this:

=undefined4=undefined0=undefined9=undefinedB and so on

Is there any news on this? I keep getting complaints from users whose messages with FileLinked attachments are rejected by receiving antispam systems because of the attachment.

Hi all, the "keep getting complaints" Johannes is refering to was amongst others from myself. :-)
This is the bug report I initially issued to him. I hope adding this information here helps determining the rootcause and solution.

This last week I sent a mail with a file of ~10MB attached via his filelink addon.
I have done this a few times this week, without any problems. (Thanks for this great add in, Johannes!!)

However this time I immediately got a "Undelivered Mail Returned to Sender" Error mail back from the amazon mail servers with the following error:
email-smtp.eu-west-1.amazonaws.com[34.242.32.157] said: 554 Transaction
failed: In parameter list <; filename=vog nieuw 2023 tijd 1600.pdf.html>,
expected ';', got "nieuw" (in reply to end of DATA command)

So from my point of view, it is not only the html that is added. But somehow it is also not html format that is accepted by amazonaws.com
Hope you or someone can fix this.

Kind regards,

Bert

@Bert: Thanks for that report. We will fix that ASAP in Bug 1858395.

See Also: → 1858395

Sorry, but bug 1858395 doesn't fix the problem. The place holder attachment is still a regular html file that causes rejection by common antivirus solution. I guess this is because the message contains a text part and an html part with different contents. This is considered an indicator of malicious content by many antispam and antivirus solutions.

In addition the German version of html file is broken as the head declares utf-8 encoding but the umlauts are actually encoded as ISO 8859 (in 115.4.3 (64-Bit) on Windows).

And it still bothers me that text is added to my mails that I can't control.

If reverting to the old type attachments isn't a solution in your eyes, I'd be very grateful if you could share the reasons. TIA

Depends on: 1865269

I will fix the encoding issue ASAP in Bug 1865269.

I will add an option in this cycle (but probably no backport) to not include an html file and/or use a txt file instead.

The html attachment is not added as an alternate part, but as an attachment, so its content differing from the body should not/must not matter.

The reason the attachment was added:

  • it was requested by users to not include a broken non-existing attachment

For me, the problem is the naming of the attachment: If I transmit a file named 'document.pdf' with *cloud, it leads to an attachment named 'document.pdf.html'.

I tested *cloud with several addressees but most of my mails were rejected as potential malware because of the double file extension .pdf.html.
It works if I remove the pdf-extension before I transfer the file (attachment name: document.html).

Why don't you just call the attachment 'downloadlink.html' independent from the file which is transferred?

Btw: I agree with Johannes Endres that it is no good style to alter the content of my message after I push the send button.

I found this report through Johannes Endres. I took the time to read through it and also the linked reports.

I must admit I do not like the way Filelink are handled with this additional html file per attachment.

rspamd btw is complaining about this clearly and gives it a spam rating:

MIME_ARCHIVE_IN_ARCHIVE (5) [.zip.html]
MIME_DOUBLE_BAD_EXTENSION (3) [.zip.html]

I would vote for an option which at least gives the user an option to disable this behaviour of adding html attachments, or actually better to let user enable this "feature" if they really need it.

E-Mail attachments are bad idea anyway and Filelink gives the users finally the option to not attach files to emails and now there is this, pardon my French, garbage attachment.

I finally was able to "teach" my users (old white man) to avoid sending attachments, but now I have to tell them, it is safer to send the attachment or upload the file manually before composing the mail and put the necessary information in the composed email instead of using this in theory far superior way of Filelink.

I would strongly urge for an optional solution, because I prefer a rather good spam detection over the mess this attached html file creates.

One option would be to remove the original file extension (still nothing I would vote for, but at least a compromise to lower the chance of false positive spam detection).

Regarding this spam detection, why I could change the spam filters I administrate, I can certainly NOT do this with all the mail servers. Which brings me back to Postels Law: "an implementation should be conservative in its sending behavior, and liberal in its receiving behavior" (RFC 760).

So please reconsider this whole behaviour of HTML attachments or make it at least configurable.

Thank you for taking time reading this message.

All the best

I tried this feature today with nextcloud.
(https://addons.thunderbird.net/fr/thunderbird/addon/filelink-nextcloud-owncloud/)
It almost works as expected, I mean :

  • file is uploaded
  • attachment text is created
  • clicking on link allow to download the page.

But there is a huge issue, I tested to send this emails to 3 different mail provider (ovh, orange, gmail) and none of them received an email...
Pretty sure they was blocked by antispam.
So for now, I consider the feature useless.

Note that current html attachment text sounds not so good. Ideally the default one should at least be accepted by antispam software but also I think it should be possible to customize attachment html content (possibility to add custom html template to change appearance).

Component: FileLink → Add-Ons: Extensions API

Thanks for your report. We are trying to find a solution for this situation.

There is a constraint that the behavior must be consistent, we try to avoid having some filelinks with and some without real attachments (based on the preference of the sender). Getting messages with filelinks filtered out by antivirus systems has to be avoided as well, of course.

I currently see two issues:

  1. *.zip.html is flagged as some sort of spoofing
  2. HTML attachments are considered evil

I have no idea how MIME_ARCHIVE_IN_ARCHIVE could have been triggered. This is just plain wrong, there simply is no zip archive attached.

Issue #1

The name of the original attachment must be included, to be able to find that email when searched for the attachment name. Instead of original.zip.html, would [original.zip] filelink.html work?

Issue #2

The attachment could be changed to text/plain (this should be a global and not a per-filelink-provider decision). However, I think that reduces the usefulness of the attachment. Is there a general agreement, that a text/plain attachment is to be preferred?

(In reply to John Bieling (:TbSync) from comment #14)

Issue #1

The name of the original attachment must be included, to be able to find that email when searched for the attachment name. Instead of original.zip.html, would [original.zip] filelink.html work?

At the risk of sounding ignorant, but why would the attachment name be required?

I am already sometimes receiving Emails referencing cloud providers / corporate cloud. Usually this happens by either directly sending an Email from the cloud provider / corporate cloud interface, and these Emails contain only a formatted link to the file.

Sometimes it happens by someone just pasting the download link.

In no case does it include any form of attachment.

There is a constraint that the behavior must be consistent, we try to avoid having some filelinks with and some without real attachments (based on the preference of the sender). Getting messages with filelinks filtered out by antivirus systems has to be avoided as well, of course.

I currently see two issues:

  1. *.zip.html is flagged as some sort of spoofing
  2. HTML attachments are considered evil

Avoiding filtering / the link not working seems to be basically impossible. I have found that some companies / public offices do not allow e.g. opening links to Google Drive. So even if the attachment works, recipients may not be able, either by personal inability or limitations set by their IT, to open the cloud link.

Without and attachment (with name) we can't easily know the message has, essentially, an attachment.

I find many times you also do not actually want the notification into the message body. YMMV.

FileLinks are actually ideal for referencing files in newsgroups, as attachments are frowned upon there.

In fact, all forms of MIME parts are frowned upon. Most servers already refuse to accept such articles. This means that the current form of FileLinks is unfortunately unusable for newsgroups.

Would it be complicated to make the pseudo-attachment optional?

Or we only use it for HTML or mixed messages and leave it out for plain text messages.

(In reply to Alfred Peters [:infofrommozilla] from comment #17)

Would it be complicated to make the pseudo-attachment optional?

I really want to avoid that. We need consistency. Imagine user John having two friends, Jim and Bob. Jim disabled the attachment. Both Jim and Bob send messages with filelinks to John, who is experiencing an inconsistent behavior regarding the attachments.

Most servers already refuse to accept such article

Could you name one or two for me to try out?

(In reply to John Bieling (:TbSync) from comment #18)

(In reply to Alfred Peters [:infofrommozilla] from comment #17)

Would it be complicated to make the pseudo-attachment optional?

I really want to avoid that. We need consistency. Imagine user John having two friends, Jim and Bob. Jim disabled the attachment. Both Jim and Bob send messages with filelinks to John, who is experiencing an inconsistent behavior regarding the attachments.

This assumes that both Jim and Bob use Thunderbird. Realistically, John is getting Emails from a lot of people. There will be a majority, that sends just plain attachments. There will be emails from doctors or banks that just say "there is a new document, due to data protection laws go to our website manually and download it with your account". There will be people sending links to Google Drive or Dropbox. In most cases, John won't know, nor care, from which Email client any of that was sent. Even formatting will vary wildly, with some people writing in 10pt, some in 20pt, some in black, some in blue, some in dark gray comic sans on a black background. Email clients will force their own style-sheets on Emails, jumbling the formatting of quoted messages.

Forget about consistency between Thunderbird installations of different people. When it comes to Email, consistency fails on many levels. Some of them come from different Email clients being used, some of them come from people using Emails differently.

Also, when looking beyond Thunderbird, sending a link without an attachment is the usual way to do it, so dropping the link-only attachments would be more consistent in a sense.

Link only is not an attachment. My clients are government people that have 1000's of emails in their inbox and find the reports I send them by sorting their emails to those with/without attachments. The html stub is problematic, but removing it altogether kills the ability to filter emails to find those with "reports attached" i.e. big reports linked by filelink. Attachments are still very much the norm and will be until outlook has a flag for "this email includes a linked document"... which seems unlikely.

Hi,

I am glad this issue is back in the discussion loop.

What I do not understand here: This extension allows me to send an email to avoid attachments, but attaches something so some users can filter their inbox for emails with attachments? I do not like to be the devils advocate, but this sounds really weird.

What does work, I tested it, by crafting an email, using myfile.html instead of myfile.XXX.html where XXX is the original fileending.

If this is a solution, I propose replacing the original fileending by .html.

The cleaner solution from my perspective is: Do not add any replacement-pseudo-attachment at all, since I am using this extension to avoid sending emails with an attachment greater than the specified size. The receiving side is usually able to use the built-in search.

All the best and many thanks for taking time looking into this.

(In reply to John Bieling (:TbSync) from comment #18)

(In reply to Alfred Peters [:infofrommozilla] from comment #17)

Most servers already refuse to accept such article

Could you name one or two for me to try out?

<news.individual.de> accepts it: <news:lofcspFjgsdU1@mid.individual.net>

<news.eternal-september.org> rejects it: 441 437 Binary: misplaced binary
<news.solani.org> rejects it: 441 437 HTML file attachment
<news.tota-refugium.de> rejects it: 441 437 HTML file attachment

(In reply to Alfred Peters [:infofrommozilla] from comment #22)

<news.solani.org> rejects it: 441 437 HTML file attachment

Solani accepts it as text/plain attachment: news:vftvvi$1kh0a$1@solani.org

<news.tota-refugium.de> rejects it: 441 437 HTML file attachment

Tota-Refugium accepts it as text/plain attachment: news:vfu05l$1kh67$1@tota-refugium.de

Only <news.eternal-september.org> also rejects the text attachment: 441 437 Binary: misplaced binary

Please! make it optional.

The reason we can't agree on this is a little philosophical. It's in our different ideas what FileLink does:

In my mind Thunderbird reminds me that sending large attachments is a terrible idea and helps me not to send an attachment at all but send a link instead. So in my mind I decide not to send an attachment. My message will not refer to an "attachment" but to a "link". If you see it that way, it's an inconsistency for the receiver if the message seems to have an attachment as I didn't intended it to send one (after FileLink reminded me). If you see it that way, the attachment makes no sense and shouldn't be there.

In your mind you are still sending an attachment, just that the contents is not in the message. If you see it that way, the attachment makes sense.

I'd be very grateful if Thunderbird would allow us to choose which of these views we share.

Ceterum censeo: I sign my messages with GPG and hence certify that I typed every word in signed part of the message and can be held accountable for it. I think that adding an attachment behind my back breaks the whole idea of signing messages.

I can not stress this enough, but I completely agree with Johannes Endres. Please make it optional. Or at least point us to the source so we can try to create a patch on our own!

Thank you

Ceterum censeo: I sign my messages with GPG and hence certify that I typed every word in signed part of the message and can be held accountable for it. I think that adding an attachment behind my back breaks the whole idea of signing messages.

Sorry, I should have been more precise here: Adding text that I didn't intend to send and didn't see before sending it, breaks the whole idea of signing messages. (This refers to the text in the placeholder attachment.)

As for the anti spam issue, a user writes:

For a few months now, however, I've hardly been able to use it [a FileLink AddIn] because > the e-mails I send to most of my contacts are returned to me.

I finally contacted my anti-spam provider: SpamExperts. They say that the rule that blocks e-mails looks for attachments with a double extension (which is the case for pdf.html ). They say that putting two extensions to a file is a common practice to hide malware.

(In reply to Johannes Endres from comment #27)

As for the anti spam issue, a user writes:

For a few months now, however, I've hardly been able to use it [a FileLink AddIn] because > the e-mails I send to most of my contacts are returned to me.

I finally contacted my anti-spam provider: SpamExperts. They say that the rule that blocks e-mails looks for attachments with a double extension (which is the case for pdf.html ). They say that putting two extensions to a file is a common practice to hide malware.

We will fix that shortly.

I would like to share the results of my exploration efforts, to find a solution:
https://docs.google.com/document/d/1oQ06lV8NWU2Q7YEQclZg7s26MotpLwqo88KXzMWb5_k/edit?usp=sharing

I posted a test message to alt.de.test via solani and eternalseptember and both went through.

We will redesign the metadata, which is added as the inline attachment (feel free to make suggestions). If the message is created as plain text, then the inline attachment will be plain text as well, otherwise html. And we need to find a way to show the user the metadata in the composer without actually adding it to the message body.

Please first fully read the linked google doc, before commenting.

(In reply to John Bieling (:TbSync) from comment #29)

I would like to share the results of my exploration efforts, to find a solution

Since no one else is commenting, I would like to say on behalf of users of my FileLink add-on that this is a good solution that should address most of the concerns mentioned above. Thank you John for doing all of the research!

Based on the provided screenshots, I also liked that you reduced the heaviness of the generated HTML by removing the borders. Anything you could do to further minimize/reduce it would be great.

By metadata, I am assuming you are referring to the info added to the CloudFileTemplateInfo object. If you are interested in adding metadata, see bug 1854939. If we no longer add this info directly to the message, it could make features like bug 1817835 easier to implement, which would be an additional improvement from the proposal.

From the Google Doc:

CloudFiles do not necessarily link to the actual file, but to a web interface where the file can be downloaded. Thunderbird can therefore not download the attachment on behalf of the user. The only thing Thunderbird can offer is to open the link. We cannot offer to save it.

If there were interest in this functionality, maybe something like a new onFileDownload event could be added to the cloudFile API. If the user had the correct FileLink provider add-on installed, it could be passed the URL and then automatically download the file on behalf of them. Most FileLink provider add-ons should already have all of the necessary credentials to do this and it could be convenient for users to then be able to download their files directly in TB.

(In reply to John Bieling (:TbSync) from comment #29)

I would like to share the results of my exploration efforts, to find a solution:

I like the result.

We will redesign the metadata, which is added as the inline attachment (feel free to make suggestions).

Although off-topic, would it be possible to add a checksum in the metadata?

While No longer include the used service is good for me too, would it also be possible to have at the end of the metadata the option of adding a standard sentence containing general information (such as a legal or technical comment, for example). This possibility could be linked to the use of either Filelink or a particular Filelink provider.

Thanks a lot.

This is GREAT John - thank you! I'd love to get this implemented so please reach out to the team if you would like design input on the metadata.

Hi John,

thanks for your effort. I read through your document and would like to see this implemented as it would solve the SPAM detection issues on my end, users are suffering when sharing via cloudfile.

If you need testers, I am happy to help with that.

Best and thanks a lot

The proposed changes look good to me and are a sensible balance.

I do have one concern with:

No longer include the used service, to reduce user confusion (they are on the receiving end and may not be familiar with Thunderbird and the different cloudFile providers)

I disagree because I do think additional text gives the user a "heads-up" on the used service other than just the URL in the link is useful. It reduces the "surprise factor" with respect to the file origin. In my case, I use Amazon S3 as my cloud service specifically because my government clients are blocked from most other cloud services. Having the "S3" logo and text displayed gives the recipient some added confidence they will be able to download the files and when their corporate systems flag the external link they are already expecting it to be S3.

Taking the screenshot in the Google-Doc at face value - having a generic "Thunderbird Send" text is not helpful at all and as it differs from the URL its somewhat confusing and tilts the balance of trust in the wrong direction. Many recipients are live long corporate Outlook users and some would not know what Thunderbird was. At best "Thunderbird Send" is not informative and at worst its confusing.

As suggested by @bugzilla.24456 above - I would prefer a user defined text string to display where the "Thunderbird Send" appears - in my case I would replace with a longer informative message "Linked files hosted by XX available for YY days" ... or something.

(In reply to S.Brooks from comment #34)

Taking the screenshot in the Google-Doc at face value - having a generic "Thunderbird Send" text is not helpful at all and as it differs from the URL its somewhat confusing and tilts the balance of trust in the wrong direction.

Just to clear this up, the "Thunderbird Send" text came from using my FileLink provider for Send add-on and was likely not meant by John to be generic. As you may know, the Send service was formally called "Firefox Send" before being discontinued by Mozilla. I call it "Thunderbird Send" in my extension to avoid confusion with the Send button in the compose window and other uses of the term "send". There are over two dozen (and counting) public Send service instances and many private ones, so the URL is frequently different depending on which instance(s) the user configured the extension to use.

(In reply to S.Brooks from comment #34)

I disagree because I do think additional text gives the user a "heads-up" on the used service other than just the URL in the link is useful. It reduces the "surprise factor" with respect to the file origin. In my case, I use Amazon S3 as my cloud service specifically because my government clients are blocked from most other cloud services. Having the "S3" logo and text displayed gives the recipient some added confidence they will be able to download the files and when their corporate systems flag the external link they are already expecting it to be S3.

Noted. The reasoning here was to remove icons from the HTML inline attachment, which could trigger anti-spam mechanism. We will see if we can keep it, or find different (text only) solutions

(In reply to John Bieling (:TbSync) from comment #36)

(In reply to S.Brooks from comment #34)

I disagree because I do think additional text gives the user a "heads-up" on the used service other than just the URL in the link is useful. It reduces the "surprise factor" with respect to the file origin. In my case, I use Amazon S3 as my cloud service specifically because my government clients are blocked from most other cloud services. Having the "S3" logo and text displayed gives the recipient some added confidence they will be able to download the files and when their corporate systems flag the external link they are already expecting it to be S3.

Noted. The reasoning here was to remove icons from the HTML inline attachment, which could trigger anti-spam mechanism. We will see if we can keep it, or find different (text only) solutions

I believe, if we put the images inline and do not load them from an external resource but use a base64 encoded logo it shouldn't be a problem.

e.g.

<img src="data:image/png;base64,$SomeBase64EncodedImage" alt="Logo of $Provider" title="$SomeFancyTitle">

This way it should also be possible to skip this for a plaintext version

Best

I believe, if we put the images inline and do not load them from an external resource but use a base64 encoded logo it shouldn't be a problem.

That is how we do it now. We will check if that can be kept once the main part of the new implementation is done.

All very logical explanations re: service string. Works for me. thumbsup

You need to log in before you can comment on or make changes to this bug.