1427920 - S/MIME signed messages from Outlook don't show embedded images due to incorrect MIME structure (comment #7), was: SMIME signature prevents displaying company logo in "signature-part" of e-mail when received in Thunderbird

Status: RESOLVED INVALID

(Thunderbird :: Untriaged, defect)

Description

[Lutz J. Schmid]

Attachment: The zip-file "SMime-Certificate_Company-logo-problem-in-Thunderbird.zip" contains the two screenshots as mentioned in the texts above.

User Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:57.0) Gecko/20100101 Firefox/57.0
Build ID: 20171226083017

Steps to reproduce:

The bug in Thunderbird involves two types of signature. I add 2 screenshots which show the difference between 2 received e-mails. Both e-mails are sent by the same user using same sender and receiver e-mail addresses. One type of signature (as it is called in the sender's configuration of the e-mail client for the used e-mail account actually means a kind of e-mail addendum. This addendum is automatically added to the end of the e-mail body and it contains text (company information) and a company logo which is a png-file. The difference between the two e-mails is given by a S/MIME signature that is used to digitally sign one of the 2 e-mails with a X.509 certificate. The signed e-mail has the envelope icon with a red dot next to the e-mail time. The certificate is issued by GlobalSign CA. The sender uses MS Outlook on iOS. 

The problem seems to be somehow similar to bug 1238914, but bug 1238914 does not correlate to S/MIME certificates.


Allgemeine Informationen

    Name: Thunderbird
    Version: 52.5.2
    User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.5.2
    Profilordner: Ordner öffnen

              (Lokaler Datenträger)
    Build-ID der Anwendung: 20171221110448
    Aktivierte Plugins: about:plugins
    Build-Konfiguration: about:buildconfig
    Speicherverwendung: about:memory
    Profile: about:profiles

  E-Mail- und Newsgruppen-Konten
    removed

  Absturzberichte

  Erweiterungen
    Lightning, 5.4.5.2, true, {e2fda1a4-762b-4020-b5ad-a41df1933103}
    Quick Locale Switcher 2, 0.0.1, true, [e-mail]

  Wichtige modifizierte Einstellungen

    Name: Wert

      accessibility.typeaheadfind.flashBar: 0
      browser.cache.disk.capacity: 358400
      browser.cache.disk.filesystem_reported: 1
      browser.cache.disk.smart_size_cached_value: 358400
      browser.cache.disk.smart_size.first_run: false
      browser.cache.disk.smart_size.use_old_max: false
      dom.apps.reset-permissions: true
      extensions.lastAppVersion: 52.5.2
      font.internaluseonly.changed: true
      font.name.monospace.el: Consolas
      font.name.monospace.tr: Consolas
      font.name.monospace.x-baltic: Consolas
      font.name.monospace.x-central-euro: Consolas
      font.name.monospace.x-cyrillic: Consolas
      font.name.monospace.x-unicode: Consolas
      font.name.monospace.x-western: Consolas
      font.name.sans-serif.el: Calibri
      font.name.sans-serif.tr: Calibri
      font.name.sans-serif.x-baltic: Calibri
      font.name.sans-serif.x-central-euro: Calibri
      font.name.sans-serif.x-cyrillic: Calibri
      font.name.sans-serif.x-unicode: Calibri
      font.name.sans-serif.x-western: Calibri
      font.name.serif.el: Cambria
      font.name.serif.tr: Cambria
      font.name.serif.x-baltic: Cambria
      font.name.serif.x-central-euro: Cambria
      font.name.serif.x-cyrillic: Cambria
      font.name.serif.x-unicode: Cambria
      font.name.serif.x-western: Cambria
      font.size.fixed.el: 14
      font.size.fixed.tr: 14
      font.size.fixed.x-baltic: 14
      font.size.fixed.x-central-euro: 14
      font.size.fixed.x-cyrillic: 14
      font.size.fixed.x-unicode: 14
      font.size.fixed.x-western: 14
      font.size.variable.el: 17
      font.size.variable.tr: 17
      font.size.variable.x-baltic: 17
      font.size.variable.x-central-euro: 17
      font.size.variable.x-cyrillic: 17
      font.size.variable.x-unicode: 17
      font.size.variable.x-western: 17
      gfx.blacklist.suggested-driver-version: 10.6
      gfx.direct3d.last_used_feature_level_idx: 0
      mail.openMessageBehavior.version: 1
      mail.winsearch.firstRunDone: true
      mailnews.database.global.datastore.id: bc1fbe7a-01e0-4f0c-b76d-832d0180fd5
      mailnews.database.global.views.conversation.columns: {"threadCol":{"visible":true,"ordinal":"1"},"flaggedCol":{"visible":true,"ordinal":"3"},"attachmentCol":{"visible":false…
      mailnews.database.global.views.global.columns: {"threadCol":{"visible":true,"ordinal":"1"},"flaggedCol":{"visible":true,"ordinal":"3"},"attachmentCol":{"visible":false…
      media.gmp.storage.version.observed: 1
      network.cookie.prefsMigrated: true
      network.predictor.cleaned-up: true
      places.database.lastMaintenance: 1514244647
      places.history.expiration.transient_current_max_pages: 122334
      plugin.importedState: true
      plugin.state.flash: 0
      print.printer_xxxxxx removed
      security.disable_button.openCertManager: false
      security.disable_button.openDeviceManager: false
      security.sandbox.content.tempDirSuffix: {7634a97d-c22e-498d-9a04-a0dc07c39ced}

  Grafik

      GPU #1
      Beschreibung: AMD Radeon R9 200 Series
      Herstellerkennung: 0x1002
      Gerätekennung: 0x6810
      RAM: 2048
      Treiber: aticfx64 aticfx64 aticfx64 aticfx32 aticfx32 aticfx32 atiumd64 atidxx64 atidxx64 atiumdag atidxx32 atidxx32 atiumdva atiumd6a atitmm64
      Treiber-Version: 13.251.9001.1001
      Treiber-Datum: 7-4-2014

      Features
      Direct2D: false
      DirectWrite: true (6.3.9600.18696)
      ClearType-Parameter: DISPLAY1 [ Gamma: 2,2 Pixel Structure: RGB ] DISPLAY2 [ Gamma: 2,2 Pixel Structure: RGB ]
      WebGL-Renderer: Google Inc. -- ANGLE (AMD Radeon R9 200 Series Direct3D9Ex vs_3_0 ps_3_0) -- OpenGL ES 2.0 (ANGLE 2.1.0.2a250c8a0e15)

      AzureCanvasBackend: skia
      AzureCanvasAccelerated: 0
      AzureFallbackCanvasBackend: cairo
      AzureContentBackend: skia

  JavaScript

  Inkrementelle GC: 1

  Barrierefreiheit

    Aktiviert: 0
    Barrierefreiheit verhindern: 0

  Bibliotheken-Versionen

      Minimal vorausgesetzte Version
      Verwendete Version

      NSPR
      4.13.1
      4.13.1

      NSS
      3.28.6
      3.28.6

      NSS Util
      3.28.6
      3.28.6

      NSS SSL
      3.28.6
      3.28.6

      NSS S/MIME
      3.28.6
      3.28.6





Actual results:

The e-mail which is S/MIME signed does not show the company logo in the correct way. The screenshot "SMime-Certificate_Attached-Signature-image-with-SMime-Signature-wrong.png" shows, that the company logo is added as a normal attachment and the position within the e-mail, where the logo should be displayed, is indicated by an empty frame.



Expected results:

In the e-mail, which is not digitally signed, the company logo is positioned at the right place as it is configured in the "signature" settings of the sender's e-mail client. The screenshot "SMime-Certificate_Signature-image-without-SMime-Signature-ok.png" shows the e-mail as it should be. 

The S/MIME signed e-mail is displayed correctly when using MS Outlook (on receiver's side). This could be proofed by sending the e-mail to a third person by the original sender. The e-mail as shown in screenshot "SMime-Certificate_Attached-Signature-image-with-SMime-Signature-wrong.png" (as received wrongly by Thunderbird) was used again for this test.

Comment 1

[Jorg K (CEST = GMT+2)]

(Please don't paste support information into comments, since it makes the bug hard to follow).

When I used TB to create a HTML mail with and embedded image and send it, the sent e-mail looks fine, the image is displayed where it should be displayed.

The MIME structure of the message is:
multipart/signed
  multipart/related
    text/html
    image/png
  application/pkcs7-signature

So basically a multipart message with two levels, the inner level is a multipart/related with text and image.

You said the faulty e-mail originates from an Outlook sender. So I assume that Outlook produces invalid MIME structures. Can you attach the full message (save it/drag it to the desktop or a folder) using "Attach File" above. Remove private information with a text editor. Alternatively, paste the message and part headers here.

We know that there are many mail clients around which produce invalid MIME structures, see bug 1405234 and especially bug 1362539.

If my assessment is right, this bug should be associated with bug 1405234 and the subject changed to
... - Make TB more tolerant to incorrect MIME structure: Bad multipart/signed with multipart/related from Outlook on iOS.

Comment 2

[Lutz J. Schmid]

Attachment: The file "Source-code_e-mail-t1_SMIME-signed_company-logo_201801092.txt" contains the source code of the SMIME-signed e-mail.

Comment 3

[Lutz J. Schmid]

Thank you for your activity. 

The sender uses Outlook 15.32 on iOS. 
The source text of the e-mail is added to file "Source-code_e-mail-t1_SMIME-signed_company-logo_201801092.txt". 
The company logo has a size of 49641 Bytes.

Comment 4

[Jorg K (CEST = GMT+2)]

Thanks for the attached data.

Referring to https://www.secureblackbox.com/kb/help/ref_howto_mime_smime_analyze.html we see that there are two formats for signed messages, "clear format" (1) and "enveloped format" (2). It looks like TB is produces "clear format", see comment #1, and the attached message uses "enveloped format" since it's just one big monolithic part:

Content-Disposition: attachment; filename="smime.p7m"
Content-Transfer-Encoding: base64
MIME-Version: 1.0

MIMBNMQGCSqGSIb3DQEHAqCDATS0MIMBNK8CAQExDzANBglghkgBZQMEAgEFADCDASTjBgkqhkiG
9w0BBwGggwEk0wSDASTOQ29udGVudC10eXBlOiBtdWx0aXBhcnQvbWl4ZWQ7DQoJYm91bmRhcnk9
IkJfMzU5Nzc1NDE1OF81NDQwMDk1OTgiDQoNCj4gVGhpcyBtZXNzYWdlIGlzIGluIE1JTUUgZm9y
...

As per the link above, when you decode that, you'll get various parts. TB can obviously decode the message, but it results in a broken image, most likely to a bad structure after decoding. I'll have to do a little research into that to see how I can get to those parts.

Comment 5

[Jorg K (CEST = GMT+2)]

Oops, forgot to paste the type:

Content-Type: application/pkcs7-mime; smime-type=signed-data;
	name="smime.p7m"
Content-Disposition: attachment; filename="smime.p7m"
Content-Transfer-Encoding: base64

Comment 6

[Jorg K (CEST = GMT+2)]

Attachment: Result of processing the message part with openssl - pkcs7.txt

I cut the signed data out of the message into a file message.txt containing only

Content-Type: application/pkcs7-mime; smime-type=signed-data;
	name="smime.p7m"
Content-Disposition: attachment; filename="smime.p7m"
Content-Transfer-Encoding: base64
MIME-Version: 1.0

MIMBNMQGCSqGSIb3DQEHAqCDATS0MIMBNK8CAQExDzANBglghkgBZQMEAgEFADCDASTjBgkqhkiG
9w0BBwGggwEk0wSDASTOQ29udGVudC10eXBlOiBtdWx0aXBhcnQvbWl4ZWQ7DQoJYm91bmRhcnk9
...[snip]...
V1WgZ3xmuRGmmulrBWVmv9JX4Vq22D9LWBHBEnH7QtRmG+uwVLPpyeqM0o+gcbYuBNEsDb3Izbwz
mHvF6dOyi4pfXaytZHBZVLgdaVLj3NQ24QZuHOGKMiQGZDAU6ypWXqs6zDXvJcY=

This file I processed with openssl (on Windows):
C:\xampp\apache\bin\openssl smime -pk7out -in message.txt -out pkcs7.txt

pkcs7.txt is attached here.

Comment 7

[Jorg K (CEST = GMT+2)]

Attachment: result of processing pkcs7.txt with openssl asn1parse

I processed pkcs7.txt with
C:\xampp\apache\bin\openssl asn1parse -in pkcs7.txt -out msgout.eml

In an editor I removed the binary data from msgout.eml and added header information, the result is attached here.

Both the original message and the extracted message (without signature) look the same.

Sadly the message has a MIME structure that doesn't allow the image to be displayed inline. The structure is

multipart/mixed
  multipart/alternative
    text/plain
    text/hmtl
  image (attachment! not embedded)

So the internal MIME structure is wrong. May the sender complain to Microsoft.

Comment 8

[Jorg K (CEST = GMT+2)]

I'm really sorry, but there is nothing I can do here. The received message has the image as attachment, and TB shows it as such. For embedded images multipart/related must absolutely be used.

We have a meta-bug 1405234 to make TB more robust regarding bad MIME structures. This bug here doesn't fall into that category. The input data is plain wrong, thanks Micro$oft, so there is nothing we can do to bend this back into shape.

Just for fun, download msgout.eml, open it in a text editor and change multipart/mixed to multipart/related. The message will then be correctly displayed when imported into TB (dragged into a folder).

If you can, let the sender file a bug with Microsoft, or get them to switch to Thunderbird.

Comment 9

[Lutz J. Schmid]

Thank you for your research and your answer. 

I don't think that it is such easy as just saying "... let the sender file a bug with Microsoft, or get them to switch to Thunderbird." Fact is, that several e-mail clients display the embedded image in a correct way. However, there are only a few clients which support SMIME. 

You wrote that the structure is as follows:

multipart/mixed
  multipart/alternative
    text/plain
    text/hmtl
  image (attachment! not embedded)

This means that within a "multipart/mixed" structure there is a "multipart/alternative" structure that contains text and html. The html-part (which is the one to be displayed) contains the text and an embedded image within the text. This is why Thunderbird displays an empty frame at the correct position. I found "https://stackoverflow.com/questions/6706891/embedding-image-in-html-email#" to describe exactly what MS Outlook is coding. I do not see why the image has to be treated as an attachment in case that the reference of the image within the html-part is correctly referring to the image data in the "multipart/mixed" structure. This is obviously true, because the image is named 'image001.png' which equals to the ID in the empty frame. 

Thus MS Outlook is not coding in the most obvious way, but it looks not to be forbidden what this client is doing. However, reading and understanding the associated RFCs is a lot of work.

Comment 10

[Jorg K (CEST = GMT+2)]

I'm not surprised Outlook can interpret the mess it creates. Which other e-mail clients display the embedded image in a correct way?

If you want to relate two MIME parts, they need to sit in the same multipart/related structure:
https://tools.ietf.org/html/rfc2112

multipart/mixed is for attachments:
https://www.w3.org/Protocols/rfc1341/7_2_Multipart.html
The primary subtype for multipart, "mixed", is intended for use when the body parts are *independent* and intended to be displayed serially.

https://tools.ietf.org/html/rfc2046#section-5.1.3
The "mixed" subtype of "multipart" is intended for use when the body
parts are independent and need to be bundled in a particular order.

Since Outlook is very well able to send valid e-mails, those which are unsigned, I don't see much point to bend over backwards to compensate for a clear infringement of internationally recognised standards on behalf of Microsoft.

(In reply to Lutz J. Schmid from comment #9)
> I do not see why the image 
> has to be treated as an attachment in case that the reference of the image
> within the html-part is correctly referring to the image data in the
> "multipart/mixed" structure.
That's what standards are for. Otherwise you create unmaintainable mess of special casing to cater for all sorts of possible infringements of standards.

I have personally collected a few such infringements in bug 1405234, but as I said in comment #8, I wouldn't add this bug to the list of desirable enhancements.

Comment 11

[Lutz J. Schmid]

Thank you for the further replies. 
Actually I have seen mail clients that display the image in the correct way but not recognizing the digital signature. The "web.de" browser-based mail client displays text and logo in a correct way, but does not recognize the signature. It displays an attachment called 'smime.p7s', while displaying a special icon when a valid digital signature is identified. The same behavior is seen on the standard e-mail client on the Android OS. However, this e-mail client never recognizes digital signatures at all and only shows the unknown attachment 'smime.p7s'. The correct behavior of MS Outlook was tested with a friend and I also think that the above given e-mail, when sent from an iPhone, is correctly displayed on the iMac e-mail client. Yet I have to ask for approval regarding this Apple behavior.