466892 - Bad (Gmail) IMAP reading HTML digitally signed (S/MIME) mails with attachments

Status: RESOLVED WORKSFORME

(Thunderbird :: Security, defect)

Description

[Hooonza]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; cs; rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4
Build Identifier: Mozilla Thunderbird v. 2.0.0.18 (20081105)

A S/MIME digitally signed HTML message with an attachment sent with Gmail is saved to Sent mails of Gmail by Thunderbird. When I check the message using IMAP then TB reports corrupted digital signature. But when the same sent message is checked by pop3 by TB again (by TB portable) then the signature is correct so the message saved at the gmail server is correct very probably. I can compare source code of these 2 views of the message: there are different some header data (what shouldn´t affect the digital signature) but there are some new empty lines in the source code displayed through IMAP.

Reproducible: Always

Steps to Reproduce:
Happens every time - probably. I cannot be 100% sure but it happens very often.

1. Send a S/MIME digitally signed HTML message with an attachment
2 [review]. Check it with IMAP and with POP3
Actual Results:  
Message displayed with the IMAP has corrupted digital siganture. The message contains some new empty lines which destroy the integrity of the digital signature.

Expected Results:  
The message should be displayed correctly with no changes so that the digital signature check passes with no errors.

You can be muddled if you get the message about bad sent mails digital signature and you unnecessarily doubt about security matter.

Comment 1

[Hooonza]

I have 2 sent mails with corrupted S/MIME digital signature before me as I describe above where I can check the different source codes: 1 of them has new empty line No. 357 in the body of the message and 2nd of them has new empty lines No. 1701 and 8807 in the body of the message. It seems the bug may depend on size of the mail or on size of the attachment?

Comment 2

[Hooonza]

The bug verified again today. A sent HTML mail with attachments (with Gmail) has displayed a bad signature with TB v. 2.0.0.19 (IMAP). But I can save the message body from Gmail web acces as an *.eml file. This opened with MSOE has a GOOD signature. 

Where is the bug? Is it a Gmail IMAP bug or a TB bug?

Comment 3

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #2)
> Is it a Gmail IMAP bug or a TB bug?

Depends on culprit who added some new empty lines. Get IMAP log. (See Bug 402793 Comment #1). If culprit is Tb, and if log analysis by developers will be required, attach log file to this bug(never paste, please).

Comment 4

[Magnus Melin [:mkmelin]]

Also try a nightly build...

Comment 5

[Ludovic Hirlimann [:Usul]]

As magnus noted try a nigthly builds and post the results here please. And could you log both the imap and pop3 protocols as described here : https://wiki.mozilla.org/MailNews:Logging and attach both logs to this bug if you can reproduce it with the nigthlies.
Thanks in advance.

Comment 6

[Hooonza]

OK, I´ll get the logs after I see the bug again. Testing it (with the Portable TB 2.0.0.19) I cannot reproduce the bug.

Comment 7

[Ludovic Hirlimann [:Usul]]

And if you don't have the issue anymore lets us know too so we can close the bug.

Comment 8

[Hooonza]

Attachment: IMAP log

BAT file I have used to made it:

set NSPR_LOG_MODULES=imap:5
set NSPR_LOG_FILE=f:\imap.log
"C:\Program Files\Mozilla Thunderbird\thunderbird.exe"

Comment 9

[Hooonza]

Attachment: POP3 log

BAT file I have used to made it:

set NSPR_LOG_MODULES=pop3:5
set NSPR_LOG_FILE=f:\pop3.log
"H:\PortableApps\ThunderbirdPortable\ThunderbirdPortable.exe"

Comment 10

[Hooonza]

I see another email with a bad signature so I am sending log files...

Comment 11

[Mark Banner (:standard8)]

We recently fixed some possibly related bugs in the Thunderbird 3 builds:

Bug 460636 - we sometimes insert extra LF characters when saving messages to disk.
Bug 429891 - Extra CR LF inserted every 10 KB in message in POP3 Sent folder

You can get Thunderbird 3 beta 2 from here http://www.mozillamessaging.com/en-US/thunderbird/3.0b2/ and that has both of those bugs fixed in it.

Comment 12

[Hooonza]

Unfortunatelly, TB 3.02b doesn't manage to resolve the bug and the message read with IMAP has a corrupted X.509 signature.
It's interesting corrupted signatures appear in important messages I send to other people every time maybe but it doesn't appear in test messages... :-)

Comment 13

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #8)
> IMAP log

Where can we find evidence that Tb is culprit of "excess null lines"? At least logs for "FETCH(download of mail data) of multipart/signed mail mail data" is required to analyze problem. Don't you think so?

(In reply to comment #9)
> POP3 log

Log for two multipart/signed mails are seen. Both two mails were corrupted?
> SEND: TOP 28 0
> SEND: TOP 33 0

Log for "TOP nn 0" only is seen. It's log of "header only download". Log merely says "mail header portion was downloaded".
How can the log be an evidence of "Tb corrupted mail data while downloading"?

Comment 14

[Hooonza]

Re: WADA (In reply to comment #13)
> Where can we find evidence that Tb is culprit of "excess null lines"? At least
> logs for "FETCH(download of mail data) of multipart/signed mail mail data" is
> required to analyze problem. Don't you think so?

Please do not tell me these speacial things I do not understand. Tell me how to submit everything about it to people who understand and can manage it :-)
But if I open the sent message with IMAP with the Windows Live Mail then everything is OK and the signature is marked as correct. So I think everythink about the culprit is clear. It is TB bug probably.

> POP3 log
> Log for two multipart/signed mails are seen. Both two mails were corrupted?
No POP3 downloaded mails are corrupted. It affects only IMAP.

> ... How can the log be an evidence of "Tb corrupted mail data while downloading"?
You can explain it to me if you wish. But I am not a programmer so rather I would like to know how to help with this to the nines. Thank you.

If it is helpful I can send both the corrupted mail (displayed by Thunderbird) and the original sent mail (from Gmail web) source codes to somebody who is concerned about it (privatelly, tell me how; or I can give you a secure link to send me your email address).

Comment 15

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #14)
> Please do not tell me these speacial things I do not understand.

Roger.

As you say, comparison of original and "mail downloaded by Tb via IMAP" will help problem analysis. Check of following is important;
 - What is extra new-line (LF? CR? Or CRLF?)
 - Where the new-lines are inserted (by Tb)
(If Bug 460636, LF is inserted after each 8K bytes of mail data)
Can you check it by yourself?
Do you have "multipart/signed mail corrupted (by Tb) in the past" which can be opened to public(attach mail data to this bug)?

Do you still have the mail at IMAP server? If yes, reading the mail by Tb nightly latest-trunk with new profile is very important check.
(Whether Bug 460636 occurred or not can be checked by this.)

Comment 16

[Hooonza]

(In reply to comment #15)
Yes I have the "corrupted mail" saved because of testing of this bug. Wow! I have installed the TB 3.02b at an other clear computer and I see the mail opened with TB has a correct signature now. Good job, guys! Thanks for your trouble!

Comment 17

[Mark Banner (:standard8)]

(In reply to comment #16)
> (In reply to comment #15)
> Yes I have the "corrupted mail" saved because of testing of this bug. Wow! I
> have installed the TB 3.02b at an other clear computer and I see the mail
> opened with TB has a correct signature now. Good job, guys! Thanks for your
> trouble!

As we're not exactly sure which bug fixed it, let's resolve this as "works for me". Unfortunately this does mean we won't be able to backport any fixes to 2.0 but at least we know it is fixed for 3.0.