799450 - Thunderbird adds the text of an email in the Drafts folder to an email I send (Confidential data in other/irrelevant draft mail is silently exposed to unexpected recipients by Tb as data of image part)

Status: VERIFIED FIXED

(MailNews Core :: Composition, defect)

Description

[John_Ha]

Attachment: Fwd Strange problem with central heating - links for valves.eml

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:15.0) Gecko/20100101 Firefox/15.0.1
Build ID: 20120905151427

Steps to reproduce:

1  Compose an email and save it as a draft
2  Edit the saved email, and send it


Actual results:

1  Thunderbird has added the text of a completely different email saved in the Drafts folder to the composed email.  The yahoo.co.uk web mail user recipient sees the composed email, and the "added text" below it.  Gmail users and Thunderbird users do NOT see the added text unless they view the message source, when they see it.
2  My copy of the sent email does not display the "added text" so I do not know this has occurred.  
3  I can see the "added text" if I View > Message source the sent email  

Other points.
1  I sent it to a yahoo.co.uk web email user who saw the "added text"

2  I copied it to a gmail.com user who uses Thunderbird.  He cannot see the "added text" on the email stored on the gmail mail server.  He cannot see the "added text" when he downloads the email into Thunderbird.  But he can see the "added text" by View > Message source in Thunderbird, showing it is there, but Thunderbird is not displaying it.



Expected results:

1  Thunderbird should not have added the email from the Drafts folder to my composed email
2  I cannot see the "added text" in the Sent copy of my email.  Thunderbird should display the "added text" in my sent copy - it is there because I can see it with View > Message source 
3  I had the problem some months ago and posted this problem report http://forums.mozillazine.org/viewtopic.php?f=39&t=2513657  I deleted and recreated my Drafts folder in an unsucessful attempt to fix the problem.
4  I do not know how many of my emails are having emails from the Drafts folder attached to them as only some recipients (yahoo.co.uk) see the added text, whereas other users (gmail.com, and Thunderbird users) do not see the added text.

I believe this is a VERY severe security exposure.  Thunderbird users are probably sending what could be confidential text without their knowing it, and without a visible record of having done so.

Comment 1

[John_Ha]

I think it is probably an intermittent fault and does not happen with all composed email saved in the Drafts folder before sending.
I have AutoSave set to 2 minutes

Comment 2

[WADA:World Anti-bad-Duping Agency]

A variant of bug 766495?

(1) Original draft mail at offset=nnn (call mail-A) :
      multipart/related
         part1.1 : text/html,  img src="cid:abc"
         part1.2 : image/xxx,  Content-Id: <abc>
(2) Edit draft mail-A at offset=nnn
      => image url : mailbox:.../Drafts%3Ennn?part=1.2&filename=...
(3) Compact happens on Drafts folder. 
    New draft mail at offset=nnn (Call mail-B) :
      multipart/(any)
         part1.1 : (any)
                          bug 766495            This bug
         part1.2 :        image/xxx             text/html
(4) Send/Send Later/Save as draft => Data in part1.2 of mail-B is used
    Generate mail data :
      multipart/related
         part1.1 : text/html,  img src="cid:pqr"
                          bug 766495            This bug
         part1.2 :        image/xxx             text/html
                          Content-Id: <pqr>     Content-Id: <pqr>
                          Data from mail-B      Data from mail-B
(5) bug 766495 : image of mail-B is used, and is shown because of image data.
    This bug   : image is broken, because of non-image data(html).
                 because of "text/html in multipart/related",
                 "html from mail-B" is not shown unless message source is viewed.
                 (recent Tb perhaps shows it as attachment)

Note: Bug 766495 is a special case of issues written in bug 532395 comment #42.

Comment 3

[WADA:World Anti-bad-Duping Agency]

(In reply to John_Ha from comment #0)
> Created attachment 669475 [details]
> Fwd Strange problem with central heating - links for valves.eml

Message source in your case.
(1) Content-Type: multipart/related;
(2) cid: url for embed image in message body(text/html, part1.1)
>      <img src="cid:part7.03080308.06010300@MyISP"
(3) Second text/hrml part(part.1.2).
> --------------080608080802020705000705
> Content-Type: text/html; charset=ISO-8859-1
> Content-Transfer-Encoding: 7bit
> Content-ID: <part7.03080308.06010300@MyISP>

Comment 4

[WADA:World Anti-bad-Duping Agency]

(In reply to John_Ha from comment #1)
> I have AutoSave set to 2 minutes

Question about other conditions.
(Q1) local Drafts folder? (POP3/Local Folders) Or IMAP Drafts folder?
(Q2) Do you enable auto-compact?
    (mail.prompt_purge_threshhold=true, mail.purge_threshhold_mb=Positive_number)
(Q3) If yes, do you request Tb to do auto-compact without confirmation?
    (mail.purge.ask=true is not set)

Comment 5

[John_Ha]

WADA
Thanks
(In reply to WADA from comment #4)
> Question about other conditions.
> (Q1) local Drafts folder? (POP3/Local Folders) Or IMAP Drafts folder?
I am using POP3 for all my accounts
> (Q2) Do you enable auto-compact?
Yes I do use AutoCompact
>     (mail.prompt_purge_threshhold=true,
Advanced config editor shows mail.prompt_purge_threshhold=true
> mail.purge_threshhold_mb=Positive_number)
Advanced config editor shows mail.purge_threshhold_mb=200 
> (Q3) If yes, do you request Tb to do auto-compact without confirmation?
>     (mail.purge.ask=true is not set)
Advanced config editor shows mail.purge.ask=true
Advanced config editor shows all the mail.purge variables are:
mail.purge.ask                  default   boolean true
mail.purge.min_delay            default   integer 480
mail.purge.timer_interval       default   integer 5
mail.purge_threshold            user set  integer 60000
mail.purge_threshold_mb         userset   integer 200
mail.purge_threshold_migrated   user set  boolean true

Comment 6

[John_Ha]

I read bug 766495 mentioned above.  Other facts which may be relevant:
1  On several occasions, I have seen many (6 or 8) copies of an email I am editing in the drafts folder, all saved at 2 minute intervals.
2  I often add images to my emails by copy image to clipboard > Crtl/v to paste it in-line (ie not adding it as an attachment).  When I forward such an email, TB often does not put the image in the forwarded email, or puts in a placeholder.  I often need to re-insert the image.
3  TB often has a long delay (10+ seconds) when I press Send (I think saying searching for attachment).  I think this is usually because TB cannot find a pasted image in the email.  The email may be a forwarded one with a pasted image in the original; but I also think it happens with a draft email containing a pasted image.  I have discovered a workaround which is 
i) cancel the send
ii) save the email as a draft
iii) open the saved draft
iv) send the email.
It now sends OK.
I will be pleased to do any tests you ask for, and also to make the original eml file or drafts and drafts.msf folders available to you (I could email them to you - I did not want to post them on the web).

Comment 7

[WADA:World Anti-bad-Duping Agency]

(In reply to John_Ha from comment #5)
> mail.prompt_purge_threshhold = true
> mail.purge.ask               = true
> mail.purge_threshold_mb      = 200

If so, "confirmation dialog before invoking auto-compact" should be shown when Tb atempts to invoke auto-compact.
When you experienced your problem, was "confirmation dialog before invoking auto-compact" shown by Tb? If yes, did you reply "OK" to the dialog? Or did you reply "Cancel" to the dialog? 

If Compact(any of auto-compact and manual-compact) occurred on local Drafts folder when you experienced your problem, you probably saw phenomena stated in bug 532395 comment #42(bug 766495 and comment #0 of this bug is a special case of the phenomena).
Any of them is consistently reproduced with very simple test mail and by simple test procedure. I'll attach test mails to this bug.

If Compact is never related to your problem, it may be phenomenon like bug 532395 comment #47, because you enable auto-save(interval=2 min).
Mechanism of "image or text of different draft mail is used" in this case can not be same as bug 532395 comment #42, because offset of referred draft mail won't be changed if Compact never occurs.

Comment 8

[John_Ha]

(In reply to WADA from comment #7)
> If so, "confirmation dialog before invoking auto-compact" should be shown
> when Tb atempts to invoke auto-compact.
> When you experienced your problem, was "confirmation dialog before invoking
> auto-compact" shown by Tb? If yes, did you reply "OK" to the dialog? Or did
> you reply "Cancel" to the dialog? 
I am almost certain that on every occasion I am asked to compact, I accept. I have also manually requested compacting after archiving many emails.
> If Compact(any of auto-compact and manual-compact) occurred on local Drafts
> folder when you experienced your problem, you probably saw phenomena stated
> in bug 532395 comment #42(bug 766495 and comment #0 of this bug is a special
> case of the phenomena).
> Any of them is consistently reproduced with very simple test mail and by
> simple test procedure. I'll attach test mails to this bug.
Thank you.  I shall run the test and report my result. 
> If Compact is never related to your problem, it may be phenomenon like bug
> 532395 comment #47, because you enable auto-save(interval=2 min).
I am happy to cancel AutoSave (or set to a large value) if it prevents adding text from another email to the composed email.  
> Mechanism of "image or text of different draft mail is used" in this case
> can not be same as bug 532395 comment #42, because offset of referred draft
> mail won't be changed if Compact never occurs.

Other comments:
1  After I recreated the Drafts folder following the original problem several months ago, I discovered r-click Drafts > Properties > Repair folder and I clicked Repair folder.  
2  I have removed all confidential emails from my Drafts folder so that if I see the problem again, I can post the Drafts.msf and drafts files from the profile.  Is there a procedure you would like me to follow to set up my Drafts folder in a "proper state" so we know what state it was in on 10 October? 
3  I do a daily incremental backup so I normally have 45 days of backups, many of which will have Drafts.msf and Drafts.  Unfortunately, I went on holiday and removed my backup disk to a secure location while I was away, and I had not replaced it before the problem email, so I only have backups from 3 August to 17 September.  I have reconnected it so I will have 45 days of historical changes if it occurs again.

Comment 9

[John_Ha]

(In reply to WADA from comment #7)
> Any of them is consistently reproduced with very simple test mail and by
> simple test procedure. I'll attach test mails to this bug.
I cannot see the test email or test procedure files in the Attachments box above.  Please let me know how to get them and I will run the test.

Comment 10

[WADA:World Anti-bad-Duping Agency]

Attachment: Test mails

4 mails :
  mail-00-A : 4000 bytes text/plain mail
  mail-01   : 4000 bytes multipart/related
                part1.1 : text/html, img src="cid:png-A", img src="cid:png-B"
                part1.2 : image/png, Content-ID: <png-A> (green)
                part1.3 : image/png, Content-ID: <png-B> (magenta)
  mail-02   : 4000 bytes multipart/mixed
                part1.1 : text/plain
                part1.2 : image/png (red)
                part1.3 : text/plain, filename="attached...TXT"
  mail-00-B : 4000 bytes text/plain mail

[Steps to reproduce]
(1) Save the attached file as "Drafts" file of POP3 account or Local Folders,
    Restart Tb, "Repair Folder" of Drafts folder, Show "Order Received" column.
(2) Edit draft mail-01 (Offser=4000)
(3) Delete mail-00-A, Compact
    => mail-02 is shifted to Offset=4000
(4) At composition window, Save as draft, Close composition window
(5) Both bug 766495 and comment #0 of this bug can be observed.
    Content-Id: <png-A> : chaned from green of mail-01 to read of mail-02
    Content-Id: <png-B> : image is broken.
                          content is "attached...TXT" data with base64 encoded.

Comment 11

[WADA:World Anti-bad-Duping Agency]

(In reply to John_Ha from comment #8)
> I am almost certain that on every occasion I am asked to compact, I accept.
> I have also manually requested compacting after archiving many emails.

Compact apparently occurs in your environment, and problem of comment #0 is consistently reproduced by simple test.
Confirming.

Comment 12

[John_Ha]

(In reply to WADA from comment #10)
Wada
Thank you.  I repeated the test several times getting different results each time.  I think this was because I did the test very slowly documenting each step, and one or more AutoSaves happened before I finished, and thye result depended on when the AutoSave occurred.  I had made no change to AutoSave (still set to 2 minutes) but it might have been sensible to switch it off for the test.  I then did the test quickly and I do not think an AutoSave happened during the test.  I got the results below.  
> [Steps to reproduce]
> (1) Save the attached file as "Drafts" file of POP3 account or Local Folders,
I closed TB, removed my Drafts files, and saved the downloaded file as ...\LocalFolders\Drafts
>     Restart Tb, "Repair Folder" of Drafts folder, Show "Order Received"
> column.
Restart TB and R-click Drafts > Properties > Repair folder
Select Show Order received column - it has 
  - mail-00-A     0
  - mail-01    4000
  - mail-02    8000
  - mail-00-B 12000
> (2) Edit draft mail-01 (Offser=4000)
Highlight mail-01 and choose Edit.  Add some text to it.  Do not save it, do not close edit window, do not do anything to the two images.
> (3) Delete mail-00-A, Compact
>     => mail-02 is shifted to Offset=4000
Return to Drafts window.  
  - r-click > Delete mail-00-A (top email)
  - r-click Drafts > Compact
   - mail-01    0000
   - mail-02    4000  >> as expected
   - mail-00-B  8000
> (4) At composition window, Save as draft, Close composition window
At the email being edited, I went File > Save as > Draft and then closed the window. 
> (5) Both bug 766495 and comment #0 of this bug can be observed.
>     Content-Id: <png-A> : changed from green of mail-01 to read of mail-02
>     Content-Id: <png-B> : image is broken.
I have the following:
  - mail-01        0   >> green and magenta image, does not have my added text
  - mail-00-B   8000   >> multiple lines of text
  - mail-01    12000   >> bolded, new datestamp, has my added text followed by red image where it is labelled green.  The image under the magenta label is not there and there is a placeholder. This email is a "copy" of the first email.  I have lost mail-02

Comment 13

[John_Ha]

Attachment: This is the Drafts folder after I have done the tests reported in Comment 12

I just renamed the Drafts file to Drafts_after_test_in_Comment_12 and did not add a qualifier.

Comment 15

[persona.accounts]

Tried procedure given by WADA with attachment 669907 [details] (comment #10) and got the same results as John_Ha described in comment #12.
(Windows 7 x86_64, Thunderbird 17.0.6)

Comment 16

[persona.accounts]

Still present in Thunderbird v24.0.1 (Fulle UA "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.0.1")

Some drafts are attached in outgoing e-mail:
----------8<----------
--------------060506060808080701020204
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: base64
Content-ID: <[e-mail]>

0aXTw0bWjxDQo8+PGhaGVh4NCRsw65EZZDyBicnublgYm5cYWl0V2MlsZX4uxludGbWVgR2V
....
ZDyBw0bWjx0aXTDQ+EcnublgGhaGVh4NCRo8YmPZisw655cYWl0V==
--------------060506060808080701020204--
>From - Mon Jul 08 18:45:47 2013
X-Mozilla-Status: 0011
X-Mozilla-Status2: 00000000
X-Mozilla-Keys:                                                                                 
FCC: mailbox://nobody@Local%20Folders/Sent
X-Identity-Key: id1
X-Account-Key: account8
BCC: [e-mail]
Message-ID: <[e-mail]>
Date: Mon, 08 Jul 2013 18:45:47 +0200
From: =?ISO-8859-1?Q?Marie_OLIVE_-_CyberCit=E9?= <[e-mail]>
X-Mozilla-Draft-Info: internal/draft; vcard=0; receipt=0; DSN=0; uuencode=0
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130620 Thunderbird/17.0.7
MIME-Version: 1.0
To: Jane Smith <[e-mail]>
Subject: Re: somestuff
References: <064501ce73d0$6d28b820$477a2860$@domain.tld>
In-Reply-To: <064501ce73d0$6d28b820$477a2860$@domain.tld>
Content-Type: multipart/related;
 boundary="------------050603080104000201040301"

This is a multi-part message in MIME format.
--------------050603080104000201040301
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

<html>
  <head>
    <meta content="text/html; charset=ISO-8859-1"
      http-equiv="Content-Type">
  </head>
...
----------8<----------

Comment 17

[Thomas D. (:thomas8)]

Original content isn't sent out -> dataloss.
Unintended content is sent out -> violating user's privacy (covertly & badly)

-> critical

Comment 18

[persona.accounts]

I can't claim to understand inner cause(s) of this issue nor I can imagine a fix for it but how come this bug (or the related) are still "assigned to nobody" / "OK to take it and work on it" after all this time.

I get that (all?) Thunderbird developers are coding on a voluntary basis but this terrible privacy issue is doing great damages to Thunderbird's reputation I and cannot imagine anyone here wanting this for our beloved e-mail client.

Comment 19

[John_Ha]

Many users don't realise that they are doing this as the user has no knowledge or record of it's happening.  It was only when a colleague told me he had received something he thought I should not have sent that I found out about it.
I hope that now it has been lifted to Critical, someone will have a look at it.  I will be pleased to test any fix.

Comment 20

[WADA:World Anti-bad-Duping Agency]

I closed bug 817245 according to fix of bug 854798. See bug 817245 comment #36 please.

Comment 21

[John_Ha]

@WADA

Thank you for fixing this bug!

Comment 22

[WADA:World Anti-bad-Duping Agency]

Unable to reproduce problem by STR & test mails of Bug 799450 Comment #10 in Tb 38.2.0.
i.e. Bug 766495, this bug(Bug 799450), (and Bug　817245 too, are resolved in Tb 38.2.0.

Comment 23

[WADA:World Anti-bad-Duping Agency]

If you see phenomenon of this bug in Tb 38 or newer, see and read bug 1201782, please.