507842 - encrypted but unsigned msgs appear truncated in IMAP folder

Status: RESOLVED INCOMPLETE

(MailNews Core :: Networking: IMAP, defect)

Description

[Nelson Bolyard (seldom reads bugmail)]

Running a not very recent TB alpha build on WinXP...
I'd guess this bug is probably related in some way to bug 499408.

I sent an encrypted but unsigned email to a co-worker and bcc'ed myself.  
A copy was also put into my sent folder as it was sent. 

The mail was a very short text/plain message and an MS Word attachment.
The person to whom I sent it received it AOK.  But my copies are bad.
I no longer have any good working copy of the message I sent.

Both copies, the copy placed in my IMAP sent folder when it was sent, and 
the received copy which TBird moved from my Inbox to my Sent folder (because
of a filter rule I have that does that), appear to be truncated.  Viewing 
the encrypted message source, they are of different lengths, truncated at different places.  

100% reproducible with GoDaddy's IMAPS server.

I figure the possibilities include:

a) server problem: the messages in the IMAP folder really are truncated
at different places in the IMAP folders

b) server problem: the copy in the IMAP server is complete, but it being
incompletely sent to the client.

c) TBird problem which truncated the message while writing it to the sent 
folder.

d) TBird problem: making message appear truncated after it is downloaded 
from an (any) IMAP folder even though the message sent by the IMAP server 
is complete (e.g. failing to read all the content of the received data 
buffers).

I guess I need another program that can act as an IMAPS client and download
that one message.  This would help determine if the problem is at the server
or at the client.

Comment 1

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #0)
> I guess I need another program that can act as an IMAPS client and download that one message.

Outlook Express(6) is available on MS Win-XP. Windows Mail is available on Vista/7. Apple Mail(probably 2) is available on Mac OS X. 
Outlook Express will help rule out Bug 92111 / Bug 390795.

> 100% reproducible with GoDaddy's IMAPS server.
>(snip) 
> I figure the possibilities include:

Why you don't get IMAP log on both Win & Mac and check IMAP level flow first, even though such very good condition for problem recreation?
> Getting log : https://wiki.mozilla.org/MailNews:Logging
> Enabling NSPR logging on Mac OS X (example in MailNews:Logging is for Mac OS) :
> OSX section oh http://www.mozilla.org/projects/netlib/http/http-debugging.html
> Adding timestamp to NSPR log : Bug 402793 Comment #17
> IMAP command and response : http://tools.ietf.org/html/rfc3501

Auto-Sync=On? Relevant IMAP folder is offline-use=On?
If offline-use=On, check mail data in offline-store file after download.
If offline-use=Off, start test after clear of Tb's Cache directory, and check files in Cache directory after download.
> Tb's Cache Directory on MS Win-XP:
> C:\Documents and Settings\<userid>\Local Settings\Application Data\Thunderbird\Profiles\wxkq5msh.Gmail-IMAP\Cache
See Bug 468637 Comment #10 and your Bug 501253 for downloaded mail data in offline-store or Cache.

To make it simple, I recommend you to check with "new IMAP folder for test" and condition of "single mail only in the IMAP folder".

Comment 2

[David :Bienvenu]

you can get the real contents of an imap message by taking advantage of the fact that we don't use the offline store when copying 2 or more messages at a time from an imap folder to a local folder - so if you select 2 messages in your imap sent folder and copy them to a local folder, you should see what their real contents are.

Comment 3

[Nelson Bolyard (seldom reads bugmail)]

Thanks for that tip David.  

I copied the two apparently-truncated encrypted-not-signed emails from my
GoDaddy IMAPS sent folder to a local folder.  That completed without error, 
and the copies in the local folder were complete, not appearing truncated.  
An added and unexpected bonus was/is that after doing so, the emails in the 
IMAPS sent folder no longer appear truncated either.  

In answer to Wada's questions, 
- I don't know what "auto-sync" is.
- On this PC, all my GoDaddy IMAP folders are configured to allow off-line 
use, that is, I can access my emails in those folders while off-line.  
I guess that's what you mean by "offline-use=On".  Yes?

Comment 4

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #3)
> - I don't know what "auto-sync" is.

I meant following.
> Account Settings/Synchronization & Storage, Message Synchronizing
>    [ ? ] Keep messages for this account on this computer
> It's corresponds to;
> mail.server.serverN.offline_download = false(Unchecked) / true (Checked)
> note: If true is set or is reset to default, this entry will not be displayed.
> If false, next is ignored(false is used). If true, next becomes effective.
>    mail.server.default.autosync_offline_stores = true
 
> I guess that's what you mean by "offline-use=On".  Yes?

Yes. I meant "Folder Properties"/Synchronization, [ X ] Select this folder for offline use by "offline-use=on".

If you change "Keep messages for this account on this computer" of an IMAP account from Uncheked to Checked, Tb enables "offline use" setting of all IMAP folders of the account, and you can see/change offline use setting of all IMAP folders of all IMAP accounts via "Advanced" button at there.
If you change "Keep messages for this account on this computer" of an IMAP account from Checked to Unchecked, Tb disables "offline use" setting of all IMAP folders of the account, and "Advanced" button is disabled.

So, if you define a dummy IMAP account, you can use "Advanced" button of the dummy IMAP account as "offline use setting utitility/panel" for any IMAP folder of any IMAP account, without changing auto-sync/offline-use setting of existent accounts/folders, if you can accept connection error message when you accidentally clicked folder of the dummy IMAP account.

Comment 5

[WADA:World Anti-bad-Duping Agency]

I guess this bug is sibling of your Bug 501253, Bug 468595 Comment #10 with mail attached to your Bug 501253 etc., because encrypted mail usually sent in multipart/signed structure or something like it.
> http://en.wikipedia.org/wiki/MIME

(In reply to comment #3)
> An added and unexpected bonus was/is that after doing so, the emails in the 
> IMAPS sent folder no longer appear truncated either.  

Same issue as Bug 468637 Comment #14? See also step (8) of Bug 468637 Comment #10.

Comment 6

[Nelson Bolyard (seldom reads bugmail)]

I agree that it's likely that many of these bugs are related in some way.
All the truncation bugs are probably related.  
What is the next step?
Do you need me to do any particular experiments? or ?

Comment 7

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #6)
> What is the next step?
> Do you need me to do any particular experiments? or ?

Following is copy of your Bug 499408 Comment #8.
> On my Windows PC system, ALL my IMAPS folders are set for offline use.
> On my Mac laptop, NONE of my IMAPS folders are set for offline use.

This bug is problem on MS Win only(offline-use=On) and your another Bug 499408 is problem on Mac only(offline-use=Off)?

Anyway, do similar check to Bug 499408 Comment #9, and if possible, attach mail data please, in order to see real mail structure.
And, see all bugs listed in dependency tree for Bug 468595 and watch Bug 468595 please.

Comment 8

[WADA:World Anti-bad-Duping Agency]

Bug 468595 is already fixed. (See Bug 468595 Comment #55 for checkin id)
Can you reproduce your problem with newest trunk nightly build?
> http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/latest-comm-1.9.1/
> http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/latest-comm-1.9.1-l10n/
> If MS Win, download win32.zip build. You can test by UNZIP only.

Comment 9

[Wayne Mery (:wsmwk)]

Nelson, does this still reproduce for you?  (ref comment 8)

Comment 10

[Nelson Bolyard (seldom reads bugmail)]

The message is still there in the IMAP folder, appearing truncated, with 
latest Thunderbird.

Comment 11

[Nelson Bolyard (seldom reads bugmail)]

To elaborate, on a different system (a Mac) with a fresh install of latest 
TBird, with a clean "cache", I tried to view the same message with the 
same results as originally reported.  The message appears truncated. 

Looking back on comment 2, apparently there is (or was) a way to coax TBird 
to retrieve all of the message, and I somehow tickled it once back then.
But having to go through extensive manual intervention to read a complete 
message is NOT an acceptable situation.

Finally, I really hate Mozilla Messaging's bug fixing strategy of "let the bug 
sit for 2 years, then demand more follow-up from the reporter and close the 
bug for lack of response from the reporter".  C'mon, guys, be honest.  Don't try to blame the reporter.  Just mark bugs WONTFIX if you won't fix them.

Comment 12

[Wayne Mery (:wsmwk)]

Thoughts on whether this is likely obsolete?

Comment 13

[Christian Riechers]

I think it is. Kind of a duplicate of the the various offline store corruption bugs mentioned in the comments.