461785 - Sending or Re-Saving a message to IMAP 'Drafts' folder fails, returns "fetch>UID" error when message contains image

Status: RESOLVED WORKSFORME

(Thunderbird :: Message Compose Window, defect)

Description

[Kory Sutherland]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3
Build Identifier: 2.0.0.17 (20080914)

It may be worth noting that I am using Gmail as my IMAP/SMTP server. However, I have reproduced this bug on an in-house IMAP server running Postfix.

When a message contains an in-line image and is saved as 'Draft' (regardless of whether the image is a local file:// address or remote http:// address) the image address gets reformatted as follows:

imap://[e-mail]:993/fetch%3EUID%3E%5E%255BGmail%255D%5EDrafts%3E1443?part=1.2&filename=myImageFile.gif

The message can be opened from 'Drafts' and edited, however, attempting to send or re-save the message produces the following error:

Unable to save your message as draft. There was an error attaching fetch>UID>^%5BGmail%5D^Drafts>1443. Please check if you have access to the file.

Ultimately this causes sending the message or re-saving it to 'Drafts' to fail.

Reproducible: Always

Steps to Reproduce:
1. Setup an account that uses IMAP (i.e. Gmail)
2. Configure 'Drafts' to be saved in the IMAP 'Drafts' folder (instead of saved locally)
3. Compose a new message
4. Add an image to the message
5. Save the message (to 'Drafts') and close
6. Open the message from the 'Drafts' folder
7. Attempt to save the message again
Actual Results:  
Thunderbird returns the 'fetch>UID' error, and sending the message or re-saving it to 'Drafts' fails.

Expected Results:  
The message should be able to be edited, then sent, or saved back to 'Drafts' folder.

After a message is saved to 'Drafts', then re-opened, an image URLs (regardless of whether they are local file:// or remote http:// address) are changed to be an imap:// address, as seen below:

imap://[e-mail]:993/fetch%3EUID%3E%5E%255BGmail%255D%5EDrafts%3E1443?part=1.2&filename=mwLogoSML.gif

I believe Thunderbird may be changing certain symbols (i.e. ' > ' greater than) into HTML entities - Running the above through a URL decoder produces the following:

imap://[e-mail]:993/fetch>UID>^[Gmail]^>Drafts>1443?part=1.2&filename=mwLogoSML.gif

WORK-AROUNDS:
- - - - - - - - -
1. Remove the in-line image(s) from the message altogether
2. Manually modify each image's URL to use the original local file:// address or remote http:// address.
3. Manually modify each image's properties, disabling the 'Attach this images to the message' option

Comment 1

[Schmidma]

The same problem we found with Thunderbird 2.0.0.18 and an own Cyrus imapd Version 2.2.12. Also with an signature with an external image.

Comment 2

[Schmidma]

New Problem with resending the Mails from the Draft folder: The Background is yellow and the font is an other font - we use Arial as default font. After reopening the Draft message it is Times New Roman.
Moving the draft folder from IMAP to local folder won't help - we see the same error.
The error is a large problem because a lot of our users write drafts save it and send it hours or days later. (Want to send it....)

Comment 3

[Daniel Veditz [:dveditz]]

Not going to block on an unconfirmed bug. Please contact QA or support for help moving this bug along.

Comment 4

[Jeff Fessler]

Status: CONFIRMED

I had this bug on a few machines, so I decided to give it a try on several more, just to see if I could get it to happen everywhere. Here's the detail:

When you attempt to resend (as new or a previously saved draft) an email with an html signature in it, Thunderbird attempt to verify the validity of the pre-supposed "website" of the html signature.  The UID's do not match from the original send, so Thunderbird renders an error and refuses to send the email at all, as it cannot be saved to your Drafts NOR your Sent Items folders after the send completes.  The workaround is to remove the e-mail signature, but for those of us who work mobile and between the office and the home office, this is without a doubt a BUG.  

Mozilla, help us here!

-jeff

Comment 5

[Jeff Fessler]

oops...I failed to mention WHY this bug is occurring.  The HTML signature is stored locally on the machine, but not saved in the email.  In fact, it is stripped after being sent to the "Drafts" or "Sent Items" folder, so Thunderbird does not know where to find the original UID.  

Fix Suggestion: Have Thunderbird capture, compress, and attach the email signature to the original email in the "drafts" and "sent" folders, so that the email sig. can be re-used at a later date.

Comment 6

[Magnus Melin [:mkmelin]]

Still a problem with http://www.mozillamessaging.com/en-US/thunderbird/early_releases/ ?

Comment 7

[Schmidma]

(In reply to comment #5)
The problem also occures if the Image is not embedded via siganture.
We use the Addon Quicktext with an included image.
Also this image is broken after saving it.

I made a doublo click on the image an see the wrong path
Local folder:
The included Image shows an error:
(The m at the beginning is missing - its not a copy-error)

ailbox:///E|/thunderbird/Mail/Local%20Folders/Drafts?numbera5318&part=1.2&filename=signature.gif" 

The path of the Thunderbird Folder is E:\thunderbird\ with the normal Subfolders - thats okay.

IMAP-Folder:
(The i at the beginning is missing - its not a copy-error)

map://user@mailserver:143/fetch%3EUID%3E.INBOX.Drafts%3E387?part=1.2&filename=signature.gif"

In the source code of the draft I see the correct mime-coded file
.....
--------------060803070801050703050906
Content-Type: image/gif;
 name="signatur.gif"
Content-Transfer-Encoding: base64
Content-ID: <part1.07060204.02090507@mailserver>
Content-Disposition: inline;
 filename="signatur.gif"
....

The preview is fine - I see the image.

At the moment I open the Mail for editing the editing part makes the error.
I think there is a little error while opening and decoding the message.
Then the path names are trunkated for 1 character at the beginning.

Comment 8

[Schmidma]

(In reply to comment #6)

The same problem with the same symtoms.

Comment 9

[Clint Collier]

This bug has been "bugging me" since version 2.something.  I use a lot of templates which have images in them that serve as a type of statioary which I "edit as new" to send an e-mail using the selected template.  They type of account seems to have absolutely nothing to do with this problem, it is simply trying to save/store/reuse an e-mail with an image in it.

When I edit as new the template I can send the e-mail out if I act quickly... once the auto save time kicks in it is dead... it can't be saved and it can't be sent.  I have taken to writing the e-mails where I want it in one of these templates separately, copying it and then pasting it into the template and hitting send within the first minute or so... always works that way.

I forget when I first encountered this bug, somewhere way back in version 2 and it has remained there ever since.  I'm amazed that they are still classifying this as an unconfirmed bug.  What are they waiting for to confirm it and fix it?

Comment 10

[WADA:World Anti-bad-Duping Agency]

To all problem reporters(except "new mail composition" case):

"Content-Transfer-Encoding: quoted-printable" case?
If yes, see Bug 487945, Bug 487905, Bug 307023.

To all problem reporters("new mail composition" case):

Do you enable option to send in quoted-printable if non-7bit data is in mail?

Comment 11

[WADA:World Anti-bad-Duping Agency]

To all problem reporters:

If ( ! "Content-Transfer-Encoding: quoted-printable" && IMAP && ("Edit Draft" || "Edit As New"), see Bug 487945 and test mail attached to it.
Same or similar mail structure?

Comment 12

[Mark Banner (:standard8)]

I can't see this in Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1b5pre) Gecko/20090515 Shredder/3.0b3pre although comment 8 says it is possible.

This implies the Steps to repeat aren't clear enough or it has now been fixed, possibly by bug 307023 which was fixed a month after comment 8.

Therefore denying blocking as we can't block on something we can no longer reproduce.

Schmidma or anyone else who sees this, please try a nightly build from here:

ftp://ftp.mozilla.org/pub/thunderbird/nightly/latest-comm-1.9.1/

If it is still a problem, please re-request blocking.

Comment 13

[Wayne Mery (:wsmwk)]

Schmidma can you reproduce with nightly build?

Comment 14

[Joshua Cranmer [:jcranmer]]

RESO INCO per lack of response to previous question. If you feel this change was made in error, please respond to the bug with your reasons why.

Comment 15

[matthew]

i can reproduce this bug (using the same procedure described in the first post) in the latest nightly build as of this posting, using any computer in the office (both xp and vista).

we use thunderbird as our office email client, but the staff cannot save/send re-edited draft messages due to the company signature containing an image. unless this bug can be fixed, we are reluctantly considering moving to a different email client.

we access ability mail server via imap.
the image in our company signature is stored on a local server.

i believe the first post on this bug (Kory Sutherland) describes the issue quite well.

Comment 16

[Dan Mosedale (:dmosedale, :dmose)]

Reopening, as per comment 15.

Comment 17

[Jeff Fessler]

I can confirm this.  Our company mail protocol is IMAP, as well, and we are all trying to use our logo in our Thunderbird email signature.  We've even stored the image locally, to try and make it work, but the same issue recurs.  We've had to remove signatures altogether just to maintain productivity, but it's very unprofessional.

Comment 18

[WADA:World Anti-bad-Duping Agency]

Do you use HTML signature file? Is image included in signature?
If both are yes, how did you create your HTML signature?
Did you follow next guideline?

> http://kb.mozillazine.org/Signatures_(Thunderbird)
> HTML signatures
>  (snip)
>  To make an HTML signature, write an HTML message in Thunderbird.
>  (snip) Choose File – Save As – File... and save the HTML file.
> Including an image in your signature
>  (snip)
>  (snip) ensure that the image tag contains the complete URL of the image file, not just the file name.
>  Thunderbird requires a complete URL so that it can attach the image.

Please be careful with HTML file which is created by "save as" of existent mail.
An example of HTML generated by "Save As - HTML" of an HTML mail with <img> in IMAP folder.
>  <img border=0 width=589 height=102 id="Imagen_x0020_1"
>   src="imap-message://[e-mail]/INBOX#1681?header=saveas&part=1.1.2&filename=image001.jpg"
>   alt="cid:image001.gif@01C7649D.456FB3E0">
This kind of HTML source can't be used as HTML signature with image.

Comment 19

[David Ascher (:davida)]

Bienvenu says fixing this is scary, but we'd certainly take a fix for it.  We won't block on it, given that there's a workaround of not using an IMAP Drafts folder, although we'd love to see a fix.

Comment 20

[KOW]

I've recently encountered this problem as well, only with templates rather than drafts.  I've noticed that the bug is related to a space in the save path.  If I remove any possible reference that will have a %20, the images work fine.  If I replace the space, everything breaks again.  For now, I'm saving things in "Templates" root or "Templates>TemplateType" rather than "Template>Template Type".  

I've tried a couple of tests copying the e-mail in question to various folders and trying to Edit the message as new and re-send it.  The error seems to be occurring only in sub-folders with a space (e-mails in "My Folders" can be edited and re-sent, e-mails in "My Folders>My SubFolder" fail on re-send.)

I'm on IMAP using TB 2.0.0.23.

Comment 21

[WADA:World Anti-bad-Duping Agency]

To bug opener and all problem reporters:

Can you attach draft mail(if edit draft or edit as new of draft) or template mail(if compose from template)? If edit as new case or new composition case, and if HTML signature with image is relevant, attach signature file also, please.

Comment 22

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #20)
> only with templates
> I've noticed that the bug is related to a space in the save path.
> If I remove any possible reference that will have a %20, the images work fine.
> For now, I'm saving things in "Templates" root or "Templates>TemplateType"
> rather than "Template>Template Type".  

If file path has %20 like string, Bug 379053 occurs upon attaching. If file path has space, it becomes %20 in URL(link) of HTML. Because Tb uses URL like format for mail folder, folder of "Template>Template Type" is represented as ".../Template/Template%20Type" internaly.
Your case sounds confusion between " " and "%20" by Tb(i.e. issue like Bug 379053 instead of this bug).
What error message is displayed? Similar message to one in Bug 379053 instead of similar one in this bug?
KOW, is your problem reproduced with latest nightly build? (Bug 379053 is already WORKSFORME)

Comment 23

[KOW]

I haven't yet had a chance to try the nightly build.  That's the next step.  My error is in fact similar (how I found this bug in the first place).  In its entirety:

Sending of message failed.
There was an error attaching fetch>UID>^Templates^T%20and^space>1.  Please check if you have access to the file.

The %20 complaints in the error message are what led me to delete them in the first place.  That particular error message was from editing a template.  Note this was a test folder where "T and space" is a sub folder of "Templates".  What I found oddest about this the space is only a problem in subfolders, not any other folder I create with a space.  

As added detail, the issue in question is a .gif (non-animated) image used as a header in an e-mail template.  Looking to get a copy to you.

More details as I scare them up.

Comment 24

[KOW]

Attachment: Template currently exhibiting described problem from comments 20 & 23 (slightly modified from original)

Template currently exhibiting described problem from comments 20 & 23 (slightly modified from original).  Also exhibits issue on 'Edit as New' from any subfolder with s space in it as well.

Comment 25

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #20)
> For now, I'm saving things in "Templates" root or "Templates>TemplateType" rather than "Template>Template Type".  
(In reply to comment #23)
> There was an error attaching fetch>UID>^Templates^T%20and^space>1. Pleas
e check if you have access to the file.

Aha!
Thanks for attaching template mail data which produces problem, and report about your workaround(remove space from folder name) which indicates cause of problem.

Original error in comment #0.
> There was an error attaching fetch>UID>^%5BGmail%5D^Drafts>1443. Please check if you have access to the file.
Internal URL for mail data in comment #0.
> imap://[e-mail]:993/fetch%3EUID%3E%5E%255BGmail%255D%5EDrafts%3E1443?part=1.2&filename=mwLogoSML.gif
"%" in "%5BGmail%5D"(escaped folder name) is escaped again and becomes "%25" in internal URL.

In your case, "T%20and"(escaped folder name) probably becomes "T%2520and"(% is escaped again) in internal URL which is used for inline image data handling. When you remove space from folder name, it's still "Tand" after escaping. It's probably the trick.
It looks that cause is inconsistency between escaping and un-escaping of escaped folder name. If this is the cause, Bug 379053 won't resolve problem.

To Kory Sutherland(bug opener) and Schmidma(Comment #7):

Can you reproduce your problem with folder path which doesn't have special character?
  Gmail IMAP's [Gmail]/Drafts -> root-level Drafts(Gmail Label=[Imap]/Drafts)  
  "Local Folders" -> POP3 account or dummy POP3 account.

Comment 26

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #24)
> Template currently exhibiting described problem from comments 20 & 23

This is simple(st) multipart/related HTML mail with <img>.
Saved the test mail in "Templates/ABX XYZ" and "[Gmail]/Drafts" of Gmail IMAP.
"Templates/ABX XYZ" is not set as templates folder(ordinal IMAP subfolder).
"[Gmail]/Drafts" is set as drafts folder.

(A) Tb 2.0.0.23
Edit as New(or Edit Draft) -> Image is NOT displayed in HTML format compose window.
Double click of the <img> -> Next URL for the <img>.
> Templates/ABX XYZ
>  imap : //[e-mail]:993
>         /fetch%3EUID%3E%5ETemplates%5E%5EABC%20XYZ%3E1?part=1.2
> [Gmail]/Drafts
>  imap : //[e-mail]:993
>         /fetch%3EUID%3E%5E%255BGmail%255D%5EDrafts%3E18?part=1.2
> Escape of "%" in "%5B"(escaped "[") & "%5D"(escaped "]") is seen.
Save -> same error as comment #23.
Problem of comment #23 was reproduced.
Note: Problem couldn't be observe with "Templates/ABX XYZ" of "Local Folders".

(B) Tb 3 nightly build.
> Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.4pre) Gecko/20090925 Shredder/3.0pre

Edit as New(or Edit Draft) -> Image IS displayed in HTML format compose window.
Double click of the <img> -> Next URL for the <img>.
> Templates/ABX XYZ
>  imap : //[e-mail]:993
>         /fetch%3EUID%3E/Templates/ABC%20XYZ%3E1?part=1.2
> [Gmail]/Drafts
>  imap : //[e-mail]:993/
>         fetch%3EUID%3E/%5BGmail%5D/Drafts%3E18?part=1.2
Save -> No problem.
Problem of Comment #23 is already resolved.

Many of bugs for "Edit As New(edit of template, edit of draft) of HTML mail with image in IMAP folder" seem report for above case.

Comment 27

[WADA:World Anti-bad-Duping Agency]

IMAP case in Comment #7 possibly involves different problem.
  imap : //user@mailserver:143
         /fetch%3EUID%3E.INBOX.Drafts%3E387?part=1.2&filename=signature.gif"
  "%3E" = escaped format of ">"
  unescape => /fetch>UID>.INBOX.Drafts>387?part=1.2&filename=signature.gif"
Folder hierarchy delimiter is "." and personal namespace looks "INBOX.". 
Does Tb 3 latest nightly build process it like next?
               /fetch%3EUID%3E/INBOX/Drafts%3E387?part=1.2&filename=signature.gif"
  unescaped => /fetch>UID>/INBOX/Drafts>387?part=1.2&filename=signature.gif"

Comment 28

[WADA:World Anti-bad-Duping Agency]

(C) Tb 2.0.0.23, mail is saved in Templates/NoSpace
Edit as New(or Edit Draft) -> Image IS displayed in HTML format compose window.
Double click of the <img> -> Next URL for the <img>.
> Templates/NoSpace
>  imap : //[e-mail]:993
>         /fetch%3EUID%3E/Templates/NoSpace%3E1?part=1.2
Cause of problem is probably incorrect escaping of "/"(and "%" by escape) or unescaping failure of escaped "/"(and "%" by escape), when folder name contains special character. (Comment #0, Comment #20)

Templates/ABX XYZ
  /ABC XYZ -> "/" is probably escaped or replaced by "^" -> ^ABC XYZ ->
  /Templates/^ABC XYZ -> "/" is escaped/replaced by "^" -> ^Templates^^ABC XYZ
  -> escape("^Templates^^ABC XYZ") -> %5ETemplates%5E%5EABC%20XYZ
[Gmail]/Drafts
  [Gmail] is already escaped -> /%5BGmail%5D/Drafts -> because of "%", escape or
  replace is executed -> "/" is escaped/replaced by "^" -> ^%5BGmail%5D^Drafts
  -> escape("^%5BGmail%5D^Drafts") -> %5E%255BGmail%255D%5EDrafts

Such over escaping is not seen in IMAP case of Comment #7. It's probably problem of ".(period) instead of /(slash) in URL like internal path" when IMAP server uses "." as folder hiearchy delimiter.

Comment 29

[Wayne Mery (:wsmwk)]

Stans, can you reproduce this?

Comment 30

[Matt]

Using the steps from the initial report I was unable to reproduce. Additionally, I opened the message after the second edit was successfully saved and sent the message using Gmail and it was saved to my Gmail sent folder. (Sorry fr the SPAM Wayne) but I think I have to say it "works for me".

Edit: the account is using maildir, but I don't think that should be impacting on the IMAP process.

Comment 31

[Stans]

(In reply to Wayne Mery (:wsmwk) from comment #29)

Stans, can you reproduce this?

Nope. As Matt reports above, I too couldn't reproduce this. The draft saved without any error each time I edited and saved it. The final edit and sending also completed without any errors. Unlike Matt's case, the Gmail IMAP account is using mbox.

Comment 32

[Ping Chen [:rnons] (away)]

Let's close this, seems embedded images now always use data uri in editor and cid uri in message file.