1164745 - Thunderbird fails to preserve attached photos in Draft form.

Status: RESOLVED INCOMPLETE

(Thunderbird :: Message Compose Window, defect)

Description

[ted thomas]

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:38.0) Gecko/20100101 Firefox/38.0
Build ID: 20150511103819

Steps to reproduce:

Took an e-mail I just sent (which contained embedded images), edited it "as new", then tried to save it as a draft. It wouldn't save the draft. It just sat there with a progress bar saying it was saving the draft, but never finished.

This bug has been present in Thunderbird for a period of years. It is so obvious, and so reliable, I thought at some point, someone at Mozilla would do something about it. 


Actual results:

The results vary. If I try to save the newly edited version of the original e-mail, the typical behavior is to sit there with a progress bar that says it's saving the e-mail, but it never does. If I try to copy and paste the contents of the e-mail to a new fresh (empty) .eml container, it appears to put all the content in the new container, but when I try to send it, it either sits there saying it's sending the e-mail (and never does), or, on occasion, I'll get a message like: "you don't have permission to read the attachments". 

So, it appears Thunderbird is trying to do something cute by perhaps caching the original e-mail's photo objects, rather than just doing the obvious and reliable thing that it does when you create the new document using the 'Forward' function. 

The reason I'm going to the trouble to report this is that I just lost 2 hours worth of work due to this long-standing, inexcusably lame chunk of code. (And it's not the first time this has happened.) 


Expected results:

The only reliable way to re-use a Thunderbird e-mail that contains embedded images (and who knows what it does with any other embedded objects), is to forward the e-mail, and then manually delete the header object. Since I know this works, it demonstrates that Thunderbird is capable of creating a new .eml container that is completely independent of it's predecessor. 

So the rotten code that needs to be repaired may be found in the "Edit as New" and "Save As Draft" portion of the application, which trips over itself constantly by failing to create a fully discreet .eml container for what should be treated as an entirely new instance of whatever original was used. 

And, BTW, when I'm editing a Draft, and I want to SaveAs a new Draft (under a new Subject line), or SaveAs a template, Thunderbird assumes I want to discard the original draft I was editing. Thunderbird is the only program I've ever used that takes it upon itself to delete the original when I do a SaveAs to a different location, or to the same location under a different name. 

You need to have a well-defined naming convention which differentiates a new instance of an e-mail, and clearly support it. If the name is based on the Subject: line, that would be fine. As it is, if I edit a Draft and change the Subject line, Thunderbird appears to throw away the original. Although I can't say for sure if it's still doing it that way, since now, it appears to be saving multiple copies of the Drafts, to the point where I'll see a half-dozen or so of the same original stacked up in the Drafts folder, albeit, I assume each one has a different set of revisions in it.

Comment 1

[Thomas D. (:thomas8)]

(In reply to ted thomas from comment #0)
> User Agent: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:38.0) Gecko/20100101
> Firefox/38.0
> Build ID: 20150511103819
> 
> Steps to reproduce:
> 
> Took an e-mail I just sent (which contained embedded images), edited it "as
> new", then tried to save it as a draft. It wouldn't save the draft. It just
> sat there with a progress bar saying it was saving the draft, but never
> finished.
> 
> This bug has been present in Thunderbird for a period of years. It is so
> obvious, and so reliable, I thought at some point, someone at Mozilla would
> do something about it. 

Sorry for the inconvenience and thanks for reporting. TB is run by volunteers with limited manpower...
Looks like an instance of Bug 532395, although we haven't explicitly had "Edit as New" as a scenario there so far. We've fixed one major cause in that area for TB 38.

> Actual results:
> 
> So, it appears Thunderbird is trying to do something cute by perhaps caching
> the original e-mail's photo objects, rather than just doing the obvious and
> reliable thing that it does when you create the new document using the
> 'Forward' function. 

Not even caching... the problem arises from *linking* to the actual objects in the original email, and that link can break in various ways. Which isn't cute at all, but just bad design. But it's very pervasive, complex, and really deep in the code, so it's not easy to fix.

My meta Bug 877159 alias attach-paradigm-fail describes and collects those issues.

> The reason I'm going to the trouble to report this is that I just lost 2
> hours worth of work due to this long-standing, inexcusably lame chunk of
> code. (And it's not the first time this has happened.) 

I thought there was a way out... but maybe not.

> Expected results:
> 
> The only reliable way to re-use a Thunderbird e-mail that contains embedded
> images (and who knows what it does with any other embedded objects), is to
> forward the e-mail, and then manually delete the header object. Since I know
> this works, it demonstrates that Thunderbird is capable of creating a new
> .eml container that is completely independent of it's predecessor. 

With forward inline, you can still get the same problems.
Forward as attachment, not sure.

> So the rotten code that needs to be repaired may be found in the "Edit as
> New" and "Save As Draft" portion of the application, which trips over itself
> constantly by failing to create a fully discreet .eml container for what
> should be treated as an entirely new instance of whatever original was used. 

Yeah, along those lines. I call it immediate snapshot, possibly with embedding (instead of referencing outer elements which can be moved, deleted, renamed or otherwise get lost if the link breaks).

> And, BTW, when I'm editing a Draft, and I want to SaveAs a new Draft (under
> a new Subject line), or SaveAs a template, Thunderbird assumes I want to
> discard the original draft I was editing. Thunderbird is the only program
> I've ever used that takes it upon itself to delete the original when I do a
> SaveAs to a different location, or to the same location under a different
> name. 

Please check if Bug 321355 fully describes that problem. If not, please file a new bug with detailed steps to reproduce.

> You need to have a well-defined naming convention which differentiates a new
> instance of an e-mail, and clearly support it. If the name is based on the
> Subject: line, that would be fine. As it is, if I edit a Draft and change
> the Subject line, Thunderbird appears to throw away the original. Although I
> can't say for sure if it's still doing it that way, since now, it appears to
> be saving multiple copies of the Drafts, to the point where I'll see a
> half-dozen or so of the same original stacked up in the Drafts folder,
> albeit, I assume each one has a different set of revisions in it.

Needless duplication of the same draft shouldn't happen. Some bugs in that corner, e.g. bug 263114.

Overall, this might eventually be duped to bug 532395.

Comment 2

[ted thomas]

Thanks for the detailed review of the problem. 

Sorry to sound so **** over this, but there's been a bit of frustration built up over the years. Inconsistent version release numbering and pre-release testing, long outstanding bugs not fixed, the "you must use our account setup wizard" problem (for example). On the upside, Tbird appears to outperform Apple Mail as a client to Google Apps Mail, whether running on Linux, or OSX. Does a much better job syncing locally. Apple Mail appears to frequently forget to resume syncing to a local folder if the user has a complex local folder setup. 

I would be willing to lend a hand working on some of these issues, but I'm concerned about the startup-time required just to get in position to do something useful. Is it possible to build everything on a Linux system (including the Windows versions)? I'd rather not have to deal with the MS IDE, and not interested if I have to buy something from them.

Also, is the Seamonkey e-mail client more/less stable? Code more/less complicated?

Thanks for the response.

Comment 3

[Thomas D. (:thomas8)]

Ted has volunteered to lend a hand with coding if we can get him started...
Competent parties CC'ed, could you pls assist... start by answering questions below:

(In reply to ted thomas from comment #2)
> Thanks for the detailed review of the problem. 

Most welcome.

> Sorry to sound so **** over this, but there's been a bit of frustration
> built up over the years. Inconsistent version release numbering and
> pre-release testing, long outstanding bugs not fixed, the "you must use our
> account setup wizard" problem (for example). On the upside, Tbird appears to
> outperform Apple Mail as a client to Google Apps Mail, whether running on
> Linux, or OSX. Does a much better job syncing locally. Apple Mail appears to
> frequently forget to resume syncing to a local folder if the user has a
> complex local folder setup.

Thunderbird appears to outperform Apple Mail... I like that ;)

> I would be willing to lend a hand working on some of these issues, but I'm
> concerned about the startup-time required just to get in position to do
> something useful.

There's a starting point here:
https://developer.mozilla.org/en-US/docs/Simple_Thunderbird_build

> Is it possible to build everything on a Linux system
> (including the Windows versions)? I'd rather not have to deal with the MS
> IDE, and not interested if I have to buy something from them.
>
> Also, is the Seamonkey e-mail client more/less stable? Code more/less
> complicated?
> 
> Thanks for the response.

Comment 4

[Thomas D. (:thomas8)]

Bug 501298 *might* be a starting point to try if you're brave and skilled
+ like this bug, about inline images which fail
+ limited problem which has been clearly identified
+ already has some starting points / support from experts
- it'll still be quite a nut to crack, needs digging into the internals

Comment 5

[:aceman]

Jcranmer, I think you did something in rewriting the composing of a message from various parts. Do you think that rewrite may fix this bug? Or is that something unrelated?

Comment 6

[:aceman]

(In reply to ted thomas from comment #2)
> Thanks for the detailed review of the problem. 
> 
> I would be willing to lend a hand working on some of these issues, but I'm
> concerned about the startup-time required just to get in position to do
> something useful.
The bug here is either trivial, just nobody looked at the cause yet. Or it is very hard to solve so is not a good starting bug. You may want to look at some easier ones (search for severity=trivial or whiteboard=[good first bug]).

> Is it possible to build everything on a Linux system
> (including the Windows versions)? I'd rather not have to deal with the MS
> IDE, and not interested if I have to buy something from them.
You can't build a Windows version on Linux, but you can code everything needed and then let it build on the testing servers of Mozilla.

> Also, is the Seamonkey e-mail client more/less stable? Code more/less
> complicated?
Seamonkey should be the same complexity/stability as Thunderbird as they share much of the backend C++ code, only the UI is a bit different (in appearance, not in complexity of the JS code).

Comment 7

[Joshua Cranmer [:jcranmer]]

(In reply to :aceman from comment #5)
> Jcranmer, I think you did something in rewriting the composing of a message
> from various parts. Do you think that rewrite may fix this bug? Or is that
> something unrelated?

The short answer: no. I've only poked at header-related stuff, and this is very much a body issue. I'm planning on working on rewriting that, but my available free time for such coding has been in rather short supply.


Some more background:
Embedded images are one of the more annoying cesspools of handling. Ultimately, they are represented as cid: links, but we don't actually process cid: internally. That means at various places in the backend, we need to do interconversion between cid: links and the special attachment links (message://......?part=1.2.1, e.g.). Drafts are especially and unfortunately extremely complex, in part because MIME turns out to be a fantastically different structure than the normal UX of email would lead you to believe it ought to be, and in part because some of our code in this area is just plain shitty.

Oh, and embedded images specifically are both undertested in our code and relatively underused in our userbase, which means it's more likely to get undetected problems.

I've not done any digging at all, so I'm going entirely from memory, but I do recall that we try to handle embedded images in drafts (there's a routine to fix up URLs post-draft-save which is... squirrelly in abstraction breakage [1]). If the thing is hanging during composition, I suspect that the code that is doing the "download" of the image is failing to process the URL properly.

[1] Random musing: this is almost certainly broken if you're trying to handle a signed or encrypted message, since the way it tries to recalculate the part numbering doesn't know about the induced parts in crypto code. Honestly, we probably should just chuck everything into a blob URL and screw trying to handle part renumbering.

Comment 8

[Wayne Mery (:wsmwk)]

Ted, does this still fail, or fail differently, if you use the current beta from http://www.mozilla.org/en-US/thunderbird/channel/ ?

Comment 9

[Phoenix]

Resolved per whiteboard

Comment 10

[:aceman]

The handling of embedded images in messages and drafts was recently rewritten in Thunderbird. You may want to try the current TB52 whether your problem is still there.