Closed Bug 602718 Opened 14 years ago Closed 13 years ago

Inline images not shown in attachment pane when "View>>message body as.plaintext" (after bug 351224, image embed in multipart/related part is not accessible as attachment, so, new "View/Messae Body As/All Body Parts" is needed)

Categories

(MailNews Core :: Attachments, enhancement)

Product:
MailNews Core
Component:
Attachments
Type:
enhancement
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
VERIFIED FIXED
Milestone:
Thunderbird 8.0

People

(Reporter: JoeS1, Assigned: jik)

References

(Depends on 1 open bug,
URL
)

Details

(Keywords: regression, Whiteboard: [See bug 674473 for malformed multipart/related specific case])

Attachments

(3 files, 3 obsolete files)

testcase
37.03 KB, message/rfc822
Details
testcase without alternative part
2.41 KB, text/plain
Details
merge two prior patches and fix code review issues
16.61 KB, patch
Bienvenu
: review+
bwinton
: ui-review+
Details | Diff | Splinter Review
Attached file testcase — 
There are case where content is not "saveable" or even viewable in some multipart-related mail.
One such case is the ability to save a background image.
A common workaround was to view as plaintext. The background image could then be saved via the attachment pane.
I'm sure there are other use cases,but it seems no content-disposition inline images are exposed in "view as plaintext" mode now.
Steps:
 Open the testcase attachment
 (you may have to use it in mbox format due to eml oddities)
 The sample has a background image "A" and an inline inserted image "D"
 Choose "view body as plaintext"
 Notice that nothing appears in the attachment pane

The "inserted" image can be saved with a context click
Good luck with saving the background image.
This is with Mozilla/5.0 (Windows NT 5.0; rv:2.0b8pre) Gecko/20101007 Thunderbird/3.3a1pre ID:20101007035341 of course.
Sorry about that, I sent the wrong testcase, but it still applies.
The "A" gif is the background image, and the "Mozi" is inserted.
So, besides the case of an inaccessible background image, suppose I am viewing in plaintext mode, and I receive an html message with an inline image.
There is no way save the image, or even know that it is there without switching view to html.

Request blocking thunderbird-next
Severity: normal → major
Keywords: regression
It's not even remotely a common use case.
Severity: major → normal
(In reply to comment #4)
> It's not even remotely a common use case

Well, I've use that trick (view as plaintext) to show message parts that were invisible/inaccessable for a long time.

I suppose what we really need in "View Page info" option similar to the FF Tools>Page info.
I'm pretty sure NN4.x optionally showed all the related mime parts in a clickable fashion.

My point was, we should be able to see all the parts of a received mail, and do with it what we wish.
Severity: normal → major
I understand the issue, and your usage model. I'm just saying probably way less than 1% of users would ever see or care about this issue.
Actually, I'm using that "rare usage model" frequently in the real world with the branch releases. It all depends what type of messages you are dealing with, how frequently they are coming in with inlined images, and for which purpose you want to save them. Right now, if you view your messages in plain text by default, you don't even have /any/ indication that an inline image is present.

Do you have any reference to the bug which removed that, and for which reason?
I see that this is blocking bug 351224 as the possible cause, however there is no corresponding check-in from that bug which would match the regression range. However, it points to http://hg.mozilla.org/comm-central/rev/f734adc19f35 which in turn points to bug 462919 which is not related to that checkin. I can't find any mail/news bug matching that changeset.

This works still fine in a 20100920 build.
Eh, stupid me - that's http://hg.mozilla.org/comm-central/rev/b731a64e2504
from bug 351224 comment #67 (David, it would help if you could always add
the changeset to your check-in comments for reference, as others do...).

Jonathan, any idea how to make this part happen again?
(In reply to comment #9)
> Do you have any reference to the bug which removed that, and for which reason?

rsx11m, as you know, bug 351224, which is set in Blocks: field of this bug, produced this bug.
According to RFC on multipart/alternative, I believe that change by that bug(completely ignore non-interested part(s) in multipart/alternative) is absolutely proper behaviour. And, as RFC never defines *correct* behaviour of MUA, handling of non-interested part(s) in multipart/alternative is up to MUA, and the handling by MUA of the non-interested part(s) in multipart/alternative is up to developer(s) of an MUA. Change in that bug is decision by David and patch creator of that bug, so it should be respected.
I believe, as Joe Sabash(bug opener) says, that new "View Page info" like feature(View Mail Info?) is better to fulfil requirements by Joe Sabash and you.
rsx11m, what do you think?
Just for the record;
Although I consider the severity here "major", I did not intentionally trump David's downgrade to normal.
That somehow must have happened on my reply ?
(In reply to comment #10)
> I believe, as Joe Sabash(bug opener) says, that new "View Page info" like
> feature(View Mail Info?) is better to fulfil requirements by Joe Sabash and
> you.

WADA, thanks for the additional information. I agree that the interpretation of alternative parts is frequently fuzzy, and we also know that there may be two different ways how to structure inlined images (and both are used):

  multipart/alternative
    text/plain
    multipart/related
      text/html
      image/*
or:

  multipart/related
    multipart/alternative
      text/plain
      text/html
    image/*

So, the latter would show inlined images whereas the former doesn't?

The question of the RFC's intention aside, the issues are (1) discoverability by the user of images being present in the message when viewing in plain text, and (2) the save-all scenario. Neither of those I see covered by a Mail Info dialog, though it would be nice to have such a thing.
Summary: Inline images not shown in attachment pane when "View>>message body as.plaintext" → Inline images not shown in attachment pane when "View>>message body as.plaintext" (ater change by bug 351224, image embed in multipart/related part of multipart/altrnative mail is not shown as if an attachment)
Actually, you don't even need a multipart/alternative construct, the related parts alone are sufficient to not show and make the image accessible in plain text mode. This simplifies the minimum testcase to:

  multipart/related
    text/html
    image/*

and sort-of voids the argument re alternative parts treatment. I'm adjusting the bug summary again (sorry for the noise...).
OS: Windows 2000 → All
Hardware: x86 → All
Summary: Inline images not shown in attachment pane when "View>>message body as.plaintext" (ater change by bug 351224, image embed in multipart/related part of multipart/altrnative mail is not shown as if an attachment) → Inline images not shown in attachment pane when "View>>message body as.plaintext" (after bug 351224, image embedded in multipart/related part is not accessible as attachment)
Version: unspecified → Trunk
David and I discussed this very issue before my patch to fix multipart/related was checked in, and we concluded that the new behavior that you're seeing now was both acceptable and correct.

There are other email clients that don't display the non-body parts in multipart/related parts no matter what.

The RFC for multipart/related would seem to suggest that this is correct behavior. As I understand them, when there is a multipart/related part, the body of that part is supposed to be displayed and everything else is supposed to be *not* displayed, as it is considered nothing but "helper" content intended for helping to display the main body.

Content that is not, in fact, intended as helper content should not be attached as sub-part of the multipart/related part. It would not, e.g., make sense to attach a PDF as a sub-part of a multipart/related part (unless, I suppose, you were displaying it inline within the multipart/related body); rather, you would attach it as an independent part, at the same level in the hierarchy as the multipart/related part.

In reply to, "I'm using that 'rare usage model' frequently in the real world with the branch releases," the fact that one sophisticated user needs particular functionality is not evidence that it is commonly used or should be present for everyone.

And there is a workaround -- save the message to an EML and use unpack or some other tool to get at the parts you want.

When you view the message as plaintext, it would not be correct to display the inline images, because they are in a different body part, one that you are not currently viewing.

In short, not only do I not believe this is a major issue, I don't think it is an issue at all.
(In reply to comment #15)
> In reply to, "I'm using that 'rare usage model' frequently in the real world
> with the branch releases," the fact that one sophisticated user needs
> particular functionality is not evidence that it is commonly used or should be
> present for everyone.

I'll have to write down that quote for other occasions. ;-)

Neither you nor David in comment #6 present any evidence either that would support your claims (specifically not the "1%" statement), so I'd suggest not to make any such assumptions unless you have something more solid to present.

To name a common use case, someone is sending you a bunch of family photographs (or screenshots in my case) and has two options to do that: Attach them (we are fine in this case as they will show as attachments and we have "Save All") or use drag-and-drop or copy-and-paste into the message body (in which case we have the problem). The recipient doesn't care how this was done, he or she wants to save them, and won't really want to do that one-by-one. My current recommendation to users (e.g., in the forums) is to switch to Plain Text mode, and then they can save them all in one step. So, that's no longer possible with bug 351224 having removed that option, thus some other way to accomplish that would be desirable.

> When you view the message as plaintext, it would not be correct to display the
> inline images, because they are in a different body part, one that you are not
> currently viewing.

Not in the minimal testcase of attachment 487217 [details], where the image is
technically part of the same body part selected for viewing (though we convert it to plain text from HTML by the user's choice). The problem of discovering that there is more to view than just the plain text remains. If you are lucky, you notice from the size of the message that there is more and switch to one of the HTML views, or - as in this example - an "ALT" text is given and appears. From the serializer's (and possibly the RFC's) point of view you may be right, but I disagree that it's not an issue at all.
> (comment #16)... use case, someone is sending you a bunch of family photographs
> (or screenshots in my case) and has two options to do that: Attach them (we are
> fine in this case as they will show as attachments and we have "Save All") or
> use drag-and-drop or copy-and-paste into the message body (in which case we
> have the problem). The recipient doesn't care how this was done, he or she
> wants to save them, and won't really want to do that one-by-one.

Coincidentally, this issue just came up today in a forum thread, and that user is a happy camper again saving from the Plain Text setting. I agree that the current solution for this use case isn't easily discoverable either, but at least that option is still available in the releases.

Based on that user's initial attempt to find it in the File > Attachments menu, maybe adding a separate block of inlined or background content could be added and saved with "Save All" (but ignored for "Delete All" or "Detach All" to avoid regressing bug 351224 in turn). This should be better discoverable by
the user even if the attachments stay unlisted in the attachment pane.
URL: http://forums.mozillazine.org/viewtop...
As for not-well-formed multipart/related, main culprit of incorrect multipart/related mails by some mailers is probably MS Outlook.
  MS Outlook doesn't strictly respect RFC for MHTML which was implemented by MS.
  a) Even if multipart/mixed, MS Outlook shows text/html of <img src=cid:...>
     ang image/* with Content-Id: as if multipart/related.
  b) Even if multipart/related, MS Outlook shows parts which is not pointed by
     <img src"cid:...> as if attachment part of multipart/mixed,
     and when forward, send the non-referred image/* as attachment.
  As for Content-Id:/cid-url and Content-Disposition:, a) is relatively
  natural enhanced interpreting of RFCs, although it is not clearly defined
  by any RFCs.
  b) is not defined by RFC for MHTML which was implemented by MS himself.
  But RFC never refers to cases not covered by RFC, and RFC never defines
  behaior whe RFC violation exists. So, Outlook's behavior can not be called
  "incorrect behavior".

For multipart/related and "View/Message Body As/Plain Text" by Tb.
- No fault of mail sender is involved.
- Who interpreted mulipart/related[text/html+image/*] as mulipart/related[
  text/plain+image/*] is Thunderbird.
So, Tb need to provide a way to access non-rendered image/*, if user wants to access the non-rendered image/*.
"Showing as if attachment when image/* part in multipart/related is not rendered" was quirks used by Mozilla family for long time, in order to provide a way to save non-rendered part to a local file.
(David, Netscape Mail's feature, wasn't it?).
Such quirks produced user's confusions and generated complaint like bug 351224 comment #0, because detach/delete was implemented after that.
And, the confusing but log-lived quirks was already killed.

I think one of next can be a solution for many variants of malformed mail case.
i)  New mode of "Show/Interpret any multipart/* as if multipart/mixed".
    Provide way to show or save(and detach/delete?) any part in multipart/*.
    note: this is effective for "reversed order in multipart/alternative" case. 
ii) New "View Mail Info or Structure" feature.
    Provide way to show or save(and detach/delete?) any part in multipart/*.

Jonathan Kamens, is such intentional RFC violation(intentional misinterpretation of RFCs for multipart/*) possible without conflict with current implementation?
(In reply to comment #18)
> i)  New mode of "Show/Interpret any multipart/* as if multipart/mixed".
>     Provide way to show or save(and detach/delete?) any part in multipart/*.

Such an additional mode may be equally tricky to discover and even more confusing if all parts are just concatenated in the display in addition to the attachment pane (and alternative text/plain + text/html would duplicate the display of the message text, or not?).

> ii) New "View Mail Info or Structure" feature.
>     Provide way to show or save(and detach/delete?) any part in multipart/*.

This would correspond to the suggestion of a "Mail Info" dialog, which may in part be taken from the browser's "Page Info" implementation. However, the use case to save all images similar to attachments would require the addition of a "Save All" button in that dialog.

Other options as mentioned here and in bug 608644:

iii) Add another block to View > Attachments (SM: Message > Attachments) to
     list all embedded images (inlined or as background) and include them in
     the "Save All" procedure.

iv)  Add a context-menu item "Save All Images" in Simple/Original HTML modes
     which acts like "Save All" in the attachment pane but applies to all
     background and inline images currently displayed.
(In reply to comment #16)
>... but I disagree that it's not an issue at all.

You're right. I was too quick to dismiss the concerns raised here.

There should be a way to access all the parts of a MIME message within the application. The lack of this functionality is not a "bug" per se, since the application is behaving correctly, but it is a functional gap which should be filled.

I think WADA is correct that all things considered, the best way to provide this functionality would be a general-purpose option to show all the MIME parts in a message. I don't think image-specific functionality as rsx11m proposed is the best way to go, because (a) it's impossible to anticipate every way in which this functionality might be used, and (b) I don't think it's too onerous in this context to ask people to find the image attachments for themselves within the list of attachments, or just save all and then delete the files that aren't images.

So I think the way to fix this is to add a View > Message Body As > All Message Parts mode, and when that view mode is selected, display all message parts both in the body area and the attachments list.

If everybody's OK with that, then I will see when I can find the time to work on making it happen, but I can't promise when I'll be able to do that, so if somebody else wants to work on it in the meantime, that's fine with me.
Yeah, looking at images only might be too restrictive, users may want to embed audio or video content as well at some point. So, showing all parts would be fine, I'm still with the problem though of displaying both plain-text and HTML parts of the same message (especially if the "All Message Parts" sticks like the other "Message Body As" options). Following suggestion based on that:

v. Similar to View > Display Attachments Inline, have another option in the
   View menu "Show all Parts as Attachments" (or similar), which would leave
   the current View > Message Body As setting untouched and just make all
   parts accessible in the attachment pane and the Attachments menu.

I think this option would satisfy both (1) and (2) of my comment #12 as well
as Joe's issue to access a background image (and also would allow audio and video parts to be accessed in this way) without in turn affecting the basic view a user chooses with the View > Message Body As selection.

Implementation would require another boolean pref setting for that option, otherwise it should be fairly similar to variant (i) proposed by WADA.
View>>Show all parts as attachments feature is an excellent suggestion.
This would address some other conditions where content has never been savable.

And yes, there are newsgroups that embed sound files with MS proprietary  BGsound tag for example, that remain hidden to the TB user.

Example:

<BGSOUND balance=3D0 volume=3D0=20
src=3D"cid:0FC6931D56864BB69F4326E7074D3C0E@2WIRE106">

------=_NextPart_000_0061_01CB7634.2BC63CA0
Content-Type: audio/mid;
	name="i_said_it_before_(oliver_gannon).mid"
Content-Transfer-Encoding: base64
Content-ID: <0FC6931D56864BB69F4326E7074D3C0E@2WIRE106>

TVRoZAAAAAYAAQAMAeBNVHJrAAAEngD/.....etc.......

Note the lack of Content-disposition in the above.
I have seen this condition causing "inability to access" cases other than rhe bgsound tag and would hope that the proposed feature would help there as well.

I whole=heartedly back this new view proposal.
Bug 436555 is for improvement of current attachment pane display, which is mailny for issue of bug 101891. (See Bug 436555 comment #4)
Confusions due to "not-inline-shown part in multipart/related is displayed as if attachment" portion in concern of bug 101891 is already resolved by bug 351224.
"View>>Show all parts as attachments" feature by Jonathan Kamens is an answer to my concern of bug 436555 comment #2 & #3.

Setting dependency to bug 436555 for ease of tracking and search.
Depends on: 436555
Jonathan, bug 564423 has a related discussion and there's an old patch in bug 554294 that removes Part 1.2-like attachments, so you might be interested in starting with/reusing that old code, given how tricky stuff with libmime is.

On a side note, you'd want to make sure this doesn't affect how libmime talks to Gloda's JS Mime emitter.
So, is there any way for users migrating from TB 3.x to 5.0 for saving or otherwise accessing "multipart/related" content that could be recommended?

Topics are showing up in the forums and it's a dealbreaker for some users...
Can you guys cut the **** match and just fix the code?
(In reply to comment #27)
> Can you guys cut the **** match and just fix the code?

https://bugzilla.mozilla.org/page.cgi?id=etiquette.html
I've just been involved in yet another forum thread on this topic, regarding SeaMonkey this time, thus it seems that this is (was) a feature which is missed and not just a rare use case.

(reiterating comment #22)
> v. Similar to View > Display Attachments Inline, have another option in the
>    View menu "Show all Parts as Attachments" (or similar), which would leave
>    the current View > Message Body As setting untouched and just make all
>    parts accessible in the attachment pane and the Attachments menu.

Given that bug 282068 now hides the attachment pane by default in Thunderbird (and it was always a scrollable list in SeaMonkey), the displayed attachments wouldn't pose any issue in terms of additional vertical space needed, also considering that it would be a user-selectable option. This option wouldn't be wasted effort towards a larger solution, rather still apply if any of the other bugs improves display and introduces identification of "fake" attachments.
Blocks: 668628
After upgrading to TB 5.0 I was hit with this bug/regression. My use case scenario was not mentioned here: I used to switch to text view and have attachments displayed to DELETE them! Before archiving. In the last 14 years of using Netscape/TB I have always carefully tried not to save any attachments with my messages - keeps the archive nice and small, especially when backing up. You just made this harder again :-(

BTW, I'm sure there must be others out there who like to delete dancing cats and smileys and company logos from their messages before archiving them...a friend in Argentina was really happy when I showed him how to do the plain text/delete inline attachment trick a few months ago.

I came here because I had posted a question at getsatisfaction (http://getsatisfaction.com/mozilla_messaging/topics/show_inline_images_in_attachment_box_to_be_able_to_delete_them) and rsx11m pointed me here (thanks!). Please fix this :-)

P.S.: To be able to log into bugzilla here I just had to reset my password. Lastpass then sent me a nice email about the password change....full with inline images. 81 KB. I'd like to keep it, but there's no way all those images are going into my archive...and I can't delete them anymore unless editing the source by hand :-(
I think my proposal to add a new message display mode that displays the entire MIME structure of the message both in the view pane and the attachments list is going to be very hard to implement. I'm not 100% certain of that, since I haven't looked at the code in a while, but my gut tells me it's not going to be easy at all.

However, I think there is a lesser solution which will be much easier to implement and will satisfy the vast majority of use cases which are currently suffering, and that is to show all the MIME parts that are currently being used in the attachments pane when View > Display Attachments Inline is unchecked.

Let me explain in a bit more detail what I mean here... When I fixed the MIME multipart interpreter to work properly, I modified it not to emit MIME parts that aren't currently being viewed. For example, if the message is multipart/alternative with a text/plain part and a multipart/related part, and Thunderbird is displaying the multipart/related part (which would be the default), the text/plain part is totally invisible to the display code, so it would be hard to get that to display in the attachments pane.

However, all the various parts of the multipart/related are accessible to the display code, so with some minor tweaks I should be able to get them to display in the attachments pane.

What do people think of this as an interim fix, which will be much faster to implement than the major work of actually making all parts of the message visible in the attachments pane? As an aside, I will mention that if people are amenable to this fix, I am likely to be able to find a bit of time to work on it, whereas if people insist on only the whole shebang fix previously proposed, I probably won't be able to get to it soon. I'm just saying'...
Thanks, Jonathan! I'd be happy about the interim fix.
Providing attachment-pane access to any multipart/related content currently part of the view would certainly help for the bulk of the cases, so I'm all for it. It'll depend on Blake's opinion if tying it to "Display Attachments Inline" is a good way to toggle that feature or if it should rather have its own preference.
My gut feeling is that this is _way_ more complicated than a regular user would care about, and so at the minimum I would like it to have its own (hidden) preference.

As a total geek, I think something to browse and edit the MIME-part tree would be really cool, and would be great as an extension.  ;)

Thanks,
Blake.
I'm fine with comment #33 using a hidden pref. It's a frequent enough use case to justify making it available, but won't necessarily need a checkbox in the UI.

A "true" tree-view option would be nice for messages with complex structure, but that probably should go rather into the bug 436555 discussion.
Comment #33 is fine as an interim solution.
If there will be a hidden pref, I think that should be "show attachment" by default.
We have new users that are missing content just because of their choice of view settings.

Seems to me that even OE would warn when "too strict" security settings prevented content from being rendered. Although not in the scope of this bug, maybe we could apply a similar notification at some point.
Something like "Due to your view settings, some content has not been displayed"
..and the obligatory X don't show this message again.
I do not agree that this should be a hidden preference.

People who use this functionality, and I count myself among them, don't want to see all the MIME parts in the attachments pane cluttering the UI all the time; they only want to see all the MIME parts when they have some reason to need to access them, e.g., to save background images or delete attachments they don't want.

As such, what they would like to be able to do is execute some command which will toggle displaying all the MIME parts in the attachments pane for the message currently being viewed, without affecting the display of other messages.

If we are concerned with cluttering up the menus, then perhaps hide the menu command by default and provide a preference to expose the menu command, rather than a preference to always show all the MIME parts in the attachments pane?
(In reply to comment #38)
> We have new users that are missing content just because of their choice of
> view settings.

What is this "missing content" that people keep referring to?

Regular attachments will show up in the attachment pane. Inline and background images will show up in the body of the message. I don't see where there is any "missing content", even without the change we're discussing.
(In reply to comment #40)
> (In reply to comment #38)
> > We have new users that are missing content just because of their choice of
> > view settings.
> 
> What is this "missing content" that people keep referring to?
> 
> Regular attachments will show up in the attachment pane. Inline and
> background images will show up in the body of the message. I don't see where
> there is any "missing content", even without the change we're discussing.

You are now unable to right click on a displayed image and either delete it or detach it. I used that functionality very often and now that it's gone, there is no way to remove images from archived emails unless you tediously save the message to an .eml format and then edit the html code by deleting from the start of an image to the end. That process is FAR more time consuming than simply being able to click on an image in the attachment pane and select "Delete".
(In reply to comment #40)
> (In reply to comment #38)
> > We have new users that are missing content just because of their choice of
> > view settings.
> 
> What is this "missing content" that people keep referring to?
> 
> Regular attachments will show up in the attachment pane. Inline and
> background images will show up in the body of the message. I don't see where
> there is any "missing content", even without the change we're discussing.

There is at least one case where content appears to be missing. See https://bugzilla.mozilla.org/attachment.cgi?id=542864 from bug 668259. The tif file is not visible in any form in the e-mail no matter which view I select. I have to view source or save the email and open in a text editor to get the tif information.
I spent several hours last night working on implementing the interim fix I proposed. I'm sorry to say that I underestimated the scope of the task. The current architecture of the code is not at all conducive to my proposed fix, or for that matter to any fix. I'm going to have to discuss this further with the owners of the relevant Thunderbird and Mozilla Core components to decide how to proceed. Unfortunately the fix isn't going to be quick.
(In reply to comment #41)
> You are now unable to right click on a displayed image and either delete it
> or detach it.

This is not "missing content." This is missing functionality. I understand that the functionality is missing and that we need to fix that. My objection was to the claim that there was "missing content" that users could not see; I see no evidence that this is the case. If it were, this bug would be a lot more severe than it is. Without actual "missing content," I agree that the bug needs to be fixed, but I don't think it is as severe as some of the commenters here would have us believe.

For example, Comment #32 about 81KB of non-detachable images is in my opinion just a little bit over the top. 81KB? Really? When you can get a terabyte of disk space for $75?
There is indeed content which can't be accessed other than through the attachment pane. This includes background images and audio per Joe's comment, and the cases in the duplicates where describing "related" content which cannot be displayed inline, thus won't even show up in the message itself.

I would appreciate it if we could get back to *how* to solve this issue rather than *why* it should be solved...
(In reply to comment #42)
> There is at least one case where content appears to be missing. See
> https://bugzilla.mozilla.org/attachment.cgi?id=542864 from bug 668259.

The MIME in that message is malformed. The structure is like this:

multipart/related
  multipart/alternative
    text/plain
    text/html
  image/tiff

Because the "image/tiff" attachment is inside a multipart/related, the MIME spec says that it is a "helper" attachment, not a standalone attachment that is supposed to be viewable separately. The way Thunderbird is displaying that message is correct; it's the message that's wrong for not linking to the image/tiff from the text/html.

Don't get me wrong, I agree with you that there should be a way to get to this attachment. But I want to be clear that in this particular case, the fault lies with whatever program formatted that MIME message, not with Thunderbird.
Jim and I duped that bug to here under the assumption that "show all parts" would be implemented. If we go a different path here which doesn't cover incorrectly or unwired related parts we should undupe and reopen bug 668259 for that purpose.
(In reply to comment #44)

> For example, Comment #32 about 81KB of non-detachable images is in my
> opinion just a little bit over the top. 81KB? Really? When you can get a
> terabyte of disk space for $75?

Well, that was just an example of a message that came in when I was typing this. But for me it's also a case of principle: I have gigs and gigs of images, but NOT in my email! In fact, no attachments should make it into my email archive. Why? Several reasons: searchability, but mainly: backup. Attachments cannot be compressed (at least images can't). All emails since I started emailing (110k of them) come in at 140mb in a zipped backup. One image-message of 81kb per day over the last 14 years or so would more than triple the size of the backup...with the according longer time for zipping the backup, burning them, uploading off-site.

So while wanting to delete all attachments from a message might seem anal at first, there are some real practical reasons behind it.
(In reply to comment #39)
> People who use this functionality, and I count myself among them, don't want
> to see all the MIME parts in the attachments pane cluttering the UI all the
> time; they only want to see all the MIME parts when they have some reason to
> need to access them, e.g., to save background images or delete attachments
> they don't want.

Oh, I meant that the hidden preference would be to show all MIME parts (or not, by default) when the user unchecked "View > Display Attachments Inline", so you should be able to only see them when you need to access them…

Thanks,
Blake.
I had a revelation today and realized that this would actually be easier to fix than I thought it would, so here is my proposed patch.

The patch makes the following functional changes:

* Add a new View > Message Body As > All Body Parts menu item.
* The menu item is hidden by default.
* Add a new mailnews.display.show_all_body_parts_menu preference, set to false by default.
* Show the new menu item when the aforementioned preference is set to true.
* When the All Body Parts choice is activated, multipart/alternative and multipart/related MIME parts are interpreted as multipart/mixed, which renders them visible in the message body and attachments pane.

Additional notes...

* The new label and accesskey I added to messenger.dtd need to be translated, but I don't know how to make that happen. Pointers would be appreciated.
* There are no significant existing unit tests related to the html_as functionality, so I didn't feel it necessary to add one.
* While I was at it, I did some cleanup to the multipart/alternative code to make it consistent with the multipart/related code. In particular, special case handline of nsMimeMessageAttach in mimemalt.cpp has been removed because it's now being handled at a higher level, in mimei.cpp, which is much cleaner and more consistent.
* I ran xpcshell-tests with these changes and they all pass.

For the record, I'm a little uncomfortable about making this functionality controlled by a hidden preference. How are people who need it going to find it? I mean, I guess we need to make sure it's documented well on support.mozillamessaging.com?
Attachment #545305 - Flags: review?(dbienvenu)
thx for tackling this - you don't have to worry about localization of strings; the localizers have tools to see what strings have been added to .dtd files, and they know they have to translate the new strings.
(In reply to comment #50)
> For the record, I'm a little uncomfortable about making this functionality
> controlled by a hidden preference. How are people who need it going to find it?

The initial motivation for the hidden preference was to avoid confusion by unexpected behavior when tying the "show all parts" functionality to the Display Attachments Inline menu item. Since this is now an explicit choice in the Display Message Body As menu, this reasoning does no longer seem applicable. Thus, that hidden preference adds code complexity without much gain in this design.
(ultimately Blake's decision, thus you may need to request ui-review? from him.)
Attachment #545305 - Flags: ui-review?(bwinton)
New patch fixes segfault when opening draft for edit.
Attachment #545305 - Attachment is obsolete: true
Attachment #545592 - Flags: ui-review?(bwinton)
Attachment #545592 - Flags: review?(dbienvenu)
Attachment #545305 - Flags: ui-review?(bwinton)
Attachment #545305 - Flags: review?(dbienvenu)
With this patch, I see Part 1.1.2 for most messages with attachments. That would be a bit annoying but I suppose it's a small price to pay for users who really need to see all attachments...
(In reply to comment #55)
> With this patch, I see Part 1.1.2 for most messages with attachments. That
> would be a bit annoying but I suppose it's a small price to pay for users
> who really need to see all attachments...
Yes. It's not the prettiest solution, in the UI, but it's the simplest by several orders of magnitude, and it's not like people are going to leave this display mode turned on all the time -- they'll only turn it on when they need it.
I've requested a try server build with this patch so people who are concerned about this issue can try it, and make sure it addresses their issue. I'll post a link when builds are odne.
I'd like to propose adding this patch to my previous one. It's not essential, and the other one can go without this one, so I'm leaving it as a separate patch to be reviewed and considered separately.

The issue this fixed is that when you use View > Message Body As > All Body Parts, the very first body part will be excluded from the attachment pane in certain circumstances. This patch undoes that so it'll be included all the time. I think it's worth including, but it's your call.

BTW, I'd still like to hear from Blake about what he thinks about showing this menu option all the time, i.e., not hiding it with a preference, which seems awkward and perhaps unnecessary to me. But he's the UI guy. :-)
Attachment #546374 - Flags: ui-review?(bwinton)
Attachment #546374 - Flags: review?(dbienvenu)
I'm not sure we want to expose the behavior you get when displaying all parts to normal users, so I would lean a bit towards the hidden pref.
I agree with David, this seems like advanced behaviour, and so saving it for advanced users (via either the hidden pref, or an addon that toggles the hidden pref) feels like the right choice to me.

Thanks,
Blake.
I don't really understand every technical detail written down here. All I know is that a recently got a mail with a PDF attached as "Content-Type: multipart/related" (see bug 671821). I don't know if the sender of the message made drag&drop to add the pdf to the message or if he used mail client commands. All I know is that his mailbox is on an Exchange 2007 server and that he has a Mac Book Pro ... so maybe not using Outlook.

From what I understand of this discussion, the problem is in which part of the message the pdf got attached. All I see is that Thunderbird 5.0 doesn't show the attachment when I open the mail, not even with File > Attachments. Therefore I even thought he had forgotten the attachment unless it was there.

Do you really want to explain to simple users those technical details rather than show the attachment ... even if its in the "wrong" part of the message?
I admit to being of two minds about this. On the one hand, I want to make Thunderbird useful to people. On the other hand, you can only bend the standards so much before you stop being part of the solution and start being part of the problem.

If an email client is putting attachments inside a multipart/related body part that are not linked to from the body of the message (BTW, are you sure there wasn't a link to the PDF somewhere in the displayed body of the message?), then that email client is broken.

I agree that we should provide a way for people to access such attachments, but it is not clear to me that the *default* behavior of the application should be to expose attachments which should, according to the MIME structure, be hidden.

The people whose mail clients are generating such messages should be complaining to the people who maintain those mail clients and asking them to fix their software so it doesn't generate non-standard emails.
That specific case actually had a Content-ID header given for the attachment, but it doesn't change much from the perspective of this bug. It's a related MIME part which cannot be displayed inline (maybe the motivation in that message was to run the Acrobat plugin or something like that), but which is no longer accessible in the attachment pane either, which is fixed by Jonathan's patch.
Comment on attachment 546374 [details] [diff] [review]
patch to get first body part into attachment pane

we don't allow goto's in new code - perhaps retrieving display.html_as before the top loop would make the code easier to write w/o goto's.
Attachment #546374 - Flags: review?(dbienvenu) → review-
Comment on attachment 545592 [details] [diff] [review]
new patch to add hidden View > Message Body As > All Body Parts functionality

this generally looks good, except that a couple tabs snuck in here:

+                      "bodyAsPlaintext",
+		      "bodyAllParts",
+		     ];

r=me, modulo that.
Attachment #545592 - Flags: review?(dbienvenu) → review+
* Merge prior two patches.
* Eliminate use of goto by restructuring that block of code so it's cleaner.
* Remove unnecessary whitespace changes.
* Remove unnecessary comma at end of JavaScript list.
* Get rid of tabs and add Emacs variable-setting line at the top of the file to prevent it from happening again.
Attachment #545592 - Attachment is obsolete: true
Attachment #546374 - Attachment is obsolete: true
Attachment #546626 - Flags: ui-review?(bwinton)
Attachment #546626 - Flags: review?(dbienvenu)
Attachment #545592 - Flags: ui-review?(bwinton)
Attachment #546374 - Flags: ui-review?(bwinton)
Comment on attachment 546626 [details] [diff] [review]
merge two prior patches and fix code review issues

thx for the new patch.
Attachment #546626 - Flags: review?(dbienvenu) → review+
Blake, can we get a UI review so we can land this?

Is there any possibility of patching earlier releases rather than just landing it on the trunk? There sure are a lot of people complaining about this functionality, so I'm wondering if we should consider it a serious enough issue to try to get the fix out sooner.
If we can land it on the trunk soon, there's a possibility that it will land on the aurora channel, which would mean the fix would ship in around 11 weeks.
Jonathan, here's my review queue: https://bugzilla.mozilla.org/request.cgi?action=queue&requestee=bwinton@mozilla.com

I'm not going to do them in strict order, but since this bug isn't blocking or tracking any given release, it won't be moved too far up in the queue.

Also, _way_ more people are complaining about the Cmd-F change.  ;)

Finally, I thought the next merge from Central to Aurora would be on August 16th…  Were you going to nominate this patch for TB 8 to try and get it in sooner, bienvenu?

Thanks,
Blake.
(In reply to comment #72)
> Jonathan, here's my review queue:
> https://bugzilla.mozilla.org/request.
> cgi?action=queue&requestee=bwinton@mozilla.com
> 
> I'm not going to do them in strict order, but since this bug isn't blocking
> or tracking any given release, it won't be moved too far up in the queue.
> 
> Also, _way_ more people are complaining about the Cmd-F change.  ;)
> 
> Finally, I thought the next merge from Central to Aurora would be on August
> 16th…  Were you going to nominate this patch for TB 8 to try and get it in
> sooner, bienvenu?
Yes, I was saying it was possible we could get it into TB 8. But not promising. And it definitely won't be in TB 7.

- David
>
Err, current trunk /is/ 8.0, thus a few weeks to go before it enters aurora.
Or, in other words, comm-aurora? would nominate it for 7.0 at this time.
I'll review this, possibly tomorrow morning. I've been busy, not even working on Conversations. I just got enough time to do the checkin-neededs yesterday :-).
(In reply to comment #74)
> Err, current trunk /is/ 8.0, thus a few weeks to go before it enters aurora.
> Or, in other words, comm-aurora? would nominate it for 7.0 at this time.

Er, right. Who said rapid release version numbers were confusing!?
Comment on attachment 546626 [details] [diff] [review]
merge two prior patches and fix code review issues

I beat you to it, Protz…  :)

I'm pretty happy with this.  I think showing the "Part 1.2.1" stuff is a little distracting, but having a "Show all parts" which didn't show all parts would be worse.

ui-r=me.

Thanks,
Blake.
Attachment #546626 - Flags: ui-review?(bwinton) → ui-review+
Comment on attachment 546626 [details] [diff] [review]
merge two prior patches and fix code review issues

So I think this bug or the patch needs a checkin-needed flag but I can't figure out how/where tocset it. Perhaps I'm not allowed.
Also, somebody should probably assign the bug to me. :-)
Comment on attachment 546626 [details] [diff] [review]
merge two prior patches and fix code review issues

So I think this bug or the patch needs a checkin-needed flag but I can't figure out how/where tocset it. Perhaps I'm not allowed.
Also, somebody should probably assign the bug to me. :-)
Assignee: nobody → jik
Keywords: checkin-needed
https://bugzilla.mozilla.org/attachment.cgi?id=442657&action=diff is a patch from bug 564423 that disables all Part 1.2-like attachments on the basis that it pollutes the standard message reader. Now that we've got this "View all parts as attachments" feature, maybe it's time to apply the patch? Thoughts?
(In reply to comment #80)
> https://bugzilla.mozilla.org/attachment.cgi?id=442657&action=diff is a patch
> from bug 564423 that disables all Part 1.2-like attachments on the basis
> that it pollutes the standard message reader. Now that we've got this "View
> all parts as attachments" feature, maybe it's time to apply the patch?
> Thoughts?

I think that would be OK as long as the patch in bug 564423 doesn't make the Part #.# attachments invisible when View > Message Body As > All Body Parts is enabled.
There are certainly cases where related content has a Content-ID but no attachment name defined, thus would show up as Part x.y in the list. So, retaining those in the list for "All Body Parts" would be important.
Just checked this in.

http://hg.mozilla.org/comm-central/rev/c1ef44a22eb2
Status: NEW → RESOLVED
Closed: 13 years ago
Keywords: checkin-needed
Resolution: --- → FIXED
Target Milestone: --- → Thunderbird 8.0
Tested with a tinderbox build:
Mozilla/5.0 (Windows NT 5.0; rv:8.0a1) Gecko/20110722 Thunderbird/8.0a1 ID:20110722094828

I have not found an "part" that is not exposed with the new "view all body parts" option. Thanks a million Jonathan, for taking this on. This gives credence to the community concept where others needs are respected and responded to.

I can even see the ubiquitous (if you correspond with an OE user) <bgsound tag attachment data which was previously totally hidden and not accessible.

But...with all improvements, there is always room for more.
With the "plain text part" and the "html part" always showing in the attachment pane, there is a need for an additional option other than "save all"
Save all will save those parts also, and that is probably not what most users want. 

So I propose an additional enhancement to the attachment pane. "Save selected"
Not sure if there is a current bug on this, but it would be quite a valuable feature.
Right click after selection will do this, but with an affirmative response required with each save. 

CC Jim Porter for an opinion.
Status: RESOLVED → VERIFIED
I plan to work on better handling for operating on multiple attachments, but I'm waiting until bug 630759 is finished, since I've already bitrotted the patch there a half-dozen times. That work will probably happen in bug 285997.

Bug 630759 will help things out a little bit though, since it will allow selecting all the attachments with Ctrl+A, deselecting the ones you don't want (via mouse or keyboard), and hitting Ctrl+S to save them.
So as not to procrastinate, I've documented this now at support.mozillamessaging.com, so that the documentation will be available when TB8 ships. In particular, I've:

* edited the Configuration Options for Attachments article to mention how to enable the new menu item;
* added a new Config Editor article to explain how to use the config editor to set a preference directly (since this one isn't available through the UI); and
* added a new Viewing All Body Parts article to explain in detail why one might wish to use this functionality and how to access it.

Now we just need one of the reviewers for support.mozillamessaging.com to review and approve my changes. I don't know how long it usually takes to do that.
Hm, may I add a few late comments here?

1) I've read the whole discussion, but I fail to see the point of showing all parts inside the message view, not just in the attachment pane (unless this is what made fixing this bug easier).
2) Why is the menu item called "All Body Parts" and not "All Message Parts"? Though I guess it does make sense when coupled with 1)...

Either way, I think comment #22 would be a better fix. Too bad that it's much tougher to implement...
(In reply to comment #87)
> 1) I've read the whole discussion, but I fail to see the point of showing
> all parts inside the message view, not just in the attachment pane (unless
> this is what made fixing this bug easier).

I think it's important to be able to see the various body parts inline in the message reader, since they're *body* parts and thus are meant to be shown in a mail client. Since this bug is really about improving handling for malformed messages, I think it's reasonable that we provide users with the maximum amount of information that we can, since there's no way to predict all the various ways that people create malformed messages.
(In reply to comment #88)
> (In reply to comment #87)
> > 1) I've read the whole discussion, but I fail to see the point of showing
> > all parts inside the message view, not just in the attachment pane (unless
> > this is what made fixing this bug easier).
> 
> I think it's important to be able to see the various body parts inline in
> the message reader, since they're *body* parts and thus are meant to be
> shown in a mail client. 

Well, not when they're two multiple/alternative parts. But yes, being able to see them is important. But not all at the same time, which is what this patch implemented.

> Since this bug is really about improving handling
> for malformed messages, I think it's reasonable that we provide users with
> the maximum amount of information that we can, since there's no way to
> predict all the various ways that people create malformed messages.

I don't think we're showing more in that view, than in Plain text + HTML views combined together, are we?

Also, if you get a malformed message, I doubt there would be a big difference in terms of functionality of the affected parts between displaying them inline "somewhere at the bottom" (can we even display a PDF file or a bgsound attachment inline?) and simply making them available from the attachment pane, especially considering that this view is hidden by default and only advanced users are likely to ever get to it.
Before we go changing this feature around more, I think we should get some real information on what kinds of messages people will be using it with (i.e. with Test Pilot). Otherwise, we run the risk of making it harder to handle certain use cases. I have a feeling we could get by without showing all the body parts, but I'd rather know for sure, since the number of ways clients send malformed messages is truly astounding.
(In reply to comment #87)
> 1) I've read the whole discussion, but I fail to see the point of showing
> all parts inside the message view, not just in the attachment pane (unless
> this is what made fixing this bug easier).
Yes, that's exactly why it was fixed this way... It was easy to fix it this way, and much harder to fix it in any other way.
> 2) Why is the menu item called "All Body Parts" and not "All Message Parts"?
> Though I guess it does make sense when coupled with 1)...
Because it's not all message parts. If it were all message parts it would include, e.g., the message header, which is not shown.
I am not sure this is of any use but what the heck?

Emails generated with MS SQLServer Reporting Services also have two MIME parts. The HTML section and the PDF or Excel report as the other part. Exchange webmail, TB 3, and Outlook happily display the PDF or Excel file as an attachment, but the attachment is invisible with TB 5.

The content type header is as follows:

Content-Type: multipart/related;
	boundary="----_=_NextPart_001_01CC4C60.00D3C5D5";
	type="text/html"
(In reply to comment #92)
> Emails generated with MS SQLServer Reporting Services also have two MIME
> parts. The HTML section and the PDF or Excel report as the other part.
> Exchange webmail, TB 3, and Outlook happily display the PDF or Excel file as
> an attachment, but the attachment is invisible with TB 5.

Unless there's a link to the PDF/Excel report from the HTML body, the MIME they're generating is wrong.

Having said that, there's a better fix than the one I implemented. Better, but harder... the UI needs to figure out which attachments in the multipart/related have links to them in the message body, and hide those, while displaying the ones that don't in the attachment list.

The architecture for TB is not at all conducive to this, but it is arguably the most correct UI.
Rimas, let's not over-think this. It's a workaround for certain use cases where you want to access related content (that's Joe's initial motivation for this bug) from the attachment pane, or where a multipart/* message is malformed (apparently frequently encountered) and wants to be treated as multipart/mixed.

It's nothing the user will run into by accident. He or she will have to flip a hidden preference, thus is likely directed to it by support forums or blog/kb posts. Also, toggling View > Display Attachments Inline hides many of the so generated pseudo-attachments in the message view while listing them in the attachment pane below. Thus, I think this works as advertised and desired for now, though based on Jonathan's comment #93 there is certainly room for future improvement to possibly make this a regular feature.

(In reply to comment #84)
> So I propose an additional enhancement to the attachment pane. "Save
> selected" Not sure if there is a current bug on this, but it would be
> quite a valuable feature.

I agree, but this should definitely be a separate enhancement request if it's not filed yet (also of relevance for the regular attachment pane without the special mode introduced here).
(In reply to comment #93)
> 
> Having said that, there's a better fix than the one I implemented. Better,
> but harder... the UI needs to figure out which attachments in the
> multipart/related have links to them in the message body, and hide those,
> while displaying the ones that don't in the attachment list.

Plus anything that cannot be displayed inline natively (even if it has a valid content-ID link from an HTML part) should be listed in the attachment pane to make it accessible. This would cover many of the related-but-not-displayable cases in the duplicates to this bug.
will the embeded HTML be visible as a saveable attachment as well?
(In reply to comment #97)
> will the embeded HTML be visible as a saveable attachment as well?
Yes.
please backport the fix to tb5 (to fix 674473)
I suggest that for multipart/related messages:
- anything that is _not_ used in the message is shown as an attachment,
- anything that is marked with a content disposition of attachment in the message is also shown as an attachment
Jim, Jonathan: Do you want to clone this bug for improvements of the handling of messages which are either using the multipart/related construct wrong or use it for content which cannot be displayed inline, thus allowing to access such parts as regular attachment in the same way as apparently other mail systems do?
(In reply to rsx11m from comment #104)
> Jim, Jonathan: Do you want to clone this bug...

Not right now. Let's wait and see how many people use this feature first (e.g. via a Test Pilot study). There's not much sense in putting large amounts of effort into further improving handling for malformed messages if those messages are very rare.
(In reply to Jim Porter (:squib) from comment #105)
> (In reply to rsx11m from comment #104)
> > Jim, Jonathan: Do you want to clone this bug...
> 
> Not right now. Let's wait and see how many people use this feature first
> (e.g. via a Test Pilot study). There's not much sense in putting large
> amounts of effort into further improving handling for malformed messages if
> those messages are very rare.

Those messages are not rare at all, I've encountered them a few times the last few weeks and it appears that some (newer?) M$ tooling is doing this...
you do the math :-)

from my point of view, compared to TB3 this is a regression
Well, I had no problem finding two reports on GetSatisfaction which were filed within the last 90 minutes, thus supporting comment #106:

 * http://gsfn.us/t/2dykb
 * http://gsfn.us/t/2dym7

The most frequent issues with attachment handling in 5.0 on GS relate to either the issue discussed here, bug 647036 on keeping the pane open, or that users don't know how to open the redesigned attachment pane at all.

As for "wait for Test Pilot study" which ordinary user do you think will install the add-on and participate in such a study? That's hardly representative...
(In reply to rsx11m from comment #107)
> As for "wait for Test Pilot study" which ordinary user do you think will
> install the add-on and participate in such a study? That's hardly
> representative...

Just saying it's not representative doesn't make it so. Unless you have specific evidence that there's a statistical correlation between willingness to install an add-on and the the MIME containers used in messages, then it absolutely *is* representative, since that's how statistics work.

I've seen this remark pretty frequently: that "ordinary" users won't be covered by statistics gathering, so we should instead rely on anecdotal evidence. However, I've yet to see anyone provide evidence for why I should believe that (e.g. polling of Firefox users to determine what kind of users install Test Pilot for Firefox). Especially for things which may be very complicated to code (like this), I think it's reasonable to ask for hard evidence to inform the development process.

We also plan on gathering information about the MIME structures of emails. For instance, how often do multipart/alternative containers have an item other than text/plain and text/html? This information would affect how we decide to change our MIME handling.

Finally, I'm not convinced this is a regression, since comment 0 says that it was previously necessary to view the message as plain text. That's not much different from being required to view all the body parts of the message.

You're welcome to file a bug, I suppose, but I certainly don't plan on looking at it until we have more information.
(In reply to Jim Porter (:squib) from comment #108)
> (In reply to rsx11m from comment #107)
> > As for "wait for Test Pilot study" which ordinary user do you think will
> > install the add-on and participate in such a study? That's hardly
> > representative...
> Just saying it's not representative doesn't make it so. Unless you have
> specific evidence that there's a statistical correlation between willingness
> to install an add-on and the the MIME containers used in messages, then it
> absolutely *is* representative, since that's how statistics work.

Well, not really…  It may or may not be, we don't have the evidence either way.  :)

Later,
Blake.
(In reply to Jim Porter (:squib) from comment #108)
> Just saying it's not representative doesn't make it so.

Just saying that it is won't ensure it either...  ;-)

> specific evidence that there's a statistical correlation between willingness
> to install an add-on and the the MIME containers used in messages, then it
> absolutely *is* representative, since that's how statistics work.

It is likely that advanced users will install an add-on that they are pointed to and thus explicitly consent to a study on this or whatever other topic. You would have to approach the users, either individually or in some way advertised, in the hope that you get a representative sample of users agreeing to be followed in their usage habits of an application. I'm sure that you can point me to a power analysis how many users would have to consent and over which time they would have to be followed to get any significant result from that study?

Obviously, gathering such usage profiles without the user being sufficiently informed and explicitly consenting to it wouldn't be acceptable practice, though to some extent done already for a few parameters, but that's a different story.

> I've seen this remark pretty frequently: that "ordinary" users won't be
> covered by statistics gathering, so we should instead rely on anecdotal
> evidence.

Who says that it has to be "instead"? In general you'd use multiple forms of information gathering to make up your mind, and not just rely on a single tool or number of add-on downloads. That "anecdotal evidence" is not a hard number in the statistical sense, but gives you an idea what problems non-power users run into.
(In reply to Jim Porter (:squib) from comment #108)
> Finally, I'm not convinced this is a regression, since comment 0 says that
> it was previously necessary to view the message as plain text. That's not
> much different from being required to view all the body parts of the message.

well if you feel this way that DO resurrect bug report 674473 since we were able to show there that the attachment in a multipart/related message is NOT accessible EVER in TB5. nothing to do with attachment pane, view message body as plaintext, or whatever.

I'll repeat that again: the attachment in a multipart/related message is NOT accessible EVER in TB5.

and THAT sure is a regression!


sure you could save the complete email as a file and then extract it yourself but that isn't in TB5 now is it???
(In reply to rsx11m from comment #110)
> (In reply to Jim Porter (:squib) from comment #108)
> > Just saying it's not representative doesn't make it so.
> 
> Just saying that it is won't ensure it either...  ;-)
> 
> > specific evidence that there's a statistical correlation between willingness
> > to install an add-on and the the MIME containers used in messages, then it
> > absolutely *is* representative, since that's how statistics work.
> 
> It is likely that advanced users will install an add-on that they are
> pointed to and thus explicitly consent to a study on this or whatever other
> topic.

What I question is whether being an "advanced" user would change the types of multipart messages received by that user. I'm willing to accept that there are correlations between, say, OS used and message types received (a Linux shop probably isn't using Outlook, and so won't have the malformed messages that prompted this bug), but I'd need to see evidence that merely being "advanced" means that you don't receive malformed multipart/related messages. That, of course, depends on us having statistical data in the first place, so that we can analyze it.

> > I've seen this remark pretty frequently: that "ordinary" users won't be
> > covered by statistics gathering, so we should instead rely on anecdotal
> > evidence.
> 
> Who says that it has to be "instead"? In general you'd use multiple forms of
> information gathering to make up your mind, and not just rely on a single
> tool or number of add-on downloads. That "anecdotal evidence" is not a hard
> number in the statistical sense, but gives you an idea what problems
> non-power users run into.

When I suggested that we try to gather more information to be sure that we're fixing this issue in the right way, you said that the data wouldn't be representative, implying that we shouldn't bother gathering the data in the first place.
(In reply to Ferry Huberts from comment #111)
> I'll repeat that again: the attachment in a multipart/related message is NOT
> accessible EVER in TB5.
> 
> and THAT sure is a regression!

A regression already fixed by this bug. Comment 0 says that in TB3, you were required to view the message as plain text to get at the attachments. I haven't seen anything suggesting that TB3 used to display multipart/related subparts as attachments when viewing the message as the original HTML.
well, I downgraded to TB3 immediately when I hit this bug for the n-th time this weekend and TB3 does just fine. I just re-checked, it just does fine. no problems at all. none. zero. 100% ok.

the message pane shows the attachment, when I open the message in a new window the attachment is shown. should I go on? ;-)

squirrelmail shows it right.
evolution shows it right.
mutt shows it right.
....


the point is: email a WAY too important (at least for me) to let this slide. for me personally this is a MAJOR bug. not being able to see my attachments is, well, a major failure of the tooling. I love TB so I just downgraded.
(In reply to Jim Porter (:squib) from comment #113)
> haven't seen anything suggesting that TB3 used to display multipart/related
> subparts as attachments when viewing the message as the original HTML.

Well, as a minimum expectation, you could at least have tried it yourself before commenting. Anyway, bug 674704 attachment 548920 [details] shows the multipart/related PDF part in TB 3.1.11 but not in 3.3a1, the milestone for bug 351224.

To reply to comment #112, I'm questioning in general your over-confidence in such tools while on the other hand not being willing to acknowledge that users have problems with a specific fix or new feature. Indeed, as a potential candidate for such a study, I haven't received any such malformed message whatsoever, whereas others get them on a regular basis. Thus, again you'd need to hope for some sufficiently random distribution of both the phenomenon and your sample to be on the safe side for your statistics. With that I'm closing my arguments in this regards as the statistics discussion is somewhat off-topic.

Reopening bug 674473 as RFE to change the default behavior for such messages to show "unwired" or "undisplayable" MIME parts independently from the fix here, i.e., without the need of toggling a hidden preference IMO should be feasible.
Blocks: 674473
I've reopened and retargeted bug 674473 per previous comment, also given that it contains already pointers to the respective RFC sections.
(In reply to Ferry Huberts from comment #103)
> I suggest that for multipart/related messages:
> - anything that is _not_ used in the message is shown as an attachment,
> - anything that is marked with a content disposition of attachment in the
> message is also shown as an attachment

Yes, this is the behavior that I would like to see as well, but unfortunately is hard to implement, and in fact the "correct" implementation would involve ripping out the internals of the Thunderbird MIME parser and replacing them with an object-based architecture rather than a stream-based architecture. I just don't have the time to do that right now, although perhaps someone else will.

I should be clear that I don't think that's the _only_ way the behavior you propose could be implemented. I think it's probably feasible to keep the current architecture and just modify the multipart/related parser to behave as you proposed. But it's not straightforward, and as the existence of this bug proves, the code is so difficult to maintain that any changes to it would tend to have unanticipated and potentially negative side effects.

FWIW, I think this is a serious enough issue that the patch I wrote for this bug should be applied to an earlier release than TB8, albeit perhaps not TB5, but I just don't have the energy to argue strongly for that.
Jonathan, let's move the discussion about further steps into bug 674473.

As for your patch going into an earlier version, it is my understanding that it can't land on aurora or beta due to the strings. The backend changes could land, but then the user would have to switch three(?) hidden preferences manually to emulate what the menu is doing. Thus, I don't know if this would help much.
More anecdotal evidence: I've had to use this feature just once since it was implemented.
Blocks: 678244
(In reply to Jonathan Kamens from comment #86)
> So as not to procrastinate, I've documented this now at
> support.mozillamessaging.com, so that the documentation will be available
> when TB8 ships. In particular, I've:
> 
> * edited the Configuration Options for Attachments article to mention how to
> enable the new menu item;
> * added a new Config Editor article to explain how to use the config editor
> to set a preference directly (since this one isn't available through the
> UI); and
> * added a new Viewing All Body Parts article to explain in detail why one
> might wish to use this functionality and how to access it.
> 
> Now we just need one of the reviewers for support.mozillamessaging.com to
> review and approve my changes. I don't know how long it usually takes to do
> that.

I truly hate to say this, but our policy is *not* to document hidden preferences (or the config editor) on the Knowledge Base, because changes made through the config editor are not supported. Some of these are documented on MozillaZine. If it's ok with you, I will make sure these prefs are documented on MozillaZine, and then link there from the Thunderbird Knowledge Base (and then reject your changes).

Sorry for the delayed response - still catching up from vacation backlog.
(In reply to jenzed from comment #120)
> I truly hate to say this, but our policy is *not* to document hidden
> preferences (or the config editor) on the Knowledge Base, because changes
> made through the config editor are not supported.

There are several different issues here that I want to comment on:

1. It seems highly sub-optimal to me to have a feature in the UI of the application that is completely and intentionally undocumented. Even if the behavior of TB is undocumented and undefined if people start changing things in the config editor, you should still document the darn thing, so that at least they'll know how to use it properly. My point here is that aside from whether it's appropriate to document the new pref I added, I strongly believe that the config editor itself should be documented.

2. I do not think a blanket prohibition against documenting prefs that need to be changed through the config editor is appropriate. I think it's appropriate on a case-by-case basis to document the prefs that are safe and appropriate to change. In short, I also strongly believe that a categorical policy that "changes made through the config editor are not supported" is unnecessarily broad and counter to the goal of making the app more useful for everyone.

3. Finally, I am exceedingly frustrated to first be told that I have to add this new functionality with a hidden pref, not even being allowed to add it to the Options / Preferences, and then to be told that not only can't the option be settable through the UI, it can't even be documented on in the official TB knowledgebase! For heaven's sake, A WHOLE LOT OF PEOPLE ARE RUNNING INTO THIS PROBLEM, and you're telling me that we're going to fix it, hide the fix, and not tell anyone how to make it work? That's patently absurd.

In short: WTF?
> 3. Finally, I am exceedingly frustrated to first be told that I have to add
> this new functionality with a hidden pref, not even being allowed to add it
> to the Options / Preferences, and then to be told that not only can't the
> option be settable through the UI, it can't even be documented on in the
> official TB knowledgebase! For heaven's sake, A WHOLE LOT OF PEOPLE ARE
> RUNNING INTO THIS PROBLEM, and you're telling me that we're going to fix it,
> hide the fix, and not tell anyone how to make it work? That's patently
> absurd.
> 
> In short: WTF?

I completely agree: what the F*&K are you guys thinking????

The things you are saying make sure that the user has _NO_ indication that part of his email his hidden from view and _NO_ way of knowing on how to make it visible. WTF!!!

This is insane! 

I'm encountering these emails on a regular basis and it **** me off bigtime that TB5 doesn't handle them properly while TB3 does _AND_ that I seem to be forced to wait until TB8 (!!!) to get a completely unsatisfactory and hidden solution.

email is a prime important application for users. hiding parts of emails is unacceptable, even more so when it used to work properly.
and no execuses about other vendors doing it wrong! deal with it. grow up.

imagine your bank deducting an important amount of money from your account without telling. I guess you'd be **** of, wouldn't you?
well I am when I'm not seeing the attachments. and yes, some of those are _very_ important...

stop whining and fix this damn regression because it's not even a bug, it's a damn MAJOR regression.
So I updated to Thunderbird 6 today and see that this regression isn't fixed yet. Actually, I knew this before, since I have been following this bug. However, I also find it comically absurd that rather than trying to get a fix for this out of the door as quickly as possible (let's say...with today's update??!!), you guys are forced to discuss whether to document or not to document this fix for us end users once it's finally coming. Bureaucracy at it's finest :)

So come on, please give us back normal attachment functionality, I'm kinda sick of having to edit my mailboxes in a text editor (have to, since I cannot delete attachments in the normal message interface anymore). Fully agree with the last two comments: TB 5 broke a major expected email feature, and yes, while many users might not notice, we are aware and unhappy that it's broken.
(In reply to Veit Kuehne from comment #123)
> However, I also find it comically absurd that rather than trying to get a fix
> for this out of the door as quickly as possible (let's say...with today's update??!!)

The issue preventing this fix to go into an earlier version is the definition of a new string <!ENTITY bodyAllParts.label "All Body Parts"> along with an access key, which needs to be translated into all languages Thunderbird is released in. This process needs some time until all localizers got a chance to translate this (and other string changes) into their respective languages, which is currently happening. Backend fixes alone can go into a release in its beta phase, if considered relevant enough, but those alone won't allow use of the workaround in a proper way either.
 
> not to document this fix for us end users once it's finally coming.

Yes, that's a self-imposed restriction by the Thunderbird support and documentation people, which personally I don't understand either. This leaves many nice features and fixes in the unknown, and requires users to find a forum first where to ask the question and get a pointer to a hidden preference.

While I don't see the point why articles couldn't be split into basic versus advanced categories and pointed to from respective trouble-shooting articles, I've added a brief follow-up to the 5.0 article in the MozillaZine KB:
http://kb.mozillazine.org/Thunderbird_6.0,_etc.#Images_and_related_content_in_HTML_messages
Thanks for the explanation, rsx11m.

One question: the fix coming in TB8 DOES allow attachments to be deleted in the attachment pane once I've toggled those two prefs?
(In reply to Ferry Huberts from comment #122)
> I'm encountering these emails on a regular basis and it **** me off bigtime
> that TB5 doesn't handle them properly while TB3 does _AND_ that I seem to be
> forced to wait until TB8 (!!!) to get a completely unsatisfactory and hidden
> solution.

You don't have to wait until TB8 is released to everyone. If this feature is extremely important to you, you can download and install a TB8 build with this feature available. Go to http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/ and download the most recent "aurora" build, since TB8 is currently in the aurora channel. I've been running nightly builds for months without any stability issues, so it's probably quite safe for you to do this. Beware, however, that if you use any binary extensions such as enigmail it will be difficult to get them to work with the nightly builds.

Also, to work around all these silly documentation restrictions, I've created an add-on to enable the menu command:

https://addons.mozilla.org/en-US/thunderbird/addon/show-all-body-parts/

Can we document *this* in the KnowledgeBase?
Cool, an addon as documentation ;-)

But seriously, thanks for the explanation! I've stayed away from Betas over the years since I practically live in TB and any bug that starts shredding or messing with my emails is time costly to fix (had a few of those over the years). I prefer others to test them for a while to reduce the risk to my mails ;-) Also, every update messes around a bit with my ton of addons, so I guess I'll wait til 8.
(In reply to Jonathan Kamens from comment #126)
> 
> Also, to work around all these silly documentation restrictions, I've
> created an add-on to enable the menu command:
> 
> https://addons.mozilla.org/en-US/thunderbird/addon/show-all-body-parts/

This says "Works with: Thunderbird 8.0a1 - 9.0a1" thus it only enables the pref and otherwise relies on the patch here? Meaning, not a backport to cover 6.0 and 7.0 releases, which don't have the backend changes included, correct?
(In reply to rsx11m from comment #128)
> This says "Works with: Thunderbird 8.0a1 - 9.0a1" thus it only enables the
> pref and otherwise relies on the patch here? Meaning, not a backport to
> cover 6.0 and 7.0 releases, which don't have the backend changes included,
> correct?

Yes, of course.
Hi, I have an idea that I'm not sure was suggested anywhere in this bug: perhaps Thunderbird's attachment pane could simply have a new context menu item which would enable displaying all body parts for that particular message? If this is indeed possible, not only the two hidden prefs would be unneeded, but also the menu item that was introduced in this bug. Furthermore, this behaviour could be non-sticky in that it would not persist between messages, and yet it would be discoverable easier and would not require the user to go to about:config at all. I think it would be a win-win.

Would that be difficult to implement?
actually that would be lose-lose because I'd have to do this for every message!

bottom line is that if there are body parts not shown, then I want to be notified of this. it doesn't really matter how but I want TB to notify me instead of me having to see whether or not this is the case.

with the current proposals it is completely hidden from the user, which is LOSE-LOSE
Just want to add that I have 25 enterprise users and this issue has popped up with some messages received from Outlook users. The messages have pdf attachments. Maybe this is evidence that the problem is not as unusual as you think. 

Here is a partial sample, in case this helps you at all. I can provide the full message if you want, but it is large.

MIME-Version: 1.0
Content-Type: multipart/related;
	boundary="----=_NextPart_000_0025_01CC616E.57391F80"
X-Mailer: Microsoft Office Outlook 11
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6109

--- clip ---

------=_NextPart_000_0025_01CC616E.57391F80
Content-Type: application/pdf;
	name="Enclos 50 Hawthorne Budget Pricing & Drawings 082211.pdf"
Content-Transfer-Encoding: base64
Content-ID: <1__=07BBF267DFE049E88f9e8a93df938@enclos.com>
Content-Location: 1_multipart%3F2_Enclos%2050%20Hawthorne%20Budget%20Pricing%20%26%20Drawings%20082211.pdf
To all reporters of phenomenon with malformed multipart/related: 

As written in comment #115 and comment #116, bug 674473 already exists for usability problem after bug 351224 with malformed multipart/related mail.
> Reopening bug 674473 as RFE to change the default behavior for such messages
> to show "unwired" or "undisplayable" MIME parts independently from the fix
> here, i.e., without the need of toggling a hidden preference IMO should be
> feasible.
There is no need to add comments to this bug for phenomenon with malformed multipart/related.
Whiteboard: [See bug 674473 for malformed multipart/related specific case]
Im a software developer.. and i have wasted 30 minutes reading through this. first,, being part of the solution instead of the problem doesnt mean the user's need to see archived emails that might be malformed,,, it means any email sent from this program should be formatted correctly. I have a network with more that 40 PC's on it that i converted to TB over 3 years ago. I use many of redhats development tools among others... EVERYONE was ignoring this until the last year when you guys started just not interpreting content just because it didnt match a spec... that i might add was vary poorly written to begin with. I have web apps in production for years that have started failing because TB all of a sudden decided to get strict. Its costing me a lot of time and money to work with the guys at Red hat and the clients,,, to patch API's that i dont even have control over because TB WONT INTERPRET CONTENT THE SAME WAY IT DID A YEAR AGO. guys this is ridiculous and it is severally misguided. I can patch these api's,, but what am i suppose to do about the endless terabytes of archived emails that cant even be view correctly now? you guys are forcing my hand. i can get in line with this spec... and still be in trouble. I can go back to MS and just avoid the problem and everyone is happy. WHAT ARE YOU THINKING??? i dont care if the message is on a postcard and is mirror reversed,,, the ONLY SOLUTION IS THAT ALL CONTENT IN A MESSAGE IS PLACED ON THE SCREEN AT ALL TIMES... i dont want some hidden menu.. some special setting.. my clients are COMPUTER ILLITERATE. its got to be easy.. its got to be on the screen and above all... BEFORE YOU PUSH SOMETHING ON YOUR USERS that you THINK isn't a big deal,,, you need to have ALL backward compatibility issues worked out.. this is a SHOW STOPPER. you've made a mess,, and though you might have fixed it in some future release,, you are losing your base as i write this because you havent pushed a satisfactory solution to production. ive got 40 clients that dont work and cant view a pdf file that they could see find last week. you guys have screwed me.. im going to sleep on it... but unless you stop fussing over the spec... (THATS PURPOSE was to keep us on track and working) and get and update out,,, by the end of the week i will have stopped fighting against the MS beast,, and i will be back in bed with them. All because you guys started fornicating with code that was WORKING FINE before you messed with it.
1. https://bugzilla.mozilla.org/page.cgi?id=etiquette.html
2. If the code had been "WORKING FINE", we wouldn't have changed it. The code was changed because it was impossible to delete attachments from multipart/related email messages.
3. If you're a software developer, then perhaps you can volunteer some of your time to help solve the problem instead of complaining about the rest of us volunteers not working fast enough for you.
no you're right... i shouldnt be complaining without helping. But keep in mind i just spent 12 hours trouble shooting everything but the 40 email clients because on last check,,, i was running a really solid mail client call thunderbird. And about every other post on here was about how this isnt a big deal.. and they are right for all the wrong reasons,, ITS A GREAT BIG DEAL that should have stopped the milestone. i might be a little heated still about the whole situation.

im not sure why deleting attachments from multipart/related email messages is desired or even beneficial unless it is happening during the composition process. if that is the case,,, then why not just delete the message and start a new one until we correctly work out a solution that will keep us backwards compatible. as soon as i get through cleaning up red hat's mail api so that it is spec compliant and cool down a bit, i will download the source to TB and be happy to help you as well.
Severity: major → enhancement
Summary: Inline images not shown in attachment pane when "View>>message body as.plaintext" (after bug 351224, image embedded in multipart/related part is not accessible as attachment) → Inline images not shown in attachment pane when "View>>message body as.plaintext" (after bug 351224, image embed in multipart/related part is not accessible as attachment, so, new "View/Messae Body As/All Body Parts" is needed)
No longer blocks: 685072
Depends on: 685072
I am posting this comment because I searched the page and did not see "hotmail" mentioned.

I've a client who uses hotmail.  When they send attachments they do not show in Thunderbird (6.02 on Win7).  This is the only sender of which I am aware that there are problems. 

If this is not the right place to post this, please let me know.  I can get example messages if anybody wants them.
i think this conversation is taking place over at https://bugzilla.mozilla.org/show_bug.cgi?id=674473

actually there are several enterprise api's that werent following the spec that makes this difficult to change in production. Redhat fixed their from seam mail 2.2.1.CR2 forward, but i know i have numerous sites in production using older revisions. and unfortunately in order to upgrade that lib, we have to due unit testing against it. as a whole the community will migrate back towards the spec, but solving the problem from the other ended isnt as quick a fix and just changing the clients functionality.
No longer blocks: 694337
Thanks for fixing this bug, guys! After installig 8 a couple of days ago, remembered that it should be fixed in this version. Took me a moment to figure out I needed Jonathan's addon (https://addons.mozilla.org/en-US/thunderbird/addon/show-all-body-parts/) to actually see the pref to show all body parts. Just tried it - now I can see images down there even with "Display attachments inline" checked...very nice. Thank you!
Hello, 

Since version 8.0, I cannot see my voice mail attachments in messages created by the PBX. 

These are empty messages with a .wav attachment and the following headers: 

Mime-Version: 1.0 (MessageCore Multimedia Messaging)
Content-type: audio/wav; name=voice.wav; codec=7
Content-Transfer-Encoding: base64
Content-Disposition: inline; filename="voice.wav"
Content-Description: Voice Message (2 secs)

I have to click Forward to see the attachment in the new mail to be composed.

I have tried using the "Show All Body Parts" Add-on, but I don't see any option "Message Body As" --> "All body parts".

Why my attachments vanished?
The issues here are related to Bug 674473, which has been reopened.
(In reply to Nick Milas from comment #142)
[...]
> I have tried using the "Show All Body Parts" Add-on, but I don't see any
> option "Message Body As" --> "All body parts".
> 
> Why my attachments vanished?

In Thunderbird 8.0 and later, SeaMonkey 2.5 and later, there is a menu "View → Message Body As → All Body Parts" in the 3-pane and message-display windows (or tabs) but only if mailnews.display.show_all_body_parts_menu has been set to true (not the default) in the Thunderbird Config Editor (via a button under "Advanced" preferences) or the SeaMonkey about:config. It's a Boolean preference, double-click it to toggle (and type "parts" without the quotes in the Filter box to find it more easily).

No extension required.

When an HTML message is viewed with "All Body Parts", it is also (implicitly) viewed as plaintext.
(In reply to Tony Mechelynck [:tonymec] from comment #144)
> In Thunderbird 8.0 and later, SeaMonkey 2.5 and later, there is a menu "View
> → Message Body As → All Body Parts"

This isn't really relevant to comment 142 (anymore), which is a dupe of bug 701261 and is fixed in 9.0.
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: