97055 - block image from server not available in mail client

Status: VERIFIED FIXED

(SeaMonkey :: MailNews: Message Display, enhancement)

Description

[mwyner (gone)]

A bunch of mailing lists I'm on toss in an ad image at the bottom of the email.
Right clicking on the image does not bring up the ability to do the "block
image" thing. Even if I open the image in a browser window, I can actually block
it from there, but in the mail client, the image still won't be blocked.

Comment 1

[Markus Gerstel]

I'd even go a step further :

An option to disable all images in Mails/News would be nice
(like the -already implemented and VERY useful- option to disable javascript in
Mails/News)

Comment 2

[Christopher Aillon (sabbatical, not receiving bugmail)]

dependent on bug 51266?  This should be in mail/news but adding morse as he does
stuff with image blocking.

Comment 3

[sairuh (rarely reading bugmail)]

not sure who should own this --punt as needed.

Comment 4

[Ninoschka Baca]

Jen, should this be a feature in mail?

Comment 5

[FZiegler]

Isn't this largely a dupe of bug 28327 ?

Comment 6

[Alfonso Martinez]

*** Bug 117183 has been marked as a duplicate of this bug. ***

Comment 7

[R.K.Aa.]

*** Bug 124308 has been marked as a duplicate of this bug. ***

Comment 8

[Alfonso Martinez]

*** Bug 128757 has been marked as a duplicate of this bug. ***

Comment 9

[John Unruh]

Adding nsbeta1 keyword. Under Prefs>Privacy>Images, setting "Do not load any 
images" does not affect the mail window.

Comment 10

[Michael Buckland]

Discussed in Mail News bug mtg w Engineering Mktng PjM.  Decided to minus the bug.  

Comment 11

[shwag]

>Decided to minus the bug.  

What's that mean ?

Comment 12

[Phil Goembel]

If you start adding up all of the votes for all of the bugs which are actually
just variations of this one, you might find a lot of people consider this to be
a  serious issue.

I find it to be an unacceptable invasion of my privacy and a serious security
hole.  By not letting me block the generation of any or all requests from being
generated when I read my email, you are basically forcing me to bend over and
present my rear end to the spammers.

Comment 13

[Tuukka Tolvanen (sp3000)]

*** Bug 131336 has been marked as a duplicate of this bug. ***

Comment 14

[laurel]

*** Bug 133640 has been marked as a duplicate of this bug. ***

Comment 15

[shwag]

"------- Additional Comment #10 From Michael Buckland 2002-03-07 18:49 -------

Discussed in Mail News bug mtg w Engineering Mktng PjM.  Decided to minus the
bug.  "

The people making this decision are probably the same lawyers who decided not to
GPL mozilla the first time around.  Now after a whole lot of wasted effort,
mozilla has almost been GPL'd.  They almost sunk the ship the first time around.
 Will they sink it the second time?  Just because they have the power, doesn't
mean they should wield it around after having a suit meeting over coffee.  Did
they do any market research?

Tell them they need to get their heads out of their asses.  They aren’t making
AOL/Time Warner any more money.  All they are doing is letter a whole bunch of
porn sites and other spammers verify my email address on a daily basis, and
hence deeply lowering their public opinion.  Do they even understand what this
feature is about?  Or do they just think "oh, those lazy users don't like
advertisements. We'll give it to them anyways."

Does AOL/Time Warner really think that the consumer is an idiot?
The consumer is not an idiot.

Please make sure the appropriate people know.

Look what mozilla e-mail’s top competitor on the Linux side is doing.
I guess we're hope Linux doesn't become any more popular.  

Straight from the press:

http://www.linuxworld.com/linuxworld/lw-2000-07/lw-07-vcontrol_4.html
"Dan Winship, one of the Evolution developers at Helix Code, put my mind at ease
when he replied, "GtkHTML does not currently support JavaScript. If it does in
the future, Evolution will disable that. Evolution does not currently fetch
remote images in HTML message bodies, because that can be used by spammers to
verify that you've received the message." "

http://lwn.net/2001/1129/
"The Evolution designers felt that HTML mail support was crucial, but they have
taken a cautious approach to it. The client will not send HTML mail unless
explicitly configured to do so; there is also a feature in the contact manager
which can enable or disable HTML mail on a per-recipient basis. If you chose to
send HTML mail, there is a set of basic formatting options available.

The HTML mail display is also, wisely, configured to never load images off the
net; to do otherwise opens up the user to a number of privacy problems. "

Comment 16

[Alfonso Martinez]

*** Bug 135548 has been marked as a duplicate of this bug. ***

Comment 17

[R.K.Aa.]

*** Bug 136525 has been marked as a duplicate of this bug. ***

Comment 18

[R.K.Aa.]

*** Bug 140288 has been marked as a duplicate of this bug. ***

Comment 19

[R.K.Aa.]

*** Bug 141559 has been marked as a duplicate of this bug. ***

Comment 20

[R.K.Aa.]

*** Bug 141688 has been marked as a duplicate of this bug. ***

Comment 21

[R.K.Aa.]

*** Bug 144006 has been marked as a duplicate of this bug. ***

Comment 22

[scottputterman]

reassigning to ducarroz.  Let's see if it'll be possible to do this for rtm.

from a previous comment:

"Just because they have the power, doesn't mean they should wield it around
after having a suit meeting over coffee."  thanks for the laugh.

Comment 23

[Stephen P. Morse]

There are two separate issues here and it's not clear whether one or both were 
what people had in mind when this was assigned adt1.

a. Adding "(un)block images from this server" to mailnews context menu
b. Adding a pref to image-pref panel for "disable images in mail and newsgroups"

This bug report (as defined in the summary and description field) is about (a) 
only.  But comment 1 morphs this to be (a) and (b).

Strictly speaking, comment 1 should not be considered part of this report and 
belongs in a separate report.  However, I'm guessing that the motivation for 
this becoming adt1 today was the recent review of Netscape 7.0PR1 which said 
"still missing: minor fixes such as the ability to reject cookies from inside an 
e-mail message".  By extension, it would seem like the press would love the same 
thing with images, and that is item (b).

That said, here's how to implement either or both of these items.

a. That is already in cookieContextOverlay.xul and it appears in the navigator 
context menu.  I'm not sure what mechanism is supressing it in the mailnews 
context menu (I did some searching with lxr and couldn't find anything).  But 
there's apparently some simple piece of code somewhere that's suppressing it, 
and anyone familiar with the mailnews overlays will know how to unsupress it.

b. This one I'm more familiar with.  You'll need to do the following:
   - modify pref-images.xul to have a checkbox saying "Disable images in Mail
     and Newsgroups".  This checkbox should be in the image-acceptance-policy
     box, below the radio buttons and above the manage-image-permissions button.
   - add code to nsImages.cpp that processes this pref.  Look at nsCookies.cpp
     for the code that processes cookie_disableCookieForMailNewsPref and put
     simlar code into nsImages.cpp.  There is some code in nsCookies.cpp that
     you will want to share (e.g., cookie_IsFromMailNews() ) which should be 
     moved to nsUtils.cpp and made a public.

Comment 24

[Stephen P. Morse]

The following sentence in the bug description is incorrect:

   Even if I open the image in a browser window, I can actually block
   it from there, but in the mail client, the image still won't be blocked.

This works fine.  But make sure that the images in the mail message and in the 
browser window are from exactly the same server.

I tried it by bringing up www.mozilla.org in the browser and then using the 
context menu to block the banner.  Then I mailed a page to myself that contained 
that banner and opened it in the mail program.  At first I was surprised to see 
that the banner displayed, but then I investigated and discovered that the image 
in the mail message was actually from the server mozilla.org rather than 
www.mozilla.org.  So next I opened mozilla.org (no www) in the browser, the 
image appeared, and I used the context menu to block it.  Now I had blockages 
for both www.mozilla.org and for mozilla.org.  Reloaded the mail message that I 
previously received and the image no longer displayed.

Comment 25

[Jean-Francois Ducarroz]

I will block remote image loaded my email message. But image which the data is
transmitted with the message (like attachment) wont be blocked. Or if you
prefer, spamer HTML message won't display image (they never send the source of
the image for performance reason) but grandma will still be able to see pictures
of her grandchildred.

The problem is in the function nsImgManager::ShouldLoad which takes care only of
base url which the sheme is http or https. I need to add all mail&news sheme:
imap, local, news, snews

ALso, the message display need to listen to the pref
"network.image.imageBehavior" to force a reload of the message when the pref change.

Patch coming soon...

Comment 26

[Stephen P. Morse]

I don't agree with the patch you just proposed.  We don't want to use the 
existing image-behavior pref because that controls whether or not we want to 
download images when browsing.  The user would turn that off if he is on a 
low-bandwidth line for example.  The mail image-blocking should be separate.  
That would be used by people on any-bandwidth line who want to get images when 
browsing but don't want to activate web bugs (hits to spammer's image site when 
reading mail).

So that's why I proposed adding a new checkbox for block-images-in-mail just 
like we do with cookies.  I agree with you that we only want to block external 
images and not the kind we send to grandma.

As far as determining which schema to block, that code already exists in 
cookie_IsFromMailNews.  If you don't want to break out that routine so you can 
share it, at least look at it so you can see what code to duplicate.

Comment 27

[Jean-Francois Ducarroz]

thanks for the tip. I'll use a separate pref...

Comment 28

[Philip Nemec]

I'm glad to see more progress being made!  Let me add there there should be
radio buttons and a checkbox (as there are with web page images and cookies):

Mail Image Policy
( ) Do not load any non-inline images
( ) Accept all inline images

[ ] Ask me before loading any non-inline images

I presume the image permissions list can chare with the web page list, but I'll
leave that open for discussion.

My reasoning is this:

I want to stop spam from loading non-inline images [non-inline means the same as
external - not the ones of grandkids].  :)

I also get "rich" email that I want to load images from (like those from Amazon
or Netflix).  My personal settings would be accept images but ask first - so I
would have some sites blocked/rejected, some sites (like Amazon) accepted, and
be asked before requesting any in response to a new spammer location.

Comment 29

[kmurray1115]

I agree that that level of control would be nice. Is there a better way of
describing the types of images that load/don't load? "Inline" doesn't seem
intuitive/descriptive to me.

Comment 30

[Stephen P. Morse]

> I agree that that level of control would be nice.

I disagree.  This bug is to solve a very specific problem, namely preventing web 
beacons (the ability of a spammer to get your e-mail address when you read a 
message he sent to you).  I'm sure that's what adt had in mind when they gave 
this bug an adt1.  To solve that, all you need is the "block images from mail 
messages" option, just like we have for cookies.  If we stick to just that and 
not try to make it into a power-user feature, there's a good chance we'll get it 
into Mach V.

Comment 31

[Jean-Francois Ducarroz]

I aggree with Steve Morse. Here is the design I am currently implementing
(almost done):

We keep the current pref for web image but just add a new checkbox right after
the radio buttons:

Specify how Mozilla handle images on web pages:
( ) Do not load any images
( ) Accept images that come from the originating server only
(0) Accept all images

[x] Do not load images in Mail & Newsgroups

The reason is that messages are displaying web pages therefore, they should
inherit properties from web page unless say otherwise.

If "Do not load images in Mail & Newsgroups" is check, we will never display
remote images in message. Local image like attachment or local inline images are
always displayed.

If "Do not load images in Mail & Newsgroups" is not checked, then the setting
for image in web pages apply except for Local Inline images and attachment which
are always displayed.

I am open for the exact terminology for the checkbox...

Comment 32

[Philip Nemec]

I'm going to check out the newer builds before commenting more - it seems that
some of the blocking features are working more than they used to...

--------

The phrase "remote images" sounds like the best one so far.

Comment 33

[Jean-Francois Ducarroz]

So far, I have only one technical limitation that I cannot solves:
I cannot block image of an HTML page sent as attachment that we display inline!
However, if I double click to open the attachment in a browser window, then
images are blocked when they are disabled for mail. this is not a big deal as
spamer rarely use attachment as they want to be sure you see the content (most
email client does not display inine attachment) but we need to fix that one day...

Comment 34

[timeless]

you might want to note that people who are concerned can disable view 
attachments inline. (i'm imagining that still works)

Comment 35

[Stephen P. Morse]

It's not the attachments that are the problem.  Those are the "good" images.  
What we want to block is the automatic fetching of images from the web due to 
image tags in the rich mail content.  That's where web beacons come from.

Comment 36

[timeless]

i know, but you're modifying a panel called images which is the descendent of 
being able to show images by default or use the view menu to show them, so if 
poeple really don't want to see images we should give them a hint about how to 
do that.

Comment 37

[Philip Nemec]

> If "Do not load images in Mail & Newsgroups" is not checked, then the setting
> for image in web pages apply except for Local Inline images and attachment which
> are always displayed.

Just as long as having the box unchecked really honors my other image settings
(blocked images, ask before loading) I'm fine with that.  However, this isn't
the current behavior - so I want to make sure both this new checkbox and
honoring the image policy are being worked on.

Comment 38

[Jean-Francois Ducarroz]

right, I am fixing that problem as well...

Comment 39

[jglick]

Some comments

>Specify how Mozilla handle images:
Remove the "on web pages" since this will potentially apply to messages as well.

>( ) Do not load any images
>( ) Accept images that come from the originating server only
>(0) Accept all images

>[x] Do not load images in Mail & Newsgroups

Would "Do no load remote images in Mail & Newsgroup messages" be more accurate? 
Since those are the only types of images that would get blocked? 

Comment 40

[Philip Nemec]

This is clear to me:

[x] Do not load remote images in Mail & Newsgroup messages

Also, make sure you preserve:

[x] Ask me before downloading an image

:)

Comment 41

[Jean-Francois Ducarroz]

Attachment: Proposed fix - cookie part, v1

This patch also include a fix for the mail cookie disabling which forget to
check for secure news (snews)

Comment 42

[Jean-Francois Ducarroz]

Attachment: Proposed fix - mailnews part, v1

Comment 43

[Stephen P. Morse]

Cookie patch looks good.  I especially like the simplification that you made to 
IMAGE_RegisterPrefCallbacks.

However I can't find the added checkbox in the image pref panel for disabling 
images in mailnews.  I'm I missing something or did you forget to include that?

Comment 44

[Jean-Francois Ducarroz]

it's in the mailnews part of the fix...

Comment 45

[Stephen P. Morse]

Comment on attachment 85350 [details] [diff] [review]
Proposed fix - cookie part, v1

r=morse

Comment 46

[racham]

Comment on attachment 85351 [details] [diff] [review]
Proposed fix - mailnews part, v1

minor nit :
+<!ENTITY disableImageInMailNews.label	     "Do not load remote images in Mail
&amp; Newsgroup">

==>

+<!ENTITY disableImageInMailNews.label	     "Do not load remote images in Mail
&amp; Newsgroup messages">

r=bhuvan

Comment 47

[Jean-Francois Ducarroz]

Attachment: Proposed fix - mailnews part, v2

change checkbox label to: "Do not load remote images in Mail & Newsgroup
messages"

Comment 48

[Jean-Francois Ducarroz]

Comment on attachment 85458 [details] [diff] [review]
Proposed fix - mailnews part, v2

r=bhuvan

Comment 49

[Alec Flett]

Comment on attachment 85458 [details] [diff] [review]
Proposed fix - mailnews part, v2

sr=alecf

Comment 50

[Alec Flett]

Comment on attachment 85350 [details] [diff] [review]
Proposed fix - cookie part, v1

"waist" -> "waste"

Also, instead of a giant set of nested if()'s, can you break that part into a
seperate function, and maybe even extract the scheme once instead of calling
SchemeIs? (SchemeIs is faster for a one-time check, but if you're constantly
comparing, you might as well extract it)

I'm thinking something like

PRBool ShouldBlockImageByScheme(nsIURI * aURI) {

nsCAutoString scheme;
GetScheme(scheme);

return	(scheme == NS_LITERAL_CSTRING("https"))  || (scheme ==.....);
} etc..

but actually, why do you need to check for these image types? won't the
"server" in this case always be the mail server? won't that effectively block
all M-MIME images (or whatever its called when the image is stored as a mime
attachment)

also, you're calling IMAGE_BlockedInMail() twice for every image in mail - in
the inner loop, shouldn't it always be true?

Comment 51

[Jean-Francois Ducarroz]

>but actually, why do you need to check for these image types? won't the
>"server" in this case always be the mail server? won't that effectively block
>all M-MIME images (or whatever its called when the image is stored as a mime
>attachment)

I am not sure why we need to check for the sheme of the base url anyway! Steve
Morse, can you explain?

Comment 52

[Stephen P. Morse]

I'm not sure I'm understanding Alec's comment.

The routine in which this code is contained is called when blocking images in 
general so it indeed deals with http and https cookies.  What's new is that the 
mailnews code is now using this routine that already existed.  Furthermore, 
won't the image be an http/https protocol if the mail message contains a link to 
a remote image somewhere on the web?  The case of images stored as 
mime-attachments are the good images (the ones to grandma) that we don't ever 
want to block.

Comment 53

[Jean-Francois Ducarroz]

What I still don't understand is why we need to check the scheme of the base
url. We are already checking the image url itself!

Comment 54

[Stephen P. Morse]

Ah, yes, I misunderstood the code here.

I think the reason we do that is to be able to reject third-party images.  Of 
course in the mail case the base URL is always mail.  But in the general 
image-blocking scenerio it is not.

Comment 55

[Jean-Francois Ducarroz]

Attachment: Proposed fix, - cookie part, v2

I have addressed all the valid point mentianned by alecf.

Comment 56

[Alec Flett]

Comment on attachment 85540 [details] [diff] [review]
Proposed fix, - cookie part, v2

ok, this looks reasonable.
sr=alecf

Comment 57

[Stephen P. Morse]

Comment on attachment 85540 [details] [diff] [review]
Proposed fix, - cookie part, v2

r=morse

Comment 58

[Jean-Francois Ducarroz]

Attachment: Proposed fix - mailnews part, v3

Same as previous patch, I have just corrected the following line which was
missing the closing double quote:
+pref("mailnews.message_display.disable_remote_image", false);

Comment 59

[Jean-Francois Ducarroz]

Comment on attachment 85792 [details] [diff] [review]
Proposed fix - mailnews part, v3

from previous patch:
r=bhuvan
sr=alecf

Comment 60

[Jean-Francois Ducarroz]

Fix checked in the trunk

Comment 61

[Carl-Daniel Hailfinger]

Does this also solve the problem of loading CSS/Flash/whatever from an external
server or is it just for peace of mind regarding images?

Comment 62

[Jean-Francois Ducarroz]

I haven't tested remote css image. Flash image are not blocked, for that you
need to disabled plugin in mail, that's part of another feature we are working on.

Comment 63

[scottputterman]

Olga, could you verify this today, if possible, on the trunk?

Comment 64

[scottputterman]

JF, can you check in the string changes today.  After verification, we'll give
approval for the rest.

Comment 65

[Jean-Francois Ducarroz]

Attachment: String changes for the branch

Just the UI string changes for the branch. We need to check those in the branch
before the UI freeze

Comment 66

[Jean-Francois Ducarroz]

Comment on attachment 86140 [details] [diff] [review]
String changes for the branch

from previous patches
r=bhuvan, morse
sr=alecf

Comment 67

[Terri Preston]

I would verify this on trunk if I could find it :-( Could some one tell me where
I should see this preference?

Comment 68

[Philip Nemec]

Preferences / Privacy & Security / Images

[X] Do not load remote images in Mail & Newsgroup messages

I see it in 2002060308.

However, the addition of this seems to have bumped off the checkbox to 
ask for every image!  Should I file a separate bug on this?

Comment 69

[Stephen P. Morse]

The removal of the warn-me-before-downloading box has nothing to do with this 
checkin.  See bug 110112, bug 145051, and bug 146513 for details.

Comment 70

[Terri Preston]

I did not see this in yesterday's trunk build or today's trunk 2002060408, win
XP?????

Comment 71

[Olga]

I am working on Win2K, 2002-06-04-08-trunk build - I don't see new expected
Preferences options. 

Comment 72

[Jean-Francois Ducarroz]

I have donwloaded today's trunk build and the preference is missing!!! Did
something brake already?

Comment 73

[Jean-Francois Ducarroz]

Something wierd here, the panel Privacy&Security/Images is missing the "Accept
image that come from the original server only" radio button and the "Manage
Image Permissions" puch button!

morse, any idea?

Comment 74

[Stephen P. Morse]

Are you running a mozilla build or a commercial build.  The 
originating-server-only button does not appear in the commercial build because 
image-management is off by default.  You can get it on in the commercial build 
by manually inserting a pref into prefs.js (there is no UI for that pref).

Comment 75

[Jean-Francois Ducarroz]

thanks steve for the tip, that explains why it works fine on my debug Mozilla
build but not on Commercial release build. Looks like the code to remove those
buttons interferes with the mailnews overlay...

Comment 76

[Jean-Francois Ducarroz]

Attachment: Fix for commercial build, v1

I fix the problem on commercial build when the pref "imageblocker.enabled" is
set to false causing some of the element to be hidden.

Comment 77

[Stephen P. Morse]

Comment on attachment 86325 [details] [diff] [review]
Fix for commercial build, v1

r=morse

Comment 78

[David :Bienvenu]

Comment on attachment 86325 [details] [diff] [review]
Fix for commercial build, v1

sr=bienvenu

Comment 79

[David :Bienvenu]

Comment on attachment 86325 [details] [diff] [review]
Fix for commercial build, v1

sr=bienvenu

Comment 80

[Jean-Francois Ducarroz]

Fix in trunk (again)

Comment 81

[Christopher Blizzard (:blizzard)]

a=blizzard on behalf of drivers for 1.0.1 for the fix and any string related
changes.

(go, go, go!)

Comment 82

[Judson Valeski]

please land on the 1.0.1 branch. once there, remove "mozilla1.0.1+" from the
keyword field, and add "fixed1.0.1".

Comment 83

[Jean-Francois Ducarroz]

I sent an email to mcarlson requesting approval for the UI changes...

Comment 84

[Michele Carlson]

l10n approved. Please check this into the branch today. thanks

Comment 85

[Olga]

Verified on today's trunk build, Win2K.
Remote link is not shown in the mail, but still appears in the browser when I
saved that message as .eml and then opened in the browser.
After unchecking new Pref option, remote image appears in the Mail.

Comment 86

[scottputterman]

adding adt1.0.1+ for checking into 1.0.1.  Please check into the branch ASAP.

Comment 87

[Jean-Francois Ducarroz]

Fix checked in the branch

Comment 88

[Olga]

I checked 06/07 today's branch build. It is not there yet. I'll check next build
on Monday.

Comment 89

[Jean-Francois Ducarroz]

strange, it should be! what's the build ID you are checking?

Comment 90

[Olga]

Verified on branch 06/10/02, Win2K, linux, Mac OSX.