Closed Bug 1235974 Opened 8 years ago Closed 8 years ago

Remote content blocking is broken

Categories

(Thunderbird :: Untriaged, defect)

Product:
Thunderbird
Component:
Untriaged
Version:
38 Branch
Platform:
x86_64
Linux
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED INVALID

People

(Reporter: u328884, Unassigned)

Details

Attachments

(2 files)

Keep Your Garage Warm All-Winter-Long with a Garage Heater.eml
7.91 KB, application/x-extension-eml
Details
A newly received phishing expedition, same as 1st one.
8.60 KB, text/plain
Details 
User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:25.7) Gecko/20150929 Firefox/31.9 PaleMoon/25.7.2
Build ID: 20150929234510

Steps to reproduce:

Here is where remote content blocking is failing in the attached message 'Keep Your Garage Warm All-Winter-Long with a Garage Heater.eml':

<img id="heater_01" src="http://bit.ly/1ICjV1N" width="600" height="133" alt="" />



Actual results:

TBird is fetching '1ICjV1N.htm' (which is actually a JPG).


Expected results:

TBird should NOT be fetching ANYTHING!!!

I don't trust TBird. I'd prefer HTML mail to NOT BE RENDERED AT ALL. I prefer just the source HTML to be shown.
I don't see this behavior. The image you reference is correctly hidden in Thunderbird (I'm on a fairly-recent nightly on Linux if this affects things).

If you'd prefer to not see any HTML email, go to View -> Message Body As -> Plain Text. In this case, that will downconvert the HTML body to plain text. (In other cases where there's a plain text alternative in the message already, it should just use that alternative.)
(In reply to Jim Porter (:squib) from comment #1)
> I don't see this behavior. The image you reference is correctly hidden in
> Thunderbird (I'm on a fairly-recent nightly on Linux if this affects things).

Not in my TBird (38.4.0 running in Linux Mint 17.2). It is fetched and rendered.

> If you'd prefer to not see any HTML email, go to View -> Message Body As ->
> Plain Text. In this case, that will downconvert the HTML body to plain text.

Down-conversion is another issue entirely. ... Unrelated I believe.
OS: Unspecified → Linux
Hardware: Unspecified → x86_64
(In reply to Mark Filipak from comment #2)
> Not in my TBird (38.4.0 running in Linux Mint 17.2). It is fetched and
> rendered.

I tried it in 38.5.0 on Windows 7 and 38.4.0 on Linux Mint and the content blocker works correctly in both cases. There's likely something wrong with your configuration. Do you have remote content allowed (in general, for that sender, or for that domain)? Check your settings in Preferences -> Privacy.
I tried it with 38.4.0 and Earlybird 45.0a2 on Linux Mint, and I don't see the image with Thunderbird configured to view Message Body As Original HTML, and Allow remote content in messages not checked. I do see a placeholder.

Expected behavior.
(In reply to Jim Porter (:squib) from comment #3)
> (In reply to Mark Filipak from comment #2)
> > Not in my TBird (38.4.0 running in Linux Mint 17.2). It is fetched and
> > rendered.
> 
> I tried it in 38.5.0 on Windows 7 and 38.4.0 on Linux Mint and the content
> blocker works correctly in both cases. There's likely something wrong with
> your configuration. Do you have remote content allowed (in general, for that
> sender, or for that domain)? Check your settings in Preferences -> Privacy.

This seems to be a moving target. The folks at bitly are apparently intercepting this and are redirecting to an actual '.jpg' file. I don't see how that would change TBird's behavior, but it apparently is.

Of course, I do not have a privacy exception that would explain this. My exceptions are:
email@SpeedyRewards-email.com
newsletters@cnet.online.com
www.mdanderson.org
(In reply to WaltS48 from comment #4)
> I tried it with 38.4.0 and Earlybird 45.0a2 on Linux Mint, and I don't see
> the image with Thunderbird configured to view Message Body As Original HTML,
> and Allow remote content in messages not checked. I do see a placeholder.

With the exact same settings, I do see the picture (or at least, I did when I first received the message).

> Expected behavior.

What do you mean?
Just to be totally safe, I tried running Wireshark and then loaded the message. No relevant network activity happened until I clicked "show remote content". At that point, Wireshark logged a query to bit.ly. Since Thunderbird didn't initially send any requests to bit.ly, I doubt anything on their end affects this (at least not on my system).
(In reply to Jim Porter (:squib) from comment #7)
> Just to be totally safe, I tried running Wireshark and then loaded the
> message. No relevant network activity happened until I clicked "show remote
> content". At that point, Wireshark logged a query to bit.ly. Since
> Thunderbird didn't initially send any requests to bit.ly, I doubt anything
> on their end affects this (at least not on my system).

My TBird must have send a request to bit.ly. How else would the picture have been included?
Sorry about that last message. There doesn't appear to be any way to edit it.

What I meant to write was this:

My TBird must have sent a request to bit.ly. How else would the picture have been rendered so I would see it?
I'm not disputing what your computer did, but that doesn't help isolate why it happened. Since I'm reasonably sure my installation is acting as it should (and WaltS48 can confirm what I'm seeing), there must be something wrong (or at least different) about your installation. Until I know what the difference is, there's no way I can reproduce the issue, and thus I stand no chance of being able to resolve it for you.

Luckily, given that multiple people have tested this out and it behaves correctly for them, this probably isn't a widespread issue affecting large numbers of users.
(In reply to Jim Porter (:squib) from comment #10)
> I'm not disputing what your computer did, but that doesn't help isolate why
> it happened. Since I'm reasonably sure my installation is acting as it
> should (and WaltS48 can confirm what I'm seeing), there must be something
> wrong (or at least different) about your installation.

Of course. My computer is in my house. Your computer is where you are. They of course will be different. So........

> Until I know what the
> difference is, there's no way I can reproduce the issue, and thus I stand no
> chance of being able to resolve it for you.

So, help me. How can I help you? My mission at bugzilla is to get TBird fixed. How can I do that?

> Luckily, given that multiple people have tested this out and it behaves
> correctly for them, this probably isn't a widespread issue affecting large
> numbers of users.

Or the phisher has come up with a method that can trick TBird somehow, or exploits some hitherto unknown defect or foible.
(In reply to Mark Filipak from comment #11)
> > Until I know what the
> > difference is, there's no way I can reproduce the issue, and thus I stand no
> > chance of being able to resolve it for you.
> 
> So, help me. How can I help you? My mission at bugzilla is to get TBird
> fixed. How can I do that?

From simplest to hardest:

0) Try replacing the image URL in the message with something that's hasn't been blocked to make subsequent tests easier. Obviously, make sure you can still reproduce the bug.
1) Try testing the message in local folders vs IMAP.
2) Try disabling all add-ons (even ones that don't seem like they could affect this) and see if it still happens.
3) Try creating a new Thunderbird profile for testing and see if you can reproduce it there.
4) Examine your prefs.js file (located in ~/.thunderbird/<your-profile>) and see if anything looks wrong. Most of the prefs should be documented online. Alternately, anonymize its contents and attach it to the bug.
5) Try copying the message to a different computer with Thunderbird and seeing if it still happens there.

> > Luckily, given that multiple people have tested this out and it behaves
> > correctly for them, this probably isn't a widespread issue affecting large
> > numbers of users.
> 
> Or the phisher has come up with a method that can trick TBird somehow, or
> exploits some hitherto unknown defect or foible.

If that were the case, I expect it would be reproducible on other systems, but I've tried 3 different ones and all behave properly. Inspection of the HTML also shows nothing unusual.
I'm sorry to say that I can't replicate this problem. That's bad news for TBird.

I cannot replicate the problem, and the reason I can't replicate it
exposes a previous mystery.

The HTML image element was:

<img id="heater_01" src="http://bit.ly/1ICjV1N" width="600" height="133"
alt="" />

That HTML line was in the email.

To cut to the end, the mystery is: How can 'http://bit.ly/1ICjV1N' fetch
'1ICjV1N.jpg'?

Ans: It can't (but it did when '1ICjV1N.jpg' was at 'bit.ly').

Confused? Let me explain. The only way a reference to 'bit.ly/1ICjV1N'
can return an image ('1ICjV1N.jpg' in this case), instead of a 404 (page
not found) is if the HTTP server has a configuration file that defines
'1ICjV1N' as am image that can be fetched (instead of as a folder or as
an HTML page). So, the only way the original email could have functioned
as it did function (until bitly pulled the image) was if bitly was in on
the phishing expedition. (Gee, I hope you're following this, because I
know it gets awfully complicated.)

I put '1ICjV1N.jpg' on 'markfilipak.com' and changed the <img>-element to:

<img id="heater_01" src="http://markfilipak.com/1ICjV1N" width="600"
height="133" alt="" />

My server runs vanilla Apache. When I opened the email as a file ('File'
> 'Open saved message...'), the 600x133 place where the <img>-element
was supposed to be was blank. That should be expected, but when I
originally got the email a couple of days ago, it wasn't blank. That
also indicates that bitly's HTTP server was in on the phishing expedition.

At any rate, bitly has now blocked that image (and presumably has
reconfigured its HTTP server), so my original symptoms cannot be replicated.

That all leaves just one question:
What was bitly doing that got TBird to fetch that image even though I
had configured TBird to not fetch remote content?

Does anyone have any ideas?
Quick, before bitly kills the target of the link. Link is:
<img id="coffee_01" src="http://bit.ly/1JMb1tk" width="600" height="86" alt="" />
-- that 'src' is being fetched even though I have remote content blocked.
Thunderbird still behaves correctly for me. I've confirmed that there's no network activity in Wireshark until *after* I click "show remote content".
Thanks, Jim.

I just solved the mystery. It's because remote content blocking is not global. It's local only to email accounts. Perhaps there's a workaround you can suggest after you read about how I'm using TBird (and why).

When I need a 'custom junk' account, I create a bogus news feed or 'other' account. Then I create an 'inbox' subfolder for it, so it then shows up with the other inboxes. Then, my filters move selected 'junk' messages from my various accounts' inboxes into that single, 'custom junk' inbox. It's only when the message hits that 'custom junk' inbox that remote content blocking is ignored.

Do you get what I'm doing? Do you understand why I'm doing it? Do you have a better way?
To clarify...

If I move the subject email message to an actual email account's inbox, remote content blocking works. But if I move the subject email message to a phoney, non-email account's inbox (that is, a news feed's 'inbox' or an 'other' account's 'inbox'), then remote content blocking is not active.

Since remote content blocking is in Preferences, I assumed it had global, application-wide effect. Yes, I know the Preference says "Mail Content" "[_] Allow remote content in messages", and therefore it could easily be argued that it applies only to mail (as opposed to any other type of account), but I didn't get that limitation and I don't think it's reasonable to expect anyone to make that distinction. After all, there is such a thing as user expectation.

TBird is one of those applications that blur the line: Is it a mail client? Is it an Internet Chat agent? Is it a News agent? ...you get the idea. This blurring makes it harder to meet user expectations for the application. The result is user confusion. The result is the need for excessive application tutorials. The result is ...mistakes ...like what I made.

Regards to all, And Happy New Year.
Status: UNCONFIRMED → RESOLVED
Closed: 8 years ago
Resolution: --- → INVALID
RSS has a totally different threat model than email, since unless someone has hacked the site hosting the RSS feed, only people who you trust can send you RSS items. That means that it's generally safe to load remote content for RSS.

I think you'd probably have better luck if you just used one Junk folder per account and used the "Unified Folders" view where you can see all the junk messages for every account. Alternately, if you don't like that mode, just move them all to a folder in the "Local Folders" account, which is a mail account and thus subject to the remote content blocking rules. I don't think creating a dummy account gives you much that either of these solutions wouldn't.
Hi Jim,

(In reply to Jim Porter (:squib) from comment #18)
> RSS has a totally different threat model than email, since unless someone
> has hacked the site hosting the RSS feed, only people who you trust can send
> you RSS items. That means that it's generally safe to load remote content
> for RSS.

But the stuff in the inbox is not RSS. It's email that I've moved there, so loading remote content is not okay. Actually, I used a bogus RSS account once, but the latest creation was a bogus Feed account (whatever that is)... "Add Feed Account..." I picked that because it didn't require an email address. Then I created an 'inbox' folder and -- voila! -- it appeared in my unified folders, inboxes. If I need another custom-junk folder, I create another bogus Feed account and another 'inbox'. Et cetera.

I hope you're really following all this, Jim.

I'm doing this because TBird is so limiting. ...I wish I could just drag the folders I want to see to some sort of drop pad, after which they would show up and the others would be hidden behind (require) "clicks". What I want to see might be an inbox, or it might be just some undistinguished folder that I move stuff to via one or more filters.

I don't know why TBird treats inboxes as special. And I don't know why TBird as sent folders at all. The mailbox structure is soooooo..... cumbersome.

> I think you'd probably have better luck if you just used one Junk folder per
> account and used the "Unified Folders" view where you can see all the junk
> messages for every account.

I don't know what you mean, Jim. The vertical space in my computer is completely taken up just displaying my inboxes (I have about 45 accounts). How would I display junk folders, too. And what about the other stuff that's technically not junk, but I don't want to leave in an inbox? Using TBird is like wearing a straight jacket.

> Alternately, if you don't like that mode, just
> move them all to a folder in the "Local Folders" account, which is a mail
> account and thus subject to the remote content blocking rules.

That's what I'm doing, but I can only do that once because there can be only one 'inbox' (I think).

> I don't think
> creating a dummy account gives you much that either of these solutions
> wouldn't.

Creating multiple dummy accounts gives me much, [much, ...] more flexibility.

Oh, by the way, it appears that bugzilla has removed this thread from the list of my bugs that are displayed in response to clicking "My Bugs"
(
https://bugzilla.mozilla.org/buglist.cgi?bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&emailassigned_to1=1&emailreporter1=1&emailtype1=exact&email1=MarkFilipak.mozilla%40gmail.com&field0-0-0=bug_status&type0-0-0=notequals&value0-0-0=UNCONFIRMED&field0-0-1=reporter&type0-0-1=equals&value0-0-1=MarkFilipak.mozilla%40gmail.com&list_id=12765194
)
so I don't know whether I'll be receiving notice of any future additions to this thread.
(In reply to Mark Filipak from comment #19)
> But the stuff in the inbox is not RSS. It's email that I've moved there, so
> loading remote content is not okay.

But you've imbued that message with RSS properties by moving it there. Arguably, it shouldn't be possible to move messages between account types with different security models, but that's a pretty narrow edge case, so I'm not surprised Thunderbird doesn't account for it.

> Actually, I used a bogus RSS account once, but the latest creation was a bogus
> Feed account (whatever that is)...

Those are the same thing.

> I'm doing this because TBird is so limiting. ...I wish I could just drag the
> folders I want to see to some sort of drop pad, after which they would show
> up and the others would be hidden behind (require) "clicks". What I want to
> see might be an inbox, or it might be just some undistinguished folder that
> I move stuff to via one or more filters.

Have you tried using the "Favorite" or "Unified" folder modes? It seems like you're already using Unified, since that would explain your desire to have things be inboxes, but it's not clear. Unified folders mode has a special top-level section for Junk folders, so it should be easy to go through all your junk mail without having to create a dummy account for it.

> I don't know what you mean, Jim. The vertical space in my computer is
> completely taken up just displaying my inboxes (I have about 45 accounts).
> How would I display junk folders, too.

You don't have to display all of them. Unified folders creates a top level menu item for each of the "common" folder types (Inbox, Drafts, Junk, etc). You could just show all your inboxes and then leave the top-level Junk folder collapsed. That folder is effectively a virtual folder that shows the junk from *every* junk folder across all your accounts. That sounds like it gives you what you want.

> > Alternately, if you don't like that mode, just
> > move them all to a folder in the "Local Folders" account, which is a mail
> > account and thus subject to the remote content blocking rules.
> 
> That's what I'm doing, but I can only do that once because there can be only
> one 'inbox' (I think).

Why do you need multiple cross-account junk folders?
 
> Oh, by the way, it appears that bugzilla has removed this thread from the
> list of my bugs that are displayed in response to clicking "My Bugs"

That's because the bug is resolved. Resolved bugs don't show up in that query.
(In reply to Jim Porter (:squib) from comment #20)
> (In reply to Mark Filipak from comment #19)
> > But the stuff in the inbox is not RSS. It's email that I've moved there, so
> > loading remote content is not okay.
> 
> But you've imbued that message with RSS properties by moving it there.

I realize that now. TBird let me do something that was undesireable without warning me. But rather than build-in more warnings (trying to make TBird smarter & smarter), how about just making TBird simpler: Email only? And do away with "inbox". Let me name folders to anything I want. Let me display what I want and put everything else behind clicks? TBird is way too heavily structured.

> Arguably, it shouldn't be possible to move messages between account types
> with different security models, but that's a pretty narrow edge case, so I'm
> not surprised Thunderbird doesn't account for it.

Simplify. People will love you for it. Do away with various account types with differing security models. Just do mail, and do it well. And let me automatically route a copy of outgoing mail to wherever I want, and automatically route incoming mail to wherever I want. And let me display some (arbitrary) folders so I can monitor them while I hide other folders behind a click.

> > Actually, I used a bogus RSS account once, but the latest creation was a bogus
> > Feed account (whatever that is)...
> 
> Those are the same thing.
> 
> > I'm doing this because TBird is so limiting. ...I wish I could just drag the
> > folders I want to see to some sort of drop pad, after which they would show
> > up and the others would be hidden behind (require) "clicks". What I want to
> > see might be an inbox, or it might be just some undistinguished folder that
> > I move stuff to via one or more filters.
> 
> Have you tried using the "Favorite" or "Unified" folder modes? It seems like
> you're already using Unified, since that would explain your desire to have
> things be inboxes, but it's not clear. Unified folders mode has a special
> top-level section for Junk folders, so it should be easy to go through all
> your junk mail without having to create a dummy account for it.
> 
> > I don't know what you mean, Jim. The vertical space in my computer is
> > completely taken up just displaying my inboxes (I have about 45 accounts).
> > How would I display junk folders, too.
> 
> You don't have to display all of them. Unified folders creates a top level
> menu item for each of the "common" folder types (Inbox, Drafts, Junk, etc).

Could we just toss out the thinking of "folder types"? Let me create arbitrary folders and let me customize how I do things instead of trying to get me to do things the way some programmers think I want to do things. This reminds me of Microsoft's distruction of their Start menu. With Windows 98 through XP, the Start menu was freeform. I could put anything in it. If it was a program, the icon became a program launcher. If it was a text file, the icon opened the text file in the defaut text file program (usually an editor, but that could be changed). If it was a picture, ... (you get the idea). If it was a folder, then the icon because an entire 2nd level of menu (not simply something that open a file browser window). If inside that 2nd level menu, there was another folder, then its icon in the 2nd level menu became an entire 3rd level of menu. ... In WinXP, I had about 7 or 8 levels of menu in some cases -- some cases were single- or 2- or 3-level menus, the point was that I had complete architectural control. Windows now (7, 9, 10) have just a single-level menu supporting just simple program launchers. Linux is the same way. It's awful.

Well, TBird is like a straight jacket... like Windows 7/8/10 & all Linux versions I've tried are straight jackets.

Computers are getting harder to use and have been getting harder for over 10 years.

I should tell you that I don't speak from a position of ignorance. I'm a retired electronics engineer who worked designing hardware in Silicon Valley for 25 years (1977-2001) and can program productively in over 10 languages. I was involved in computer system architecture at Intel, Altos Computers, Wyse Tech., Atari, NAC, among other jobs (plus consulting). Once upon a time, it was possible make an HTML your Windows desktop and then program it like it was a web page. I programmed mine to be a navagable 3-D trip down a cartoon inner-city alley (think of the movie "Heavy Traffic") -- like a video game. Various objects kicking around in the alley were clickable and were my interface with things in the operating system (launchers, menus, etc.). It was a gas. I'd show it to people and they'd be amazed. You can't do anything like that anymore with any computer. They have just become a lot less fun. Until I showed them how to do it, people couldn't believe that users could create such fun user interfaces in Windows.

> You could just show all your inboxes and then leave the top-level Junk
> folder collapsed. That folder is effectively a virtual folder that shows the
> junk from *every* junk folder across all your accounts. That sounds like it
> gives you what you want.
> 
> > > Alternately, if you don't like that mode, just
> > > move them all to a folder in the "Local Folders" account, which is a mail
> > > account and thus subject to the remote content blocking rules.
> > 
> > That's what I'm doing, but I can only do that once because there can be only
> > one 'inbox' (I think).
> 
> Why do you need multiple cross-account junk folders?

If you stop thinking of them a junk folders, I think you'll begin to get the idea. They're just folders I want to monitor at the top level, without having to do clicks to get to them. I don't even want them associated with particular accounts. Accounts are too constraining. I want 'places' that may fetch mail from accounts or connect to accounts to send messages, but otherwise are just 'places'. I know it all sounds uncontrolled (and it is), but it actually makes for a simpler programming architecture. Of course, if all the folders were maildirs, then TBird could/might jetison the jquery.

> > Oh, by the way, it appears that bugzilla has removed this thread from the
> > list of my bugs that are displayed in response to clicking "My Bugs"
> 
> That's because the bug is resolved. Resolved bugs don't show up in that
> query.

Well, that's just silly.

Ciao, Jim.
(In reply to Mark Filipak from comment #21)
> (In reply to Jim Porter (:squib) from comment #20)
> > (In reply to Mark Filipak from comment #19)
> > > But the stuff in the inbox is not RSS. It's email that I've moved there, so
> > > loading remote content is not okay.
> > 
> > But you've imbued that message with RSS properties by moving it there.
> 
> I realize that now. TBird let me do something that was undesireable without
> warning me. But rather than build-in more warnings (trying to make TBird
> smarter & smarter), how about just making TBird simpler: Email only?

While I wouldn't mind seeing RSS pulled out into an add-on (perhaps installed by default), that doesn't actually resolve your issue. People who don't use RSS don't see much sign of its existence in Thunderbird, except as an additional option when creating an account.

> > Arguably, it shouldn't be possible to move messages between account types
> > with different security models, but that's a pretty narrow edge case, so I'm
> > not surprised Thunderbird doesn't account for it.
> 
> Simplify. People will love you for it. Do away with various account types
> with differing security models. Just do mail, and do it well.

I assure you that people won't love us for that. Our attempts to simplify parts of Thunderbird are frequently met with anger and frustration from users who were used to the old way. I'd be very hesitant to remove any significant feature from Thunderbird unless we were sure it helped the rest of the application. Removing RSS wouldn't help much, since it's fairly isolated from the rest of Thunderbird's code already.

> Could we just toss out the thinking of "folder types"? Let me create
> arbitrary folders and let me customize how I do things instead of trying to
> get me to do things the way some programmers think I want to do things.

We *could*, but that's the opposite of simplifying Thunderbird. That's adding more complex configuration for a feature that only a small number of power users will likely ever touch. Every time we add features like this, the maintenance burden for Thunderbird goes up, and it just gets harder and harder to write correct patches to enhance Thunderbird in the future.

However, there's already a bug filed on this particular issue: bug 577260.

> Computers are getting harder to use and have been getting harder for over 10
> years.

I disagree. Perhaps they've become less customizable (especially for mainstream OSes and window managers), but that's because writing good customization code is a lot of work that won't even be noticed by the vast majority of users.

> I should tell you that I don't speak from a position of ignorance. I'm a
> retired electronics engineer who worked designing hardware in Silicon Valley
> for 25 years (1977-2001) and can program productively in over 10 languages.

If that's the case, why not help out writing patches for Thunderbird? We're desperately in need of programmers with enough available time to help make a difference.

> > That's because the bug is resolved. Resolved bugs don't show up in that
> > query.
> 
> Well, that's just silly.

If you're not happy with the default query, make a new query for your bugs that includes resolved ones. Just do an "advanced search" for the criteria you want and then after getting the results, go to the bottom and click "Remember Search".

The default "My Bugs" query is designed to hide bugs that are no longer relevant; if it showed closed bugs, my list would be 474 bugs long (only 73 of those are open).
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: