1266356 - Allow Remote content 'Options' list incorrectly/excessively populated with "allow remote content from moz-nullprincipal:...."

Status: RESOLVED INVALID

(Thunderbird :: Security, defect)

Description

[Anje]

User Agent: Mozilla/5.0 (Windows NT 6.0; rv:45.0) Gecko/20100101 Firefox/45.0
Build ID: 20160407164938

Steps to reproduce:

Using Windows Vista and TB version 45
View an email in Message Pane.
It has remote content auto disabled.
There is the usual correct message and the 'Options' button.
Click on Options.
List is over populated with all the various remote possibilities in that email.
I noticed this occurring on my system and this morning someone else is also reporting the same issue.
See : https://support.mozilla.org/en-US/questions/1119265



Actual results:

Since updating to TB 45.0, when remote images are included within the body of the email, rather than requesting permission for a specific domain(s), each individual image is being treated as a unique 'source'. 
That is leading to rather long lists being displayed in the pop up which uses up a lot of screen real estate and/or means a long scroll to get to the "allow content from ********" at the bottom of the options. In the case of a large HTML email with multiple images, the list can be huge. 


Expected results:

None of the remote images should be individually listed.

Comment 1

[Anje]

In the release notes it states:
Remote content exceptions: Improved options to add exceptions.

This has created a massive list of numerical exceptions which is impossible to identify which is which and each one is specific to an unknown individual remote content. 

A typical bill from BT has 9 additional exceptions.
A typical bill from British Gas has 43 additional exceptions.

Knowing how many exceptions is really a very good improvement, but actually listing them is of no use as they cannot be identified and this leads to massive lists and scrolling to access the bit you need which is at the bottom.

Comment 2

[Jorg K (CEST = GMT+2)]

I usually get to see a few origins listed, maybe two to five, foe example:
images-eu.ssl-images-amazon.com, images-eu.amazon.com and www.amazon.de

The is something fishy going on with the e-mail in
https://support.mozilla.org/en-US/questions/1119265

I've never seen moz-nullprinciple ... 

Please attach the message here so we can investigate.

Comment 3

[rsx11m]

(In reply to Anje from comment #1)
> A typical bill from BT has 9 additional exceptions.
> A typical bill from British Gas has 43 additional exceptions.

Are those rather "normal" host names or those "moz-nullprincipal:" ones as well?
These may be two different issues.

http://mxr.mozilla.org/comm-central/search?string=moz-nullprincipal shows a bunch of hits for cookies, wherever they might be coming from within the context of (not loaded) remote content.

Comment 4

[Shiner]

Hi, It was my post on https://support.mozilla.org/en-US/questions/1119265 so just adding some notes to the bug report here.

User Agent 	Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.0
Application Build ID 	20160407161913

Comment 5

[Magnus Melin [:mkmelin]]

Could you attach the email to this bug? Save as .eml.

Comment 6

[Shiner]

Attachment: Email with 45 moz-nullprincipal: objects

This is an email that has '45' moz-nullprincipal: objects

Comment 7

[Shiner]

Attachment: Screenshot of partial pop-up panel.

This shows a part of the pop up panel for an email with just 45 'objects'.

Comment 8

[Jorg K (CEST = GMT+2)]

Attachment: This is the popup I see on TB 45 - all working

I see exactly two origins listed. I don't see any moz-principals.

Please try this in safe mode and/or on a new profile (start Thunderbird with -p).

Comment 9

[Jorg K (CEST = GMT+2)]

I think your permissions database got itself into a knot. You could remove/rename permissions.sqlite from the profile. That should fix things.

Comment 10

[Anje]

Every email that has remote content disabled is showing "moz-nullprincipal:" in the list. Identical to shiner's.

Comment 11

[Anje]

Checking in  Thunderbird Safe mode.
Now shows only three of what appear proper http links instead of the original 43 moz-nullprincipal.

Discovered that this moz-principals. is being caused by 'Conversations 2.11.1'

So. I can say that 'Conversations is the culprit.

However, this has brought abouit more worries.

Accidentally selected one of those http://img.trackerbox.com because I was attempting to copy paste the info and it disappeared from the list reducing to two http links.
So how do I undo that action ?
Edit > Undo does not undo the action.
It is not listed anywhere in exceptions.

It is far too easy to accidentally select those exceptions, I just clicked on Options again and managed to select another by accident as I presume the highlight was selected and I did not realise it. It is very difficult to see what is highlighted as it is so pale, it is virtually invisible and I've also discovered that if you use the arrow keys to up and down the list, there is one point second from bottom when it completely disappears as if there is a hidden item.
So now two items have been selected by accident and I need to unselect them or reverse the action.
I cannot see any means of reversing the action.

It still says remote content is blocked, but two tracker images http's have now disappeared from list, so are they enabled or not? Is remore content actually beiong blocked or not if items disappear from list?

Actually, having these items listed is of no use and you do not even know if you have selected anything unless you have previously  counted them. They disappear from list.
I mean who wants to manually enable each of these. You either want to see remote or you don't.

How do I get rid of this entirely and only display what was there in versions prior to 45?
It's very worrying when you do not know what if anything happens when you acciedntly select something.

So I located another unread bill - same origin and it only shows one http://img.trackerbox.com, when i know it had three.

This means that whatever I accidentally selected in a previous email, it is now being applied to all emails and I have no idea what it was, where it is, how to reverse it or why selecting one will make it disappear from the list in all other emails.

Any advise on how to locate and reverse whatever tracker thing was selected ? 
This is somewhat worrying.

Comment 12

[Jorg K (CEST = GMT+2)]

Many thanks Anje for tracking this down. This had us worried for a moment ;-(

To manage the remote exceptions go to Tools > Options, Privacy and click on the "Exceptions" button. There you can view, add and delete exceptions. That's been like this since TB 31.

Comment 13

[Jorg K (CEST = GMT+2)]

BTW, I could reproduce the problem after installing "Conversations". You need to switch to "classic view" to see it.

Comment 14

[Anje]

I had originally checked that location, maybe I had not refreshed it.
Upon rechecking, there was only the one exception:  http://link.trackerbox.com
There had been two in the list which were selected.
Removing the exception added one exception back into list not two.

Originally I had three exceptions listed.
http://img.trackerbox.com
http://link.trackerbox.com
http://link.trackerbox.com

I also located another exception which I could not recall ever selecting, so I removed it from the exception list and now it appears in the Options drop down list for British Gas, but it was not listed before.
Now I see:
http://img.trackerbox.com
http://link.trackerbox.com
https://portal.edm-comm.com

I checked the source code and it does have:
19 instances of img.trackerbox.com
18 instances of link.trackerbox.com
2 instances of portal.edm-comm.com (one in the headers)

So I'm certain all three variants should have been listed originally, but I know it only listed two with one duplicated.
However, it obviously added the selected exceptions correctly to the exceptions list in Options.


If I come across this odd behaviour again, I will remember to take a screenshot first to help prove the case.

Suggestion - maybe?
It would be more useful to show that exception as being selected in the drop down list instead of simply disappearing.

Comment 15

[Shiner]

(In reply to Anje from comment #11)

> > Discovered that this moz-principals. is being caused by 'Conversations
> 2.11.1'

Weird.  I had not seen this behaviour prior to TB45 but I did have the Conversations Addon installed and configured.  Sure enough, disabling the Addon got rid of the moz-principals.
Even though the Addon was enabled, it hadn't actually completed it's initial set-up and wasn't applying the views it should.  As soon as I disabled and re-enabled, it correctly configured my UI as expected.
That's probably why I didn't know it was running in the background as it hadn't actually changed my view to it's "conversation" view.  I wasn't using the features of that Addon so have disabled for now. 

Many thanks to those that took the time to have a look into this.  Much appreciated.

Comment 16

[Jonathan Protzenko [:protz]]

If someone could sum up exactly what my addon is doing wrong, and point me to the right fix, I'd be more than happy to fix it. The code is here https://github.com/protz/thunderbird-conversations/blob/master/modules/message.js#L907 for "always show remote content for this message", and the handler right after that is for "always show remote content from this sender".

Comment 17

[atttx123]

Attachment: img test.eml

this email work fine in both imap and exchange

Comment 18

[atttx123]

Attachment: img as attachment.eml

this email work fine in imap but crash when use ExQuilla (https://addons.mozilla.org/en-US/thunderbird/addon/exquilla-exchange-web-services/?src=search#id=244848)

Comment 19

[Kent James (:rkent)]

(In reply to atttx123 from comment #18)
> Created attachment 8746409 [details]
> img as attachment.eml
> 
> this email work fine in imap but crash when use ExQuilla
> (https://addons.mozilla.org/en-US/thunderbird/addon/exquilla-exchange-web-
> services/?src=search#id=244848)

I copied that email to an ExQuilla folder and had no problem with it. What do you do to make it crash?

Comment 20

[atttx123]

(In reply to Kent James (:rkent) from comment #19)
> (In reply to atttx123 from comment #18)
> > Created attachment 8746409 [details]
> > img as attachment.eml
> > 
> > this email work fine in imap but crash when use ExQuilla
> > (https://addons.mozilla.org/en-US/thunderbird/addon/exquilla-exchange-web-
> > services/?src=search#id=244848)
> 
> I copied that email to an ExQuilla folder and had no problem with it. What
> do you do to make it crash?

use TB 45 and conversations 2.11.1, open this mail and it will crash;
when i change to TB 38.1 and conversations 2.10.2 it work fine

Comment 21

[atttx123]

(In reply to Kent James (:rkent) from comment #19)
> (In reply to atttx123 from comment #18)
> > Created attachment 8746409 [details]
> > img as attachment.eml
> > 
> > this email work fine in imap but crash when use ExQuilla
> > (https://addons.mozilla.org/en-US/thunderbird/addon/exquilla-exchange-web-
> > services/?src=search#id=244848)
> 
> I copied that email to an ExQuilla folder and had no problem with it. What
> do you do to make it crash?

i can make it crash on TB 38.1 too:

1. i have tow inbox in TB, one imap other is exchange
2. open ExQuilla inbox and open this mail (img as attachment.eml)
3. change to another inbox
4. change back to ExQuilla inbox, TB will automotic show the mail you last open, it will crash.

Comment 22

[atttx123]

(In reply to Kent James (:rkent) from comment #19)
> (In reply to atttx123 from comment #18)
> > Created attachment 8746409 [details]
> > img as attachment.eml
> > 
> > this email work fine in imap but crash when use ExQuilla
> > (https://addons.mozilla.org/en-US/thunderbird/addon/exquilla-exchange-web-
> > services/?src=search#id=244848)
> 
> I copied that email to an ExQuilla folder and had no problem with it. What
> do you do to make it crash?

some other way to make TB(38.7.2) crash:

* use double click to open this mail in new tab, TB open a new tab and automatic focus on the new tab, it will crash

* use right click "Open Message in New Tab", switch to the new tab and switch back, then TB will crash

Comment 23

[Kent James (:rkent)]

[e-mail]: Are the crashes with ExQuilla occurring when Conversations is also installed? Does it crash without Conversations but just with ExQuilla?

Comment 24

[Jonathan Protzenko [:protz]]

Sounds like a different bug to me. File a new one and CC me?

Comment 25

[Jorg K (CEST = GMT+2)]

Why would be file bugs on add-ons here?

Comment 26

[Jonathan Protzenko [:protz]]

My assumption is that if Thunderbird crashes, it's a Thunderbird bug.

Comment 27

[Jorg K (CEST = GMT+2)]

No. You can do things in add-ons that trigger asserts, for example bug 1268474 comment #5. One would have to analyse the crash.

Comment 28

[Jonathan Protzenko [:protz]]

The point of an assert is to mean "this should never happen". If an 
addon can break the assert, then this shouldn't be an assert in the 
first place, but rather a check followed by a return code.

Comment 29

[Magnus Melin [:mkmelin]]

Agreed, that's misusing MOZ_ASSERT. JavaScript calling cpp should not be allowed to crash, only get an exception if it does something terribly wrong.

Comment 30

[Jonathan Protzenko [:protz]]

For future reference, the Thunderbird Conversations bug is at https://github.com/protz/thunderbird-conversations/issues/1090 but I haven't looked into what it is that needs to change on my side to avoid the unpleasant behavior.