950485 - Permissions whitelist does not work properly on wunderground.com

Status: UNCONFIRMED

(Core :: General, defect)

Description

[asmpgmr]

User Agent: Mozilla/5.0 (Windows NT 5.1; rv:20.0) Gecko/20100101 SeaMonkey/2.17.1 (Beta/Release)
Build ID: 20130410205058

Steps to reproduce:

In prefs.js:
permissions.default.script is set to 2 (block)

In permissions.sqlite:
wunderground.com script 1
wuxg.com script 1
google.com script 1
googleapis.com script 1
cloudflare.com script 1

Goto website www.wunderground.com


Actual results:

In Firefox 21 and above or SeaMonkey 2.18 and above the scripts on the page do not function so several aspects of the site do not work. For example Rapid Fire updates do not function, the line graphs next to Temperature, Dew Point, Pressure and Rainfall are not displayed and in the 10-Day Forecast section you cannot scroll to the next 5 days. 

Errors are displayed in the web console (error console on SeaMonkey):
ReferenceError: $ is not defined
ReferenceError: jQuery is not defined

jQuery libs are loaded from googleapis.com:
<script type="text/javascript" src="//ajax.googleapis.com/ajax/libs/jquery/1.7.2/jquery.min.js"></script>
<script src="//ajax.googleapis.com/ajax/libs/jqueryui/1.8.19/jquery-ui.min.js"></script>


Expected results:

Scripts should work since scripts are allowed (setting 1) for the appropriate sites in permissions.sqlite even though scripts are blocked in general via prefs.js (permissions.default.script = 2)

Note everything works OK up to Gecko 20 (Firefox 20 / SeaMonkey 2.17.2). All releases after that exhibit the issue and it still remains as of Gecko 26 (Firefox 26 / SeaMonkey 2.23)

Comment 1

[Boris Zbarsky [:bzbarsky]]

Would you be willing to use something like http://mozilla.github.io/mozregression/ to hunt down when the problem first appears for you?

Comment 2

[asmpgmr]

I stated in the actual results section that it first appeared in Firefox 21 / SeaMonkey 2.18 - this is why I opened bug against the 21 Branch even though the 26 Branch is current.

Comment 3

[Boris Zbarsky [:bzbarsky]]

Yes, I understand that.  mozregression will narrow down the problem to a single day's worth of changes to the code, typically.

Comment 4

[asmpgmr]

Well I normally use SeaMonkey. A quick check shows that SeaMonkey 2.18 beta 1 has the problem so it must have occurred early in the Gecko 21 development cycle.

SeaMonkey 2.18b1 info:
User agent: Mozilla/5.0 (Windows NT 5.1; rv:21.0) Gecko/20100101 SeaMonkey/2.18
Build identifier: 20130403022820

Comment 5

[asmpgmr]

Firefox 21 beta 1 has the problem as well. Firefox 20.0.1 / SeaMonkey 2.17.1 are the last released versions which work properly. The first two relevant messages in the error console are:

Error: ReferenceError: $ is not defined
Source File: http://www.wunderground.com/cgi-bin/findweather/getForecast?query=94618
Line: 46

Error: ReferenceError: jQuery is not defined
Source File: http://script.wxug.com/scripts/wui.global.min.js?v=1.3.6
Line: 1

Comment 6

[asmpgmr]

One other bit of info: if permissions.default.script is set to 1 (allow) then everything works as expected so the problem appears to be due to some change in permission handling between Gecko 20 and Gecko 21 beta 1.

Comment 7

[asmpgmr]

I tested various Nightly and Aurora builds from the Mozilla FTP site:

Nightly (Firefox 21.0a1)
PASS - Build identifier: Mozilla/5.0 (Windows NT 5.1; rv:21.0) Gecko/20130201 Firefox/21.0
PASS - Build identifier: Mozilla/5.0 (Windows NT 5.1; rv:21.0) Gecko/20130205 Firefox/21.0
FAIL - Build identifier: Mozilla/5.0 (Windows NT 5.1; rv:21.0) Gecko/20130206 Firefox/21.0
FAIL - Build identifier: Mozilla/5.0 (Windows NT 5.1; rv:21.0) Gecko/20130207 Firefox/21.0
FAIL - Build identifier: Mozilla/5.0 (Windows NT 5.1; rv:21.0) Gecko/20130208 Firefox/21.0
FAIL - Build identifier: Mozilla/5.0 (Windows NT 5.1; rv:21.0) Gecko/20130210 Firefox/21.0

Aurora (Firefox 21.0a2)
FAIL - Build identifier: Mozilla/5.0 (Windows NT 5.1; rv:21.0) Gecko/20130220 Firefox/21.0
FAIL - Build identifier: Mozilla/5.0 (Windows NT 5.1; rv:21.0) Gecko/20130222 Firefox/21.0

Last good build:
ftp://ftp.mozilla.org/pub/firefox/nightly/2013/02/2013-02-05-03-10-33-mozilla-central/

First bad build:
ftp://ftp.mozilla.org/pub/firefox/nightly/2013/02/2013-02-06-03-10-27-mozilla-central/

So a change between the Feb 5 2013 and Feb 6 2013 builds caused the issue.

Comment 8

[asmpgmr]

I discovered a workaround, in permissions.sqlite add:
ajax.googleapis.com script 1

This issue seems to be due to the "fix" introduced for bug 816956 - the whole premise of the bug seems wrongheaded. I would expect that a permission set for googleapis.com would apply to ajax.googleapis.com as well.

Comment 9

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

The problem is that for some reason googleapis.com is listed as an eTLD. I have no idea why that is.

The idea with bug 816956 was simply to forbid adding rules for "com" and "co.uk". No idea why "googleapis.com" is listed in the database as belonging to those.

Comment 10

[Boris Zbarsky [:bzbarsky]]

Gerv, do you know why that's an eTLD?

Comment 11

[asmpgmr]

Some of the eTLDs listed in the private domains section near the end of the file "effective_tld_names.dat" are unacceptable to exclude from permissions. It should be up to the user to decide whether they want to allow or block them.

For example under Google:
appspot.com
blogspot.com
codespot.com
googleapis.com
googlecode.com

This breaks permissions in an unexpected fashion.

Comment 12

[Gervase Markham [:gerv]]

It's an public suffix because Google requested that it be one.

It is unwise to create new uses for the Public Suffix List without being certain that it actually meets your requirements. The PSL is split into two categories - the ones defined by ICANN registries, and the more "private" ones like blogspot.com. It might be time for our PSL service to start to recognise the distinction, and for the smaller category (only the ICANN ones) to be used for this use.

CCing Mounir, who did the original patch.

Gerv

Comment 13

[Jonas Sicking (:sicking) No longer reading bugmail consistently]

I'd really like for our permission manager to work differently from how it works today. Right now it is the caller reading from the permission database that decides if a stored entry should apply to a domain or a domain+subdomains. This has the result that if we check for permission "x" in multiple places, a stored entry will sometimes apply to just a domain, and other times to all subdomains as well

The way it should IMO be is that the caller who *writes* to the database should decide if the written rule should apply to a domain, a domain+subdomains or an origin.

When reading we should simply ask "Someone with this principal is asking to do 'x', is that permitted per the current rules".

That way there's less of a risk of misinterpretations. Someone who footguns himself and gives additional permission to "*.com" will need to be explicit that that's what they're doing.

I'm ok with backing out bug 816956, but I'd prefer to instead create a sane permission manager API.

We're still not great at getting the Taipei team involved in core Gecko work. Perhaps we can find someone over there to help with this.

Comment 14

[asmpgmr]

I generally expect a permission to apply to domain+subdomains but adding a field to specify domain or domain+subdomains certainly makes sense.

One thing I would request is that permissions only be written by the user and not automatically added just by visiting sites. I noticed that with SeaMonkey 2.23 there is a flood of TLS permissions (falsestart-rc4 and falsestart-rsa) being set to allow in addition to the STS permissions that were already being added.

As for the original issue, now that I know what is happening I can work around it as per comment #8. Hopefully the private eTLD list isn't expanded causing more surprises.

Comment 15

[asmpgmr]

The private section of the eTLD list really needs to be ignored for permission checking. I just discovered some sites use scripts from maps.googleapis.com in addition to ajax.googleapis.com - it would be greatly preferred to be able to allow all googleapis.com sites and not have to do this one by one.

Comment 16

[Ioana (away)]

Keyword no longer needed since the request was answered in comment 7.