1362849 - TLD in Match patterns in Permissions

Status: RESOLVED WONTFIX

(WebExtensions :: General, enhancement, P5)

Description

[erosman]

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:54.0) Gecko/20100101 Firefox/54.0
Build ID: 20170505181442

Steps to reproduce:

Nowadays,there are numerous sites that redirect to localised TLD. Google is a prime example.

While "content_scripts" provide globs in addition to "match patterns", the "permissions" do not.

I had a single permission set as "*://www.google.com/" which failed due to redirects to for example ""www.google.co.jp"

The options at the moment are:

- Listing all 100s of domains, which is impractical and error-prone
- Use a catch-all permission such as "<all_urls>" or "http://*/*", "https://*/*", which is not the best practice AFA security is concerned

A mechanism, to allow wildcard for TLD would greatly improve the matching system.

Comment 2

[erosman]

PS. I can help with writing the code needed for "MatchPattern.jsm" and pass it on to be tested/implemented

Comment 3

[Shane Caraveo (:mixedpuppy)]

(In reply to erosman from comment #2)
> PS. I can help with writing the code needed for "MatchPattern.jsm" and pass
> it on to be tested/implemented

That file is going away.  see Bug 1368102

Comment 4

[Tomislav Jovanovic :zombie]

Is the use case here basically just "Google"?  I can't even find another company that *actively* uses more than a few country-specific domains.

(Also, not all countries follow the same domain structure, for example "google.uk" is not a valid domain)

Comment 5

[Kris Maglione [:kmag]]

Yeah, I'm generally not a fan of this idea. In practice it seems to mostly be used for google.*, but most of the TLDs that it winds up matching are not valid ("google.yoga", "google.blogspot.com", "google.yandex"). Matching them anyway is probably not super harmful, but I still don't think it's a particularly good idea (and I wouldn't be surprised if it's exploitable in some circumstances).

Comment 6

[erosman]

> Is the use case here basically just "Google"?
Amazon is another example:
amazon.cn amazon.in amazon.co.jp amazon.fr amazon.de amazon.it amazon.nl amazon.es amazon.co.uk amazon.ca amazon.com.mx amazon.com amazon.com.au amazon.com.br

Comment 7

[Caitlin Neiman [:caitmuenster]]

Hi erosman, this has been added to the agenda for the January 9, 2018 WebExtensions APIs triage meeting. Would you be able to join us? 

Here’s a quick overview of what to expect at the triage: 

* We normally spend 5 minutes per bug
* The more information in the bug, the better
* The goal of the triage is to give a general thumbs up or thumbs down on a proposal; we won't be going deep into implementation details

Relevant Links: 

* Wiki for the meeting: https://wiki.mozilla.org/WebExtensions/Triage#Next_Meeting
* Meeting agenda: https://docs.google.com/document/d/15JYw3L1490dKbr6yTLz1uirH8rfhcSiLJCubSWDlnfs/edit#
* Vision doc for WebExtensions: https://wiki.mozilla.org/WebExtensions/Vision

Comment 8

[kernp25]

Many add-ons needs this!

The following example should work:
webRequest.onBeforeRequest.addListener(
  processRequest,
  {urls: ["*://www.google.*/url?*&url=*"], types: ["main_frame", "sub_frame"]},
  ["blocking"]
);

This matches:
https://www.google.com/url?...&url=...
https://www.google.de/url?...&url=...

Comment 9

[erosman]

As an afterthought .....

While it is not an important issue and only an enhancement, following consideration is noteworthy:

> A mechanism, to allow wildcard for TLD would greatly improve the matching system.

For example using a magic keyword *://*.google.TLD/*

> but most of the TLDs that it winds up matching are not valid ("google.yoga", "google.blogspot.com", "google.yandex").

Using nsIEffectiveTLDService should prevent non-TLD matches. Therefore "google.blogspot.com" should never match since "blogspot.com" is not a TLD, nor is "yandex".


> Matching them anyway is probably not super harmful, but I still don't think it's a particularly good idea (and I wouldn't be surprised if it's exploitable in some circumstances).

Currently, developers often use "<all_urls>" or "http://*/*", "https://*/*" and using "<all_urls>" or "http://*/*", "https://*/*" creates exponentially more "exploitable situations" comparing to *://*.google.TLD/*

Comment 10

[kernp25]

You said in the meeting:
Big concern is how this would be shown to users and how they would agree to the permission

I have installed this add-on (later removed it):
https://addons.mozilla.org/firefox/addon/searchpreview/

This is the add-on manifest:
{
"applications": {
  "gecko": {
    "id": "{EF522540-89F5-46b9-B6FE-1829E2B572C6}",
    "strict_min_version": "55.0"
  }
},
   "manifest_version": 2,
   "background": {"scripts": ["background.js"]},
   "content_scripts": [ {
      "js": [ "sp-content.js" ],
      "matches": [
"*://*.bing.com/*",
"*://*.duckduckgo.com/*",
"*://*.search.yahoo.com/*",
"*://encrypted.google.com/*",
"*://www.google.ad/*",
"*://www.google.ae/*",
"*://www.google.al/*",
"*://www.google.am/*",
"*://www.google.as/*",
"*://www.google.at/*",
"*://www.google.az/*",
"*://www.google.ba/*",
"*://www.google.be/*",
"*://www.google.bf/*",
"*://www.google.bg/*",
"*://www.google.bi/*",
"*://www.google.bj/*",
"*://www.google.bs/*",
"*://www.google.bt/*",
"*://www.google.by/*",
"*://www.google.ca/*",
"*://www.google.cat/*",
"*://www.google.cd/*",
"*://www.google.cf/*",
"*://www.google.cg/*",
"*://www.google.ch/*",
"*://www.google.ci/*",
"*://www.google.cl/*",
"*://www.google.cm/*",
"*://www.google.co.ao/*",
"*://www.google.co.bw/*",
"*://www.google.co.ck/*",
"*://www.google.co.cr/*",
"*://www.google.co.id/*",
"*://www.google.co.il/*",
"*://www.google.co.im/*",
"*://www.google.co.in/*",
"*://www.google.co.je/*",
"*://www.google.co.jp/*",
"*://www.google.co.ke/*",
"*://www.google.co.kr/*",
"*://www.google.co.ls/*",
"*://www.google.co.ma/*",
"*://www.google.co.mw/*",
"*://www.google.co.mz/*",
"*://www.google.co.nz/*",
"*://www.google.co.pn/*",
"*://www.google.co.th/*",
"*://www.google.co.tt/*",
"*://www.google.co.tz/*",
"*://www.google.co.ug/*",
"*://www.google.co.uk/*",
"*://www.google.co.uz/*",
"*://www.google.co.ve/*",
"*://www.google.co.vi/*",
"*://www.google.co.za/*",
"*://www.google.co.zm/*",
"*://www.google.co.zw/*",
"*://www.google.com/*",
"*://www.google.com.af/*",
"*://www.google.com.ag/*",
"*://www.google.com.ai/*",
"*://www.google.com.ar/*",
"*://www.google.com.au/*",
"*://www.google.com.bd/*",
"*://www.google.com.bh/*",
"*://www.google.com.bn/*",
"*://www.google.com.bo/*",
"*://www.google.com.br/*",
"*://www.google.com.bz/*",
"*://www.google.com.co/*",
"*://www.google.com.cu/*",
"*://www.google.com.cy/*",
"*://www.google.com.do/*",
"*://www.google.com.ec/*",
"*://www.google.com.eg/*",
"*://www.google.com.et/*",
"*://www.google.com.fj/*",
"*://www.google.com.gh/*",
"*://www.google.com.gi/*",
"*://www.google.com.gr/*",
"*://www.google.com.gt/*",
"*://www.google.com.hk/*",
"*://www.google.com.jm/*",
"*://www.google.com.kh/*",
"*://www.google.com.kw/*",
"*://www.google.com.kz/*",
"*://www.google.com.lb/*",
"*://www.google.com.lv/*",
"*://www.google.com.ly/*",
"*://www.google.com.mm/*",
"*://www.google.com.mt/*",
"*://www.google.com.mw/*",
"*://www.google.com.mx/*",
"*://www.google.com.my/*",
"*://www.google.com.na/*",
"*://www.google.com.nf/*",
"*://www.google.com.ng/*",
"*://www.google.com.ni/*",
"*://www.google.com.np/*",
"*://www.google.com.om/*",
"*://www.google.com.pa/*",
"*://www.google.com.pe/*",
"*://www.google.com.pg/*",
"*://www.google.com.ph/*",
"*://www.google.com.pk/*",
"*://www.google.com.pl/*",
"*://www.google.com.pr/*",
"*://www.google.com.py/*",
"*://www.google.com.qa/*",
"*://www.google.com.ru/*",
"*://www.google.com.sa/*",
"*://www.google.com.sb/*",
"*://www.google.com.sg/*",
"*://www.google.com.sl/*",
"*://www.google.com.sv/*",
"*://www.google.com.tj/*",
"*://www.google.com.tn/*",
"*://www.google.com.tr/*",
"*://www.google.com.tt/*",
"*://www.google.com.tw/*",
"*://www.google.com.ua/*",
"*://www.google.com.uy/*",
"*://www.google.com.vc/*",
"*://www.google.com.ve/*",
"*://www.google.com.vn/*",
"*://www.google.cv/*",
"*://www.google.cz/*",
"*://www.google.de/*",
"*://www.google.dj/*",
"*://www.google.dk/*",
"*://www.google.dm/*",
"*://www.google.dz/*",
"*://www.google.ee/*",
"*://www.google.es/*",
"*://www.google.fi/*",
"*://www.google.fm/*",
"*://www.google.fr/*",
"*://www.google.ga/*",
"*://www.google.ge/*",
"*://www.google.gg/*",
"*://www.google.gl/*",
"*://www.google.gm/*",
"*://www.google.gp/*",
"*://www.google.gr/*",
"*://www.google.gy/*",
"*://www.google.hn/*",
"*://www.google.hr/*",
"*://www.google.ht/*",
"*://www.google.hu/*",
"*://www.google.ie/*",
"*://www.google.iq/*",
"*://www.google.is/*",
"*://www.google.it/*",
"*://www.google.je/*",
"*://www.google.jo/*",
"*://www.google.jp/*",
"*://www.google.kg/*",
"*://www.google.ki/*",
"*://www.google.kz/*",
"*://www.google.la/*",
"*://www.google.li/*",
"*://www.google.lk/*",
"*://www.google.lt/*",
"*://www.google.lu/*",
"*://www.google.lv/*",
"*://www.google.md/*",
"*://www.google.me/*",
"*://www.google.mg/*",
"*://www.google.mk/*",
"*://www.google.ml/*",
"*://www.google.mn/*",
"*://www.google.ms/*",
"*://www.google.mu/*",
"*://www.google.mv/*",
"*://www.google.mw/*",
"*://www.google.ne/*",
"*://www.google.ne.jp/*",
"*://www.google.nl/*",
"*://www.google.no/*",
"*://www.google.nr/*",
"*://www.google.nu/*",
"*://www.google.off.ai/*",
"*://www.google.ph/*",
"*://www.google.pl/*",
"*://www.google.pn/*",
"*://www.google.ps/*",
"*://www.google.pt/*",
"*://www.google.ro/*",
"*://www.google.rs/*",
"*://www.google.ru/*",
"*://www.google.rw/*",
"*://www.google.sc/*",
"*://www.google.se/*",
"*://www.google.sg/*",
"*://www.google.sh/*",
"*://www.google.si/*",
"*://www.google.sk/*",
"*://www.google.sm/*",
"*://www.google.sn/*",
"*://www.google.so/*",
"*://www.google.st/*",
"*://www.google.td/*",
"*://www.google.tg/*",
"*://www.google.tk/*",
"*://www.google.tl/*",
"*://www.google.tl/*",
"*://www.google.tm/*",
"*://www.google.tn/*",
"*://www.google.to/*",
"*://www.google.tt/*",
"*://www.google.us/*",
"*://www.google.vg/*",
"*://www.google.vn/*",
"*://www.google.vu/*",
"*://www.google.ws/*"      
      ],
      "run_at": "document_start"
   } ],
   "default_locale": "en",
   "description": "__MSG_ex_description__",
   "icons": {
      "128": "sp_128.png",
      "32": "sp_32.png",
      "48": "sp_48.png"
   },
   "name": "__MSG_ex_name__",
   "permissions": [ "contextMenus", "storage", "https://*.searchpreview.de/", "https://widgets.alexa.com/" ],
   "options_ui": {
		"page": "options.html"
	},
   "version": "10.1"
}

Please look at the picture how the permission dialog will look like.

Comment 11

[kernp25]

Attachment: preview.PNG

Comment 12

[kernp25]

The permission dialog lists also 210 sites that i cannot see (check).

Comment 13

[kernp25]

I think the current situation is also really bad.

Comment 14

[kernp25]

Maybe you can improve the permission dialog, so i can view all urls?
Maybe add an extra button to view all urls?

Comment 15

[kernp25]

Or make the permission dialog scroll able?
Add an scrollbar?

Comment 16

[Andy McKay]

The meeting voted against this approach because of the lack of guarantee about what a TLD is as per comment 5.

Kernp25, your comments are a duplicate of bug 1427622.

Comment 17

[erosman]

> The meeting voted against this approach because of the lack of guarantee about what a TLD is as per comment 5.

I understand. I was wondering ...... Doesn't 'nsIEffectiveTLDService' define it correctly?

Comment 18

[erosman]

Another use case:

https://discourse.mozilla.org/t/local-ip-wildcard-for-webextensions-host-permission/26360
> I need to create an extension that communicate with local webserver (i.e. qbittorrent web ui). I need to define host permission like "http://192.168.1.*", so that anyone else can use it as long as their webserver is local.