The default bug view has changed. See this FAQ.

Add ability to implement programmable custom protocol handler

NEW
Assigned to

Status

()

Toolkit
WebExtensions: Request Handling
P2
normal
11 months ago
27 days ago

People

(Reporter: YUKI "Piro" Hiroshi, Assigned: mixedpuppy)

Tracking

(Blocks: 3 bugs)

Trunk
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(e10s+, firefox49 affected)

Details

(Whiteboard: [design-decision-approved]triaged)

Attachments

(2 attachments, 5 obsolete attachments)

(Reporter)

Description

11 months ago
My addon "URN Supprot" https://addons.mozilla.org/en-US/firefox/addon/urn-support/ provides ability to redirect some URN links to something related URLs. To migrate it from XUL-based to WebExtensions, I need something to alter the old method.

On Google Chrome, it seems available via `navigator.registerProtocolHandler()`.

https://developers.google.com/web/updates/2011/06/Registering-a-custom-protocol-handler

And it is already available on Firefox:

https://developer.mozilla.org/en-US/docs/Web/API/Navigator/registerProtocolHandler

However, the API seems to accept only static URL pattern as the handler. On the other hand, my "URN Support" addon generates a URL from a URN. For example, if an URN "urn:isbn:4-7980-1224-6" is given, it will be redirected to "http://www.amazon.co.jp/exec/obidos/ASIN/4798012246" or something. To convert the given URN to a URL of the product page, we need a programmable handler.

Thus, we need something new WebExtensions API to provide programmable protocol handler, or extend the `navigator.registerProtocolHandler()` to accept function or something as the handler.
(Reporter)

Updated

11 months ago
Blocks: 1215059

Updated

11 months ago
Whiteboard: [design-decision-needed]triaged

Comment 1

11 months ago
You could do this in Firefox if extensions were permitted to pass a `moz-extension:` URI to `navigator.registerProtocolHandler`. In your background script then you'd `registerProtocolHandler`, listen for the handler URI, then extract the "isbn:..." from the URI in the listener and cancel (or redirect) the request.  

navigator.registerProtocolHandler("urn", "/urn-handler?%s", "A urn: handler");
chrome.webRequest.onBeforeRequest(
  function (details) {
    // extract the isbn:... from details.url, then
    return {cancel: true};
  },
  {urls: [location.origin + "/urn-handler?*"]},
  ["blocking"]
);
(Reporter)

Comment 2

11 months ago
Thank you for an advice!

(In reply to Nancy Grossman from comment #1)
> You could do this in Firefox if extensions were permitted to pass a
> `moz-extension:` URI to `navigator.registerProtocolHandler`. In your
> background script then you'd `registerProtocolHandler`, listen for the
> handler URI, then extract the "isbn:..." from the URI in the listener and
> cancel (or redirect) the request.  

I've tried it but actually it didn't work from security errors on Nightly 49.0a1:

~~~
SecurityError: Permission denied to add http://moz-extension//1e79b00f-77ea-47c1-953a-c049021e84f1/urn-handler?%s as a content or protocol handler
getSecurityError()
WebContentConverter.js:212
checkAndGetURI()
WebContentConverter.js:164
registerProtocolHandler()
WebContentConverter.js:393
<anonymous>
redirector.js:421
~~~

Codes:

WebContentConverter.js:393:
~~~
  /**
   * See nsIWebContentHandlerRegistrar
   */
  registerProtocolHandler(aProtocol, aURIString, aTitle, aBrowserOrWindow) {
    LOG("registerProtocolHandler(" + aProtocol + "," + aURIString + "," + aTitle + ")");
    let haveWindow = (aBrowserOrWindow instanceof Ci.nsIDOMWindow);
    let uri;
    if (haveWindow) {
      uri = Utils.checkAndGetURI(aURIString, aBrowserOrWindow); // <= HERE!
    } else {
      // aURIString must not be a relative URI.
      uri = Utils.makeURI(aURIString, null);
    }
~~~

WebContentConverter.js:164:
~~~
const Utils = {
...
  checkAndGetURI(aURIString, aContentWindow) {
    let uri;
    try {
      let baseURI = aContentWindow.document.baseURIObject;
      uri = this.makeURI(aURIString, null, baseURI);
    } catch (ex) {
      throw NS_ERROR_DOM_SYNTAX_ERR;
    }

    // For security reasons we reject non-http(s) urls (see bug 354316),
    // we may need to revise this once we support more content types
    if (uri.scheme != "http" && uri.scheme != "https") {
      throw this.getSecurityError( // <= HERE!
        "Permission denied to add " + uri.spec + " as a content or protocol handler",
        aContentWindow);
    }
~~~

Any "moz-extensions:" URL seems not allowed as a valid protocol handler.

Moreover, even if I give a dummy URL like "http://moz-extensions/xxxx" as the protocol handler, it still raises an error: 'NS_ERROR_XPC_JAVASCRIPT_ERROR_WITH_DETAILS: [JavaScript Error: "aBrowserWindow.gBrowser is undefined" {file: "resource://app/components/WebContentConverter.js" line: 542}]'[JavaScript Error: "aBrowserWindow.gBrowser is undefined" {file: "resource://app/components/WebContentConverter.js" line: 542}]' when calling method: [nsIWebContentHandlerRegistrar::registerProtocolHandler]'

WebContentConverter.js:542:
~~~
  _getBrowserForContentWindow(aBrowserWindow, aContentWindow) {
    // This depends on pseudo APIs of browser.js and tabbrowser.xml
    aContentWindow = aContentWindow.top;
    return aBrowserWindow.gBrowser.browsers.find((browser) => // <= HERE!!
      browser.contentWindow == aContentWindow);
  },
~~~

I think that the "aBrowserWindow.gBrowser is undefined" error is from some codes not supported e10s.
(Reporter)

Comment 3

11 months ago
Created attachment 8751120 [details]
Experimental version XPI

Experimental build from https://github.com/piroor/urnsupport/tree/fb31a8400eb363e986bf8a989731d9b18d9c6052

When I load the package experimentally, it reports some security errors like above.
(Reporter)

Comment 4

11 months ago
(In reply to YUKI "Piro" Hiroshi from comment #2)
> SecurityError: Permission denied to add
> http://moz-extension//1e79b00f-77ea-47c1-953a-c049021e84f1/urn-handler?%s as
> a content or protocol handler
> getSecurityError()
> WebContentConverter.js:212
> checkAndGetURI()
> WebContentConverter.js:164
> registerProtocolHandler()
> WebContentConverter.js:393
> <anonymous>
> redirector.js:421

I commented wrong error information... Here is the correct error information.

~~~
SecurityError: Permission denied to add moz-extension://1e79b00f-77ea-47c1-953a-c049021e84f1/urn-handler?%s as a content or protocol handler
getSecurityError()
WebContentConverter.js:212
checkAndGetURI()
WebContentConverter.js:153
registerProtocolHandler()
WebContentConverter.js:393
<anonymous>
redirector.js:421
~~~
tracking-e10s: --- → ?

Comment 5

11 months ago
Apologies; I didn't explain myself clearly. Currently `registerProtocolHandler` only permits http: and https: handler URIs. Thus, the "correct error" in comment #4 is expected. So, my first thought was that `registerProtocolHandler` will also have to permit moz-extension: URIs when called from a WebExtension, because the handler URI must have the same origin as the document that registers the custom protocol.

However, according to MDN's documentation [1], the "same origin" requirement doesn't apply to extensions. I see an `onBeforeRequest` event for your "dummy URL" when I load it from the URL bar, so your "dummy URL" handler URI should have worked OK. So, the "wrong error" in comment #4 is unexpected.

Is there a way to test with e10s disabled?

Comment 6

11 months ago
I experimented a little in Firefox 46 (stable) with this statement:
window.navigator.registerProtocolHandler("urn", "http://example.com/urn?%s", "URN WebExtension");

(1) Adding a custom protocol from the background script always fails:

uncaught exception: Permission denied to add http://example.com/urn?%s as a content or protocol handler [background.js:43:1]
uncaught exception: Permission denied to add http://example.com/urn?%s as a content or protocol handler [<unknown>]

(2) Adding a custom protocol from a content script succeeds when the content script and the handler URI have the same origin. For example,

window.navigator.registerProtocolHandler("urn", "http://example.com/urn?%s", "URN WebExtension");

succeeds from a content script in "http://example.com/", but fails from the same content script in "https://userstyles.org/":

uncaught exception: Permission denied to add http://example.com/urn?%s as a content or protocol handler [content.js:2:1]
Permission denied to add http://example.com/urn?%s as a content or protocol handler [ExtensionUtils.jsm:30:0]

(3) Adding it succeeds from the page context (via Web Console) when the page and the handler URI have the same origin.

So, WebExtensions cannot register protocol handlers targeting other sites. The documentation [1] must refer to some other sort of add-on.

Electrolysis isn't enabled in my browser [2], so there may be other, e10s-related, issues. 

For now, you might hack around the problem by sending users to a known page when the add-on is installed, and register a protocol handler with that origin from a content script on that page. 

[1] https://developer.mozilla.org/en-US/docs/Web/API/Navigator/registerProtocolHandler (Apologies, I forgot to include this link earlier.)
[2] https://wiki.mozilla.org/Electrolysis#Enabling_and_Disabling_Electrolysis_on_Release

Comment 7

11 months ago
Created attachment 8751511 [details]
Demo WE: custom protocol test

The attached WebExtension's content script registers a custom "urn:" protocol when you browse to http://example.com/. In the background script, a webRequest listener reports "urn:*" URLs to the console. It almost works in Firefox 46, but there are issues.

(1) `onBeforeRequest` isn't called when using the custom protocol. The first event is `onBeforeSendHeaders`.

(2) The request can't be cancelled from `onBeforeSendHeaders`. Loading "urn:something-something-else" from the location bar throws an error:

NS_ERROR_XPC_NOT_ENOUGH_ARGS: Not enough arguments [nsIHttpChannel.cancel] WebRequest.jsm:366:0

Maybe (2) has already been fixed?
(Reporter)

Comment 8

11 months ago
Thanks a lot! Finally I successfully implemented URN handler addon based on current WebExtensions APIs. It worked with e10s, thus my previous comment about e10s compatibility was from misinterpretation. I'm very sorry.

However, I've realized that still there are some problems around user experience. I'll add comments later.
(Reporter)

Comment 9

11 months ago
Created attachment 8751599 [details]
Experimental version XPI based on currently available WE APIs

The updated experimental XPI. After installed, you need to visit the website https://piro.sakura.ne.jp/ to register its custom protocol handler. The step should be done via chrome.runtime.onInstalled (bug 1252871) but you have to do it manually for now. Then, you'll try actual URN links on some demo pages:

http://piro.sakura.ne.jp/xul/_urnsupport.html.en#what
https://jsfiddle.net/piro_or/z6m5zayn/3/

I think there are some user experience problems around the current experimental version:

1) Internal URI like "moz-extension://..." should be allowed as a protocol handler, and it should be registered from a background script. As pointed at the comment #5 and the comment #6, currently we have to do something hacky workaround with a URL of an actual webpage. This limitation kills ability to install protocol handling addon in offline.

(And I also know that such protocol handlers must be unregistered when the addon is disabled/uninstalled. It seems to be done via "chrome.runtime.onSuspend".)

2) Registering of a new protocol handler by an addon should be done without any manual operation. Currently each user has to click the "Add Application" button on the "Add URN Handler (www.example.com) as an application for urn links?" info bar manually, to complete installation steps of the addon's functionality. This means that the user can skip the required installation step unintentionally. Such a confirmation for special permissions should be done through the installation process by the addon manager itself, like apps installed from Google Play on Android.

3) New protocol handler added by an addon should become the default handler automatically, if there is no other existing handler. Currently you need to turn the "Remember my choice for urn links" checkbox on manually at the first time you clicked a URN link, otherwise the "Launch Application" dialog appears for every click on URN links.
Attachment #8751120 - Attachment is obsolete: true
(Reporter)

Comment 10

11 months ago
(In reply to YUKI "Piro" Hiroshi from comment #9)
> The updated experimental XPI. After installed, you need to visit the website
> https://piro.sakura.ne.jp/ to register its custom protocol handler.

Oops, you need to visit the page https://piro.sakura.ne.jp/xul/urnsupport/handler?urn=- instead of the top page.

Updated

11 months ago
Blocks: 905436
tracking-e10s: ? → +
Duplicate of this bug: 1296885
Note: there's a lot of relevant info for this bug in bug 1296885.
Component: WebExtensions: Untriaged → WebExtensions: Request Handling

Updated

5 months ago
Assignee: nobody → mixedpuppy
Whiteboard: [design-decision-needed]triaged → [design-decision-approved]triaged

Comment 13

5 months ago
These seems like a good thing to do, because the content script and other workarounds whilst they work seem pretty hacky and not a great experience. As Nancy and Piro point out, we've already got navigator.registerProtocolHandler(), so the idea of changing that to allow pointing moz-extension:// URL or something sounds pretty neat to me (there might be more nuance to that in the bug comments).

Updated

5 months ago
Priority: -- → P2
(Assignee)

Updated

5 months ago
Blocks: 1310316

Comment 14

5 months ago
They're more bugs than nuances. The basic approach is, (a) `navigator.registerProtocolHandler(scheme, document.origin + fake_pathname)` from the background page; (b) listen for the handler URL in a blocking `webRequest.onBeforeRequest` listener; (c) return a redirect URL. If the protocol is to be used from a content page, (d) mark the handler URL as a web-accessible resource in the manifest. 

Currently in Firefox you (1) can't register a protocol from the background page, and you (2) don't get `onBeforeRequest` events for URLs with custom schemes so you can't redirect them (comment 7). Deal breakers.

Also, the Firefox documentation says that extensions can register handlers for any site [1], but (3) content scripts can only register handlers for the same origin, and maybe would be true of background scripts as well. That's OK; in fact I'd prefer that a custom protocol stop working when the WE which added it is disabled or uninstalled.

(4) The scheme whitelist [2] isn't enforced in Firefox. That's convenient for e.g. 'Custom Buttons' (amo/addon/2707), which distributes code as `custombutton:` URLs. Also for users who'll grouse about typing an extra "web+" into the URL bar.

There are other complications if the handler generates the document instead of redirecting to an existing resource. (5) The custom URI will usually have to redirect to a `data:` URI. The alternative is a `blob:` URI, but (5a) these are same-origin, so background `blob:`s can't be linked from a content page and (5b) they don't expire, so if you pass them to a UI page you'll leak unless you explicitly revoke them (but when to do it?).

The worst is, (6) you can't use asynchronous APIs to build the response. You can use Web Storage but not IndexedDB, you can't use FileReader to make the `data:` URL, etc. That breaks e.g. 'Greasemonkey' (amo/addon/748). `runtime.onMessage` has a mechanism for responding asynchronously. Maybe `webRequest.onBeforeRequest` could too.

As an aside, (7) you can control whether other extensions or content pages can use the custom protocol. `blob:` URIs can only be used by your own UI pages, and `data:` URIs can only be used by content pages if they're marked 'web-accessible'.

There may be others. Maybe you could ask the developers of the add-ons in Honza Bambas' list [3].


Shane Caraveo, I don't find 'protocolhandler' in the 'Video Downloadhelper' xpi. How does this block bug 1310316?

[1] https://developer.mozilla.org/en-US/docs/Web/API/Navigator/registerProtocolHandler
[2] https://developer.mozilla.org/en-US/docs/Web/API/Navigator/registerProtocolHandler#Permitted_schemes
[3] https://bugzilla.mozilla.org/show_bug.cgi?id=1296885#c11
(Assignee)

Comment 15

5 months ago
> Shane Caraveo, I don't find 'protocolhandler' in the 'Video Downloadhelper'
> xpi. How does this block bug 1310316?

It's based on a conversation with the author about functionality that would be needed/or help to implement the functionality using the WebExtensions api.  But it may not be necessary, just tracking.
(Assignee)

Comment 16

5 months ago
I've been thinking about this quite a bit, and feel that we should not touch nav.registerProtocolHandler.  However, we can create a protocol api for webext that uses the same backend, resulting in the same functionality.  

1. nav.registerProtocolHandler is a web content api and I'd rather not touch anything that may have implications across the web.  

2. we may be able to design a better api than registerProtocolHandler that works better for WebExtensions.

3. but starting with a straight forward re-implementation gives us a starting point for hacking and discussion.

Thus today's scratch my itch hack is presented:

https://github.com/mixedpuppy/web-ext-rph

In dev edition, you can load about:debugging.  Then load the manifest in the top level, then load the manifest in the sample directory.  After that, open a new tab and type "web-ext:foobar" and see what happens.

Comment 17

5 months ago
Let me update comment #14: Bug 1254204 will allow the use of asynchronous APIs in `webRequest` handlers (6), and bug 1294996 will allow any origin to load `blob:` URIs created by the background script (5a).

(In reply to Shane Caraveo (:mixedpuppy) from comment #16)
> 1. nav.registerProtocolHandler is a web content api and I'd rather not touch [it]

I think white-listing `moz-extension:` while keeping the 'same-origin' registration policy could suffice. I don't like the idea of duplicating an existing Web API. The heart of a Chrome extension is an HTML document in a hidden window that behaves like every other. If a Web API doesn't work in the hidden window then it should be fixed, not replaced with a nearly identical extension API.

> 2. we may be able to design a better api

Most of the add-ons in Honza Bambas' list are using the protocol either to kick-start the add-on from the URL bar or a web page, or to use the add-on as a database. Allowing a function instead of a URL would simplify that, and the handler wouldn't need the `webRequest` permission.

> 3. but starting with a straight forward re-implementation gives us a starting point

I experimented with your demo. (Thank you.) Here's what happened: 
(1) I was able to create custom protocols that redirected to the extension's UI page (its stock behaviour), to a local filesystem `file:///`, and to a `blob:moz-extension;//" created in the background script, and could retrieve them all from a content page.

(2) Most of the `chrome.*` APIs were gone, including `chrome.webRequest`.

(3) The web protocol interface Firefox is awful. (3a) An origin can register 2+ handler URLs for one scheme. (3b) An origin can't remove its handlers. (3c) `http://example.com` can register a scheme for `https://example.com` and vice versa. (3d) A scheme persists in Preferences>Applications after its last handler is removed. (3e) `nsIWebHandlerApp` (but not `navigator.registerProtocolHandler`) permits static handler URLs (i.e., no '%s' in the template). (3e) The "Launch Application" dialog lists handlers by origin, but WebExtension origins ("moz-extension://da0c8efb-42ea-47f3-80e0-a0a416d8ded7") are opaque gibberish.
(Assignee)

Comment 18

5 months ago
(In reply to Nancy Grossman from comment #17)
> Let me update comment #14: Bug 1254204 will allow the use of asynchronous
> APIs in `webRequest` handlers (6), and bug 1294996 will allow any origin to
> load `blob:` URIs created by the background script (5a).
> 
> (In reply to Shane Caraveo (:mixedpuppy) from comment #16)
> > 1. nav.registerProtocolHandler is a web content api and I'd rather not touch [it]
> 
> I think white-listing `moz-extension:` while keeping the 'same-origin'
> registration policy could suffice. I don't like the idea of duplicating an
> existing Web API. 

If the goal were simply a 100% duplication I would lean towards agreeing.  I would rather examine what we actually need rather than falling back on an API that has relatively limited flexibility.  I think your response to the second item above is a perfect example of adding better functionality.

> > 3. but starting with a straight forward re-implementation gives us a starting point
> 
> I experimented with your demo. (Thank you.) Here's what happened: 

This feels like a bug hunt on demo code, if not I need to understand better.  Otherwise I feel like it's too early for that.  But one item did stick out.

> (2) Most of the `chrome.*` APIs were gone, including `chrome.webRequest`.

I would assume webextension APIs should be available and working if the page is a proper moz-extension origin.  If that's failing I'll want to understand why.  I'm not clear on how you see this breaking.

Moving forward:

In a way I feel like WebExtensions just needs a way to get a message for a protocol handler.

background.js:
chrome.protocol.addListener({ scheme: "myprotocol", name: "My Protocol Handler" }, details => {
  // details.url contains the full url clicked on.
  chrome.window.create({url: details.url}); // or whatever the addon needs to do
});

The downside to that may be that the addon does nothing user-visible which also wouldn't make sense, but maybe that is not a concern of the api.
(Assignee)

Comment 19

5 months ago
Further thinking:

handlerDetails = {
 scheme: the scheme to handle, required
 name: a name, required
 type: [url, page_action, browser_action], required
 url: url to open in tab, required if type url
}

- url specifies a url should be opened in a tab (url must be provided), must be in permissions or same-origin to extension
- page_action specifies to open the page_action defined in manifest
- browser_action specifies to open browser_action defined in manifest

chrome.protocol.register(handlerDetails);
chrome.protocol.onFired.addListener(details => {
  // details.url contains the full url clicked on
});

Comment 20

5 months ago
(In reply to Shane Caraveo (:mixedpuppy) from comment #16)
> 1. nav.registerProtocolHandler is a web content api and I'd rather not touch [it]

Only two small changes to `Utils.checkAndGetURI` [1] are needed, one to allow extension pages as handlers and one to allow extension pages to register handlers:

152    if (uri.scheme != "http" && uri.scheme != "https" && uri.scheme != "moz-extension") {
...
162        (!["http:", "https:", "moz-extension:"].includes(aContentWindow.location.protocol) ||

URLs with custom schemes are triggering `onBeforeRequest` events in Firefox 50, so with those two changes you could be done.

(In reply to Shane Caraveo (:mixedpuppy) from comment #18)
> I would rather examine what we actually need

17 of the top 29 add-ons actually use custom protocols (attachment 8783909 [details]), 22 different schemes in all. (I omit 'JavaScript Debugger' (216) which broke in FF33.) Some, such as 'x-lucidor` which reads from a zip archive, must wait on bug 1254204, but only `x-hds` ('Ant Video', 8174) can't be realized as a template URL and an `onBeforeRequest` listener. (Or maybe it can; it wasn't clear to me what `x-hds` was doing.) Seems flexible enough to me. Is there a use-case that isn't covered?

Then too there's Chrome parity to consider. Let's not write `if (Firefox) else` handlers for the next five years without a very good reason.

> second item above is a perfect example of adding better functionality.

Only until bug 1254204 lands. Then it's just sugar. (Mmm!) "Better functionality" would be, say, `unregisterProtocolHandler` or `isProtocolHandlerRegistered`. Anyway, `registerProtocolHandler` could be overloaded too.

> This feels like a bug hunt on demo code, if not I need to understand better.

It's not about your code or your feels. Firefox's custom protocol handling is deficient in six important ways. The deficiencies are in the production code you wrapped, not in your Experiments demo per se. Today it doesn't matter. Around 0% of Firefox users have ever seen that bit of UI. WebExtensions will raise that above 3%. You want to avoid touching `registerProtocolHandler`, but someone's gonna have to touch it.

> I would assume webextension APIs should be available and working...
> I'm not clear on how you see this breaking.

I added a `chrome.webRequest.onBeforeRequest` listener to background.js but forgot to add the permissions. Apologies; I'd forgotten how spare `chrome.*` is in a minimal extension.

[1] https://dxr.mozilla.org/mozilla-central/source/browser/components/feeds/WebContentConverter.js#141

Updated

4 months ago
Duplicate of this bug: 1310427
(Assignee)

Comment 22

4 months ago
Created attachment 8815869 [details] [diff] [review]
programatic protocol hanlders
Attachment #8815869 - Flags: feedback?(kmaglione+bmo)
(Assignee)

Comment 23

4 months ago
Created attachment 8815870 [details] [diff] [review]
nav.rph support
Attachment #8815870 - Flags: feedback?(kmaglione+bmo)
(Assignee)

Comment 24

4 months ago
The two patches add support for RPH through either an even in a background page or via the old nav.rph.  

- "handlers" may not be a great namespace
- "content_handler" permission was choosen to hopefully support content handlers, though I see they are limited to rss/atom/etc feeds
- nav.rph support relies on the "content_handler" permission to bypass the regular popup asking for permission.
(Assignee)

Comment 25

4 months ago
After looking into this further, I'm am inclined to only use the second patch and drop support for programmatic handling.  While the first patch works, it is not yet complete and there are simply more hurdles with existing platform assumptions that we'd have to jump over.  I'm not convinced there is a high benefit to programmatic handlers over a page load for *this* api.
Shane, can you clarify the difference?
(Assignee)

Comment 27

4 months ago
(In reply to Cameron Kaiser [:spectre] from comment #26)
> Shane, can you clarify the difference?

With the first patch you specify your protocol in manifest.json and use browser.handlers.onHandler.addListener to get a callback when the protocol is used.  The idea is that you would then open a tab, or do some preprocessing.  It also manages uninstalling the handler when the addon is uninstalled.

With the second patch you just use navigator.registerProtocolHandler, though the permission is still in the manifest, so the permission prompt would happen on install rather than when you make the call.

The problem with the first approach is that several assumptions are made in code (e.g. the tab is already opened before onHandler would be called).

A third option is to [sort of] blend the two.  Define the protocol in the manifest so that we manage uninstall, there would be no need to call registerProtocolHandler.  But there would be no onHandler call, we just open the tab/frame with the urlTemplate you provide.
(Assignee)

Comment 28

4 months ago
I'm inclined towards the 3rd option in comment 27 at this moment.  If we used nav.rph and relied on addons doing the right thing on shutdown or uninstall, a bug or forgetfulness could leave a non-working handler installed.

Updated

3 months ago
webextensions: --- → +
(Assignee)

Comment 29

3 months ago
Created attachment 8823830 [details] [diff] [review]
rph

This is the approach I've settled on.  Protocol handlers are defined in the manifest, and we manage adding/removing the handler.  This avoids the potential for an addon to leave an entry behind if it is uninstalled.  I think that is important since these will be addon urls rather than web urls.  There is no "handler" event fired, it works just as if you used nav.registerProtocolHandler, opening the url in a tab/frame, etc.
Attachment #8815869 - Attachment is obsolete: true
Attachment #8815870 - Attachment is obsolete: true
Attachment #8815869 - Flags: feedback?(kmaglione+bmo)
Attachment #8815870 - Flags: feedback?(kmaglione+bmo)
Attachment #8823830 - Flags: feedback?(kmaglione+bmo)
Comment hidden (mozreview-request)
(Assignee)

Comment 31

2 months ago
Comment on attachment 8823830 [details] [diff] [review]
rph

obsoleted by reviewboard
Attachment #8823830 - Attachment is obsolete: true
Attachment #8823830 - Flags: feedback?(kmaglione+bmo)
Will this prevent us from making nsIURI and URL parsing in general work off main thread?  This is a MAJOR perf and developer headache for us.  I thought we were finally going to get away from this requirement with web extensions.
Flags: needinfo?(mixedpuppy)
(Assignee)

Comment 33

2 months ago
(In reply to Ben Kelly [:bkelly] from comment #32)
> Will this prevent us from making nsIURI and URL parsing in general work off
> main thread?  This is a MAJOR perf and developer headache for us.  I thought
> we were finally going to get away from this requirement with web extensions.

All this (comment 30) does is add the config data for a protocol handler to nsIExternalProtocolService, the use of that handler happens elsewhere (see nsIExternalProtocolService/nsIWebHandlerApp).
Flags: needinfo?(mixedpuppy)
(Assignee)

Comment 34

2 months ago
(In reply to Ben Kelly [:bkelly] from comment #32)
> Will this prevent us from making nsIURI and URL parsing in general work off
> main thread?  This is a MAJOR perf and developer headache for us.  I thought
> we were finally going to get away from this requirement with web extensions.

That's also an odd and vague comment.  I don't understand what you mean by "get away from this requirement".  Probably not the right bug in which to have that discussion.
In the past addons could register URL parsing handlers to create custom protocols.  Like "foo://my+url+is+uses+pluses+for+separators".  This means that currently every time we parse a URL string we have to do it on the main thread because it might run js.  As you can imagine we parse URLs *a lot*.  Requiring every subsystem to touch the main thread to parse URLs is a major drag on browser perf.

If web extensions are not going to add custom URL parsing back, then I think my concerns are addressed.  Sorry if I misunderstood this bug.

Does that all sound correct to you?
Flags: needinfo?(mixedpuppy)
Yes, but the protocol handler needs to look at the URI to know where the resource being requested should be fetched from, right? How is that distinguished from the situation you mention?
(In reply to Cameron Kaiser [:spectre] from comment #36)
> Yes, but the protocol handler needs to look at the URI to know where the
> resource being requested should be fetched from, right? How is that
> distinguished from the situation you mention?

Looking at the URL is fine as long as it conforms to standard parsing rules.  For example, something like:

  scheme://username:password@host/some/path/?query=value#fragment

If you want web extensions to be able to define different parse schemes for urls, like:

  foo://host?path?separated?by?questions!?somethingelsenonstandard

Then we will be baking this long term perf penalty back into the browser.  This would be very disappointing to me since I thought one of the goals of web extensions was to avoid APIs that restricted our ability to improve the browser.
FWIW I wrote a separate bug for the off-main-thread URL parsing goal.  See bug 1332355.
See Also: → bug 1332355
(In reply to Shane Caraveo (:mixedpuppy) from comment #33)
> (In reply to Ben Kelly [:bkelly] from comment #32)
> > Will this prevent us from making nsIURI and URL parsing in general work off
> > main thread?  This is a MAJOR perf and developer headache for us.  I thought
> > we were finally going to get away from this requirement with web extensions.
> 
> All this (comment 30) does is add the config data for a protocol handler to
> nsIExternalProtocolService, the use of that handler happens elsewhere (see
> nsIExternalProtocolService/nsIWebHandlerApp).

I think the approach used here is the correct one.  This patch merely uses nsExternalProtocolHandler, which knows how to parse its URIs (it parses them as simple URIs).  This approach allows the extension to do custom parsing of the URI path to do what they need to without tying our hands in Gecko in terms of URI parsing.
Flags: needinfo?(mixedpuppy)

Comment 40

2 months ago
Hi, Greasemonkey author here.  I'm sorry I wasn't involved sooner, and there's a lot of comments to read (which I haven't 100%... but comment #14 sounds a lot like my concerns).  However, I'd like to explain our situation.


Greasemonkey is a user script manager, essentially we are a small extension ecosystem built into an extension.  One of the things we let scripts do is include named, resources of data, refer to them, and build URIs to serve them.  So if your script has a UI, you can include (potentially large) images, refer to them by URL, and have them work efficiently. (I.e. not by constructing giant data:base64 URLs.)

For example, in https://arantius.com/misc/greasemonkey/imgur-minimal-gallery.user.js I specify:

    // @resource    spinner img/spinner.gif

Then later in the code I do:

    document.body.innerHTML =
        "<img id='spinner'"
        +" style='position: fixed; top: 50%; left: 50%; margin-top: -15px; margin-left: -15px;'"
        +" src='"+GM_getResourceURL('spinner')+"'>";

Inserting that image into the page, by URL reference.  That URL is something like:

    greasemonkey-script:6b21c9e9-0500-4f34-8c4a-fb99202cb663/spinner

Or, functionally identical to the moz-extension: scheme, an opaque prefix identifying the script, and the name of the resource.  Internally we register an nsIProtocolHandler handler, which (conditionally) just returns a channel to the equivalent file:/// where we have previously stored those contents ( https://github.com/greasemonkey/greasemonkey/blob/master/modules/scriptProtocol.js#L128 ).

Note the use case there, in that it's serving an image referred to by the DOM of a content page, or similarly for CSS.  I don't completely understand all the above, but constructing an HTML document or a redirect to some public internet URL is not what we do with this.  Perhaps in theory we could construct a data: URL for the binary content of the image and redirect to that, but ick.
So I just wanted to register the comment I made on Myk's github repo here.

https://github.com/mozilla/moz-handler/issues/1#issuecomment-274633424

Synopsis: this API is just a partial solution to the general problem, and one that has privacy shortcomings in the general case.

Because this leads to a URI being sent to the same server in every case, that server gains the ability to track the use of any resource that uses the scheme.  In the case where the target of a custom scheme handler is the site that is controlled by the addon author, this is not a problem.  Anywhere else and you are stuck.

I think that the more general solution here would be to permit a web extension to install a service worker.  That service worker would be exempt from the usual same-origin restrictions and could be installed for any protocol.  I realize that's probably a considerable amount of work to achieve, but it would neatly avoid a bunch of the concerns raised in this thread.  I think that it covers Ben's threading concerns and Anthony's resource aliasing concern.

Whether we ALSO want a simpler API is not a question that I have much of an opinion on.  If there's a general view that a simple API is still valuable then maybe having both isn't disastrous.  I think that it probably isn't that useful if a SW can be implemented in a reasonable time frame.
Why necessarily would a WebExtension need a host to have the URI string sent to? Wouldn't it do that processing locally?
No, you are right, this could be handled by a chrome:// resource.
(In reply to Martin Thomson [:mt:] from comment #41)
> I think that the more general solution here would be to permit a web
> extension to install a service worker.

Your comment 43 may moot your point about this, but I think it's worth addressing because it's a tempting notion that I expect to come up again in the future, so speaking as a Gecko ServiceWorker dev:

The abstract concept makes sense, but this would need to be be its own distinct idea and implementation.  Something like a "WebExtVirtualServer" API that dispatches "webext-fetch" events at the extension's background page.

The ServiceWorker implementation and related logic (necko http channels, fetch, etc.) would be massively complicated by attempting to also address web-extensions use cases.  Of particular concern is that ServiceWorkers and their interaction with fetch are well-specified but also tremendously complex (and currently are moving targets).  Unless there's cross-browser interest to augment the specs to explicitly provide webext support for synthetic origins/protocols, Gecko would effectively end up with a massive complicated fork of the specs, forever.

Comment 45

2 months ago
ah nice that this is in prograss, this functionality is required to get my man:// protocol working :D
(Assignee)

Updated

2 months ago
Depends on: 1310427
(Assignee)

Updated

2 months ago
Attachment #8825218 - Flags: review?(kmaglione+bmo)
(Assignee)

Updated

2 months ago
Attachment #8825218 - Attachment is obsolete: true
(Assignee)

Updated

a month ago
webextensions: + → ---

Comment 46

28 days ago
FYI: I'm continuing to research how/whether Greasemonkey can be re-written as a WebExtension.

What we'd really want, as simple as I can write it without skipping key detail:

A) The ability to download arbitrary content at run time (user scripts and their resorces),
B) Stored into some place with a simple key,
C) A custom-protocol-like handler we can register, so that
D) Whenever _any part_ of the browser accesses a URL in that protocol,
E) We can quickly and efficiently return the key from step B, to serve the previously stored content.  (This means, e.g., no giant data: URIs encoding large piles of data, and human-readable URLs.)

This came up again today as I started working on actually evaluating user scripts.  If I don't do things carefully, I get invalid URLs (in the console, in case of log/error) like "mycontentscript.js%20line%203%20%3E%20eval:2:7" or "ExtensionContent.jsm:283:22" (which happens to be the `Cu.evalInSandbox()` line, for tabs.executeScript()).  They're both clickable, but the target of the click is unhelpful.

I can use `//# sourceURL=...` to help make these readable, but I can't point it at something that makes clicking it actually show which line was the problem.  But if I had a custom protocol to serve, then I _could_.

And as mentioned before, I'd like to also be able to serve (content accessible) things like CSS and images from this handler as well.

Comment 47

27 days ago
The IPFS Add-On [1] I created has a bit different use for custom protocol handlers than examples mentioned so far, but I feel the use case is generic enough to be described here.  
                                                                                                                                                                                                                      
Distilled needs are:                                                                                                                                                                               
                                                                                                                                                                                                                      
A) Ability to register custom protocol handler (to make this example generic let's call it "foo://")

B) Opening "foo://origin/path/to/resource" loads HTTP(S) resource
   from "https://httpgateway/foo/origin/path/to/resource" but with two caveats:

    B.1) "origin" as the Origin (this is crucial, so that cookies, local storage
         etc is the same no matter which "httpgateway" is used).

         Ideally, it would be great if we could specify which URL segments
         are used for Origin computation, for example:

         Opening "bar://originprefix/origin/path/to/resource" loads HTTP(S) resource
         from "https://httpgateway/bar/originprefix/origin/path/to/resource"
         with "originprefix/origin" as Origin.

    B.2) Keeping canonical "foo://origin/path/to/resource" in Location Bar
         (hiding underlying HTTP transport for improved user experience)


In short, keeping canonical address in Location Bar together with programmable Origin calculation would enable us to extend Firefox with new protocols in a way that provides great UX and is compatible with Origin-based security model.

I hope this is an useful data point for this discussion.
                                                                                                                                                                                                                      
[1] https://addons.mozilla.org/en-US/firefox/addon/ipfs-gateway-redirect/
You need to log in before you can comment on or make changes to this bug.