1518843 - Add messaging support for WebExtensions

Status: RESOLVED FIXED

(GeckoView :: Extensions, enhancement, P1)

Description

[[ex-Mozilla] Agi Sferro | :agi]

As part of the WebExtension support in Bug 1518841, we need a way to let embedders send and receive messages from content through WebExtensions.

Comment 2

[Ryan Kelly [:rfkelly]]

Hi :agi, I'm interested in following this work for the reasons in Bug 1500828. Are there existing API docs that could give me a bit of an overview of what the final experience will look like here?

+:stomlinson and :vladikoff as it looks like this will be the path we need for integrating with webchannel messages from FxA in Fenix.

Comment 3

[Shane Tomlinson [:stomlinson]]

Hi :cpeterson and :agi, is the plan to use the same WebChannel abstraction [1] that is used in desktop? I'm not sure what your requirements are, WebChannels are a way for trusted embedders to send & receive messages from the browser.

[1] - https://developer.mozilla.org/en-US/docs/Mozilla/JavaScript_code_modules/WebChannel.jsm

Comment 4

[Chris Peterson [:cpeterson]]

(In reply to Shane Tomlinson [:stomlinson] from comment #3)

Hi :cpeterson and :agi, is the plan to use the same WebChannel abstraction [1] that is used in desktop?

I don't know. That's a questions for Agi. :)

Comment 5

[Chris Peterson [:cpeterson]]

Adding [geckoview:fenix:m2] whiteboard tag because the Fenix team says they would like to start using WebExtensions for ad blocking and reader mode in their Fenix M2 or M3 milestones.

Comment 6

[[ex-Mozilla] Agi Sferro | :agi]

Sorry for the delay, I'm still trying to figure out what to do here.

I've looked around a little bit and :snorp idea of using a native channel in Bug 1500828 sounds good to me.

• We could expand Native Messaging on android letting apps define a set of native apps manifests in a folder inside the apk, e.g. in /assets/native_apps/.

  • In the future, if we want to allow users to install custom native apps we could do so allowing a /sdcard location for manifests too.

• For our use case, a geckoview app would define a "browser" native app which is used for internal web extensions to talk to geckoview and the app.

  • The path member of the native manifest would be unused for now.
  • All messages would go through the WebExtension object with a native messaging delegate, the app would be able to forward it to other apps if needed (how is TBD).

public class WebExtension {
    public static class NativeApp {
        String name;
        String description;
        // String path; unused for now
        List<String> allowedExtensions;
    }
    public static class NativeMessagingDelegate {
        void onNativeAppRegistered(NativeApp);
        // Redirects to other apps if needed
        GeckoResult<String> onMessage(String app, String json); // Maybe JSON object?
    }
    void setNativeMesssagingDelegate(NativeMessagingDelegate);
    GeckoResult<String> sendNativeMessage(String app, String json);
}

• On the extension side, the "browser" native app would look like an ordinary one:

var port = browser.runtime.connectNative("browser");

/*
Listen for messages from the app.
*/
port.onMessage.addListener((response) => {
  console.log("Received: " + response);
});

// Send message to GeckoView
port.postMessage("ping from web extension");

Comment 7

[James Willcox (:snorp) ([e-mail]) (he/him)]

This looks pretty close to what I was imagining. I would use JSONObject instead of String for values. I think for simplicity sake, though, maybe we shouldn't actually use the native messaging API and instead add some other way for extensions to get the Port. Maybe a special extension id, like browser.runtime.connect({ name: 'app' })? Kris, I'm sure you have an opinion here?

I like that onMessage and sendNativeMessage return GeckoResult, but AFAIK the extension messaging is one way, so these probably don't make sense. We could make sendNativeMessage return GeckoResult<Void> to indicate that the message was delivered, but we don't really do that anywhere else (yet).

Comment 8

[[ex-Mozilla] Agi Sferro | :agi]

(just repeating what I said on slack)

I have a mild preference for native messaging because the app is more like a native app than an extension. We could provide a hard-coded native app to avoid having people add a manifest.

I like that onMessage and sendNativeMessage return GeckoResult, but AFAIK the extension messaging is one way, so these probably don't make sense. We could make sendNativeMessage return GeckoResult<Void> to indicate that the message was delivered, but we don't really do that anywhere else (yet).

Ah you're right. I was fooled from runtime.sendMessage from which you can return a Promise as a response: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/runtime/onMessage#Sending_an_asynchronous_response_using_a_Promise

Comment 9

[Ryan Kelly [:rfkelly]]

I would use JSONObject instead of String for values

FWIW, with WebChannels for FxA we started out sending JSON objects from content to chrome code, but were advised to only send strings for security reasons: https://bugzilla.mozilla.org/show_bug.cgi?id=1238128

The story may be different here of course, just sharing for related historical context.

Comment 10

[[ex-Mozilla] Agi Sferro | :agi]

On a conversation with :snorp and :sebstian on Slack, it came up that we probably want messaging from within content scripts too directly, and the possibility to map the message back to a GeckoSession instance.

Using runtime.sendMessage with a custom extensionId="browser" or similar, would be pretty nice as:

• It's vailable from content scripts
• The meassage should be easily mapped back to a GeckoSession
• (minor but) It has a return value so it could embed a GeckoResult response (see comment #7 and comment #8).

The drawback is that we would need to have a "magic" value for extensionId.

I'm going to look at the code a little bit to see if this approach is feasible.

Comment 11

[[ex-Mozilla] Agi Sferro | :agi]

The approach in Comment #10 seems to work well. Looks like all we need is a MessageChannel that responds to browser for sending messages. I'm going to look next into receiving messages in web extensions and hopefully establishing channels too.

Comment 12

[Chris Peterson [:cpeterson]]

[geckoview:fenix:m2] not [geckoview:fenix:p1]

Comment 13

[[ex-Mozilla] Agi Sferro | :agi]

One thing I was thinking about, we probably want to add a new permission to web extensions to explicitly expose this feature.

Which makes me think we should not use sendMessage and just have a new API tied to the pref that uses it, we already pretty much don't do anything in sendMessage('browser' other than just redirecting the message to the java layer.

It would be something like runtime.sendBrowserMessage(...) tied to the geckoviewAddons permission.

This would work well for 3rd party extension too, because the GeckoView app would just check that external extensions don't have this permission in the manifest.

Comment 14

[[ex-Mozilla] Agi Sferro | :agi]

:kmag could you look at the above comment to see if it makes sense to you?

I'm proposing to add a new Extension API to runtime called sendBrowserMessage(...) similar to runtime.sendMessage that would be used by GeckoView apps to exchange message between the app layer and content scripts. This API would be behind a privileged pref geckoviewAddons.

We would need a pref because a GeckoView app might have both internal extensions that use browser messages and external extension which shouldn't have access to browser messages.

Comment 15

[Shane Tomlinson [:stomlinson]]

(In reply to Agi | :agi | ⏰ EST | he/him from comment #14)

We would need a pref because a GeckoView app might have both internal extensions that use browser messages and external extension which shouldn't have access to browser messages.

Would this API only be available to extensions? Would we be able to build a WebChannel abstraction [1] on top of this scheme? I ask because FxA needs a way to communicate with Fenix as part of the signin/signup for Sync process.

[1] - https://developer.mozilla.org/docs/Mozilla/JavaScript_code_modules/WebChannel.jsm

Comment 16

[[ex-Mozilla] Agi Sferro | :agi]

(In reply to Shane Tomlinson [:stomlinson] from comment #15)

(In reply to Agi | :agi | ⏰ EST | he/him from comment #14)

We would need a pref because a GeckoView app might have both internal extensions that use browser messages and external extension which shouldn't have access to browser messages.

Would this API only be available to extensions? Would we be able to build a WebChannel abstraction [1] on top of this scheme? I ask because FxA needs a way to communicate with Fenix as part of the signin/signup for Sync process.

[1] - https://developer.mozilla.org/docs/Mozilla/JavaScript_code_modules/WebChannel.jsm

WebExtensions is the way we envision GV apps like Fenix will be interacting with content, so yes it will be only available to Extensions, but we expect Fenix to implement something similar to WebChannel with a WebExtension.

From a cursory look it should be very straightforward to implement WebChannel as a WebExtension with the help of messaging that is implemented in this bug.

I'll make sure I have a prototype for WebChannel as a WebExtension before landing this.

Comment 17

[Chris Peterson [:cpeterson]]

A-C will use this extension messaging API to implement Reader View:

https://github.com/mozilla-mobile/android-components/issues/1346
https://github.com/mozilla-mobile/android-components/issues/1385

Comment 18

[[ex-Mozilla] Agi Sferro | :agi]

Little API update on this.

WebExtension API

There are two new runtime WebExtensions API being added as part of this, both of these APIs will be behind geckoviewAddons pref which is only available to builtIn WebExtensions (i.e. no third-party web extensions).

• runtime.sendBrowserMessage: similar to runtime.sendMessage, sends a message to the app and optionally gets a response Promise that is sent back to the WebExtension
• runtime.connectBrowser: similar to runtime.connect, will connect to the app and return a Port object that allows for bidirectional communication.

e.g. to send a simple message a web extension can do:

let response = browser.runtime.sendBrowserMessage('Ping from WebExtension');
response.then(message => {
    console.log(`Message from app: ${message}`);
});

to establish bidirectional communication:

let port = browser.runtime.connectBrowser();
port.onMessage.addListener(message => {
  console.log(`Message from port ${message}`);
});
port.postMessage("Ping from content");

GeckoView Java API

WebExtension has a new property called messageDelegate that allows GeckoView apps to respond to and send messages

class WebExtension {
    String location;
    String id;
    MessageDelegate messageDelegate; // new
}

interface MessageDelegate {
    GeckoResult<Object> onMessage(WebExtension source, Object message, GeckoSession session);
    void onConnect(WebExtension source, Port port, GeckoSession session);
}

the Object message can either be a primitive type, e.g. for sendBrowserMessage('test') the message would be a String, or a JSONObject if the message is a complex javascript object.

onConnect returns an instance of Port (described below) that can be used to send and receive messages from the WebExtension.

Port is the java equivalent of the WebExtension Port:

class Port {
    void postMessage(JSONObject message);
    void setDelegate(PortDelegate delegate);
}

interface PortDelegate {
    void onPortMessage(WebExtension source, Object message, GeckoSession session);
    void onDisconnect(WebExtension source, Port port);
}

onPortMessage is similar to onMessage except it cannot send responses.

E.g. for a simple single-tab app:

public class MyActivity extends AppCompatActivity implements WebExtension.MessageDelegate, WebExtension.PortDelegate {
    private WebExtension.Port mContentPort;

    @Override
    public void onConnect(@NonNull WebExtension source,
                          @NonNull WebExtension.Port port,
                          @Nullable GeckoSession session) {
        try {
            mContentPort = port;
            mContentPort.setDelegate(this);
            mContentPort.postMessage(new JSONObject("{\"message\": \"ping from java\"}"));
        } catch (JSONException ex) {
            throw new RuntimeException(ex);
        }
    }

    @Override
    public void onDisconnect(@NonNull WebExtension source, @NonNull WebExtension.Port port) {
        if (mContentPort == port) {
            mContentPort = null;
        }
    }

    @Override
    public void onPortMessage(@NonNull WebExtension source, @NonNull Object message,
                              @Nullable GeckoSession session) {
        Toast.makeText(getApplicationContext(), message.toString() + " " + session,
                Toast.LENGTH_SHORT).show();
    }
}

Where mContentPort can be used at any time to send messages to the WebExtension for a specific GeckoSession instance.

Comment 19

[Jovan Gerodetti]

I'm considering to build something with GeckoView and this API raises a question for me. From how I understand the current GeckoView API, The GeckoSession represents one currently loaded site in Gecko, like one Browser tab in Firefox Desktop. GeckoRuntime on the other hand represents the entire Gecko instance for my app. If this new WebExtensions API associates messages and message channels with one GeckoSession like the method signatures show, how would I communicate with my web extension background script / page? Would there be a new session just for the background script?

Comment 20

[[ex-Mozilla] Agi Sferro | :agi]

(In reply to Jovan Gerodetti from comment #19)

how would I communicate with my web extension background script / page? Would there be a new session just for the background script?

Messages from background scripts are handled the same way, simply the session parameter would be null in that case.

Comment 21

[[ex-Mozilla] Agi Sferro | :agi]

Try looks pretty green: https://treeherder.mozilla.org/#/jobs?repo=try&revision=f40c3e7bd3818d154f9e895db919c177ecfc00b9

Comment 22

[[ex-Mozilla] Agi Sferro | :agi]

Attachment: Bug 1518843 - Use isPrivileged instead of restrictScheme for addons permissions. r=kmag!

Comment 23

[[ex-Mozilla] Agi Sferro | :agi]

Attachment: Bug 1518843 - Add GeckoView API to unregister WebExtensions. r=snorp!

Depends on D22620

Comment 24

[[ex-Mozilla] Agi Sferro | :agi]

Attachment: Bug 1518843 - GeckoView WebExtension Messaging.

This change adds runtime.sendBrowserMessage and runtime.connectBrowser to
the WebExtension API.

This APIs are available behind the new privileged-only permission
geckoviewAddons and are used by GeckoView apps to communicate between content
and the app.

Depends on D22621

Comment 25

[[ex-Mozilla] Agi Sferro | :agi]

Attachment: Bug 1518843 - Move CallbackResult out of GeckoSession.

Depends on D22620

Comment 26

[Chris Peterson [:cpeterson]]

67=wontfix. Fenix MVP will use GeckoView 68, so we don't need to uplift this fix to 67 Beta.

Comment 27

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/48f975c56cc6
Use isPrivileged instead of restrictScheme for addons permissions. r=kmag
https://hg.mozilla.org/integration/autoland/rev/4a6e7c0051cb
Move CallbackResult out of GeckoSession. r=snorp

Comment 28

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/48f975c56cc6
https://hg.mozilla.org/mozilla-central/rev/4a6e7c0051cb

Comment 29

[[ex-Mozilla] Agi Sferro | :agi]

only half of the patches landed, sorry for the noise

Comment 30

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8fd075dbd205
Add GeckoView API to unregister WebExtensions. r=snorp,mbrubeck
https://hg.mozilla.org/integration/autoland/rev/5e0478607244
GeckoView WebExtension Messaging. r=snorp,esawin,robwu,kmag

Comment 31

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/8fd075dbd205
https://hg.mozilla.org/mozilla-central/rev/5e0478607244

Comment 32

[Gerco]

Is there already a good example of how we can use this? https://mozilla.github.io/geckoview/consumer/docs/web-extensions is out-dated in my opinion.

I tried to convert the above example to use with the new API, but I did not succeed.

I have to communicate from my own web application to the Android code and back.

What I already did is create a background.js script (where I created a port by calling browser.runtime.connectNative) and a content.js script (which can access the DOM of my application) which can receive messages by browser.runtime.onMessage.addListener.
But the part that is missing is how I can call/access the 'port.postMessage' from my content.js script (because the port is created in the background.js script).

Thanks in advance!

Comment 33

[Chris Peterson [:cpeterson]]

Moving some WebExtension bugs to the GeckoView::Extensions component.