1268594 - Support TypedArray in native messaging

Status: RESOLVED INACTIVE

(WebExtensions :: General, defect, P5)

Description

[Andrew Swan [:aswan]]

This would be an incompatibility with Chrome, but extensions that want to send binary data to/from native messaging hosts currently have to base64 encode the data, yuck.  We can probably make this better for those cases.

Comment 1

[Kris Maglione [:kmag]]

This seems like a good idea, but we'd need a way to specify the message type without breaking compatibility with JSON.

I guess we could treat the first byte of the message as a message type ID, if it's not a valid character in JSON, and then wind up with something like:

0x01: JSON
0x02: ArrayBuffer
...

Comment 2

[Andrew Swan [:aswan]]

(In reply to Kris Maglione [:kmag] from comment #1)
> This seems like a good idea, but we'd need a way to specify the message type
> without breaking compatibility with JSON.

Yeah, I made the same comment in the original email, but neglected to include it here.
The more bulletproof approach would be to just define a new incompatible protocol and use a new value for type in the host manifest to select the new protocol.

Comment 3

[Richard Bateman]

Just in the spirit of brainstorming, because I'm not actually sure this would be a good idea, you could do something with the message size bytes; whenever you send a message it starts with a 32 bit int indicating the message size.  You could use the sign bit as an indicator.

Yeah, probably not a good idea, but maybe provokes a better one?

I think I probably like the idea of a mode selectable in the manifest; it would need to somehow indicate to the host what mode it was loading it in, though. perhaps this could be done with argc/argv? More brainstorming, you could possibly use bson as an alternative communication method; it's an open format AFAIK and might not be too hard to incorporate, and even if you only supported a subset of the special types it would provide a "standard" way to communicate which was still similar in principle to the original messages.

Just throwing out some ideas; I haven't played with the c++ bson libraries, so I have no idea how easy they'd be to use or if there would be licensing issues.  I believe monogdb uses them with spidermonkey, though, so it might not be hard to pull in.

Comment 4

[Andrew Swan [:aswan]]

(In reply to Richard Bateman from comment #3)
> I think I probably like the idea of a mode selectable in the manifest; it
> would need to somehow indicate to the host what mode it was loading it in,
> though. perhaps this could be done with argc/argv?

That seems like a reasonable way to handle it.  In practice, I think it would be reasonable for now for native executables to just assume that format and they would only break if an extension managed to get a message sent that serializes to a string longer than 2GB.

For those that want to be extremeley careful, if we mimic the Chrome behavior of passing the path to the manifest file on the command line, then the native executable has enough information to determine the mode (obviously this is only relevant at some point in the future when multiple modes exist).

> More brainstorming, you
> could possibly use bson as an alternative communication method; it's an open
> format AFAIK and might not be too hard to incorporate, and even if you only
> supported a subset of the special types it would provide a "standard" way to
> communicate which was still similar in principle to the original messages.

If we support TypedArray then you could use a js implementation of bson and directly send the encoded representation.

Comment 5

[Kris Maglione [:kmag]]

If we do go that route, we could just make the size a signed integer, rather than unsigned, and use negative numbers to indicate the size of a binary blob rather than a JSON string.

That would limit us to a 2GB message size, though.

(In reply to Andrew Swan [:aswan] from comment #4)
> If we support TypedArray then you could use a js implementation of bson and
> directly send the encoded representation.

That sounds like a good idea. I suspect it would be nearly as fast as a native implementation, if done correctly.

Comment 6

[noitidart]

Message for other ext devs that stumble upon here: An alternative to base64 encode the data, I think they can also do a new Uint8Array(new ArrayBuffer(1000)) (optionally .toString()) and deal with a uint8_t array on the other end. This worked for me.

Comment 7

[BMO Automation]

Per policy at https://wiki.mozilla.org/Bug_Triage/Projects/Bug_Handling/Bug_Husbandry#Inactive_Bugs. If this bug is not an enhancement request or a bug not present in a supported release of Firefox, then it may be reopened.