780744 - Have a native code component to expose JS array buffer as nsIInputStream

Status: NEW

(Core :: XPCOM, enhancement)

Description

[Honza Bambas (:mayhemer)]

Similar to nsIStringInputStream that takes stings.  Currently the only consumer would be TCP socket JS implementation using multiplex stream to buffer and send data.  JS implementation of array buffer to string conversion is extremely slow.

Best explained on an example:

  var arr = [0,1,2];
  var buf = new Uint8Array(arr);

  var stream = Cc.createInstance("@mozilla/buffered-array-input-stream;1").
               Ci.nsIBufferedArrayInputStream;
  stream.init(buf);

|stream| then exposes nsIInputStream that can be used to read the buffered array.

Best would be an implementation that adopts (don't copy) the data.

Comment 1

[Boris Zbarsky [:bzbarsky]]

If you adopt the data but then someone changes the typed array from JS, that will change the data you send.  Is that desirable?

Comment 2

[Donovan Preston [:fzzzy]]

Yeah, it seems bad if the js would be able to modify the typedarray at an arbitrary time between calling send and when the data is actually sent on another thread. Copying seems safest, but that's unfortunate. Is there any way to mark a typedarray as readonly so that js cannot change it after it has been passed to the networking thread?

Comment 3

[Boris Zbarsky [:bzbarsky]]

Typed arrays can be neutered; this is meant to be used to send them to a worker.  Similar things could be done here; that would make attempts to edit buf after the init() call throw exceptions.

Comment 4

[Ben Turner (not reading bugmail, use the needinfo flag!)]

We can't actually do that yet, we're waiting on JS engine folks in bug 720949 I think.

Comment 5

[Donovan Preston [:fzzzy]]

There hasn't been a comment on 720949 since march, is this likely to get done anytime soon? Can we bump up the priority?

Comment 6

[Honza Bambas (:mayhemer)]

Motivation here is to make the array buffer conversion in TCP socket faster.  We will probably soon land that feature on m-c, so people will start using it.  This may give us a clue on whether the conversion code really is a significant bottleneck.  I personally think it is, tho.  

For the first implementation copying is probably good enough.  

Not sure whether it is an issue to change the behavior later to make the array immutable after send() has been called.

Comment 7

[Boris Zbarsky [:bzbarsky]]

Sure seems to me like it would be, like anything that makes previously-fine code throw.

Comment 8

[Honza Bambas (:mayhemer)]

Agree.

Hence the first implementation will copy, as the current JS code in the tcp socket does.

We can later add an option to the tcp socket to adopt the data (off by default).  Then when some web app will copy large amount of data, it may use it to optimize the load.

BTW, I think we could have such option on the tcp socket interface right now, just unimplemented to make developers aware of the large memory copy issue and a way to improve it.