Open Bug 1474543 (WritableStream) Opened 2 years ago Updated 2 months ago

Implement WritableStream

Categories

(Core :: JavaScript Engine, enhancement, P2)

enhancement

Tracking

()

People

(Reporter: jya, Assigned: jorendorff)

References

(Depends on 1 open bug, Blocks 4 open bugs)

Details

(Keywords: dev-doc-needed)

I found no bug created for this.

This is to track implementation of WritableStream
https://developer.mozilla.org/en-US/docs/Web/API/WritableStream

https://streams.spec.whatwg.org/#ws

It doesn't appear to be supported at this stage, and various media 3rd parties have shown their interest.
Priority: -- → P3
Component: DOM → JavaScript Engine
Priority: P3 → P2
Alias: WritableStream
At Facebook we've recently shipped improvements to our video players utilizing the full set of Stream APIs, including WritableStream. Currently only users of browsers supporting these APIs get to benefit from the new experience. We would love to be able to ship this to Firefox users too, but are blocked on availability of the WritableStream API.
Blocks: streams-meta
Assignee: nobody → jorendorff

Our use case for this with Peergos is downloading or streaming large encrypted files which are reassembled and decrypted in the browser.

For an example see the following. This opens a 120MB version of Sintel and streams it to a video element. In Chrome this is done 5MiB at a time so it can start much faster than in Firefox where it has to fallback to loading the entire video into RAM and decrypting (the keys are in the url hash fragment) before it can start playing:

https://demo.peergos.net/#pQd8rmrEhBN1XBodCnuhegoEumwDbiFEjkYongNGkhRSAk1anF74Yv4t/pQd8rmrEhBN1PqDwhTEcvK5ezKzQ1UWFPmehpE146BxZpC6vHdrAnmZv/CX9xfVdKF7tPW2LRjz3RvoePmwAsNVsTAkAftgrbqKhW/5Pf7SvvCW9dLjFaQB5GXRHuQ5HZU6cjpkVU7vxzzLtgmPt6ULV9?streaming=true&open=true

This means that currently in Firefox we are limited to loading files that will fit into ram, which means we have to recommend Chrome. We would love to be able to recommend Firefox given our privacy focus.

Blocks: 1323414

(In reply to ianopolous from comment #2)

Our use case for this with Peergos is downloading or streaming large encrypted files which are reassembled and decrypted in the browser.

For an example see the following. This opens a 120MB version of Sintel and streams it to a video element. In Chrome this is done 5MiB at a time so it can start much faster than in Firefox where it has to fallback to loading the entire video into RAM and decrypting (the keys are in the url hash fragment) before it can start playing:

https://demo.peergos.net/#pQd8rmrEhBN1XBodCnuhegoEumwDbiFEjkYongNGkhRSAk1anF74Yv4t/pQd8rmrEhBN1PqDwhTEcvK5ezKzQ1UWFPmehpE146BxZpC6vHdrAnmZv/CX9xfVdKF7tPW2LRjz3RvoePmwAsNVsTAkAftgrbqKhW/5Pf7SvvCW9dLjFaQB5GXRHuQ5HZU6cjpkVU7vxzzLtgmPt6ULV9?streaming=true&open=true

This means that currently in Firefox we are limited to loading files that will fit into ram, which means we have to recommend Chrome. We would love to be able to recommend Firefox given our privacy focus.

I'm curious on why you would need Writeable Stream to do this.

There are lots of Web sites having that exact requirements, none of them using writeable streams.

(In reply to Jean-Yves Avenard [:jya] from comment #3)

(In reply to ianopolous from comment #2)

Our use case for this with Peergos is downloading or streaming large encrypted files which are reassembled and decrypted in the browser.

For an example see the following. This opens a 120MB version of Sintel and streams it to a video element. In Chrome this is done 5MiB at a time so it can start much faster than in Firefox where it has to fallback to loading the entire video into RAM and decrypting (the keys are in the url hash fragment) before it can start playing:

https://demo.peergos.net/#pQd8rmrEhBN1XBodCnuhegoEumwDbiFEjkYongNGkhRSAk1anF74Yv4t/pQd8rmrEhBN1PqDwhTEcvK5ezKzQ1UWFPmehpE146BxZpC6vHdrAnmZv/CX9xfVdKF7tPW2LRjz3RvoePmwAsNVsTAkAftgrbqKhW/5Pf7SvvCW9dLjFaQB5GXRHuQ5HZU6cjpkVU7vxzzLtgmPt6ULV9?streaming=true&open=true

This means that currently in Firefox we are limited to loading files that will fit into ram, which means we have to recommend Chrome. We would love to be able to recommend Firefox given our privacy focus.

I'm curious on why you would need Writeable Stream to do this.

There are lots of Web sites having that exact requirements, none of them using writeable streams.

Can you name one? What technique do they use? The closest I'm aware of is Mega, and they have the exact same problem. In Chrome they solve it with a Chrome specific file system api, and in Firefox they resort to loading it into RAM, as I've suggested, and downloading from a blob.

The general problem is for large files that are generated in the browser to be streamed directly to a host file (or or video tag). We generate them by reassembling fragments and decrypting in the browser with keys that are not exposed to the server.

P.S. N.B. the link I gave longer works - we are in middle of changing some things. I'll post another link when we're done.

Duplicate of this bug: 1530363

As promised, here is an updated link to streaming Sintel using writable streams if they are present. This can start within a few seconds in Chrome, but in Firefox has to wait until the entire 120mb video is downloaded and decrypted (and it may crash if it doesn't fit in ram):

https://demo.peergos.net/#6MDZhRRPT4ugkJuUfcPPhf1US9u7FvRALmj42mJ6e3yDibnLtqfhchE6Frm6Lf/6MDZhRRPT4ugkJuUfcZdxu6JLKyrLBE36Kasxb4jix7An4dbeiekpDF6h2fDBM/G5KGmeQzGbaA16NGXRR6MtzkTw54YABd8wxaYxZKecth/5Pf7SvriCWAvPWM2ubDhCXbFasZFmPTZEyjHLKJKa26tYxCxN1R?open=true

Blocks: 1502355
Depends on: 1582348

Can an update be provided on this ticket?
I can see that writeable streams has been in nightly for a number of releases. The release build has functionality behind javascript.options.writable_streams, which when enabled works for our purposes.
Is it expected to be enabled by default soon?
In Firefox 74 I have noticed that the statement:
'serviceWorker' in navigator.
Is now also returning false in the release build; nightly returns true.

I don't plan to enable this in nightly (and then forward into releases) until bug 1502355 is fixed. From what people have been telling us, generally writable streams only is not what most people want, they also want piping at the same time.

Moreover, the number of web-platform-tests for writable streams is not particularly comprehensive enough, for me to really want to trust shipping what we've implemented. I'd prefer us to have the additional pounding that piping will afford us, before I flip this on.

(In reply to Jeff Walden [:Waldo] from comment #8)

I don't plan to enable this in nightly (and then forward into releases) until bug 1502355 is fixed. From what people have been telling us, generally writable streams only is not what most people want, they also want piping at the same time.

Moreover, the number of web-platform-tests for writable streams is not particularly comprehensive enough, for me to really want to trust shipping what we've implemented. I'd prefer us to have the additional pounding that piping will afford us, before I flip this on.

Until all the parts are working (i.e. writable streams and piping) it will not be a huge improvement to have writable streams. Also it's easier to workaround the absence of WritableStream than the absence of interoperability of streams with other APIs (e.g. fetch PUT requests which don't accept ReadableStream in any browser AFAIK).

You need to log in before you can comment on or make changes to this bug.