Closed Bug 891286 Opened 11 years ago Closed 9 years ago

Implement Streams API


(Core :: DOM: Core & HTML, defect, P5)






(Reporter: rgauthier, Unassigned)


(Blocks 2 open bugs)


The W3C has a Working Draft for the Streams API.

It would be really useful to see that implemented in Firefox for applications that require the processing of large files with JavaScript.

Example use cases:

- End-to-end client side encryption
- Large file processing
- Piping files download/upload between 2 different sources
- Large file sharing in the context of WebRTC

My main use case is the last one. For now, to download a file sent via WebRTC DataChannels we have to load the whole file in memory to create a blob url. This reduce our ability to share large files (see for this kind of issue).

The Streams API should solve this problem via it's URL extensions.

Here is a few applications that could take advantage of these API:
Stream API as such isn't quite what we want. See the discussion on the webapps wg mailing list.

Gecko-only XHR responseTypes moz-chunked-text and moz-chunked-arraybuffer are closer to what is needed.
Flags: needinfo?(annevk)
Yes, Streams API is going to change. We should not implement that draft.
Flags: needinfo?(annevk)
I think we could do a prototype implementation of the API drafted here though:

This also has support for the "chunked" mode.

I believe google is working on a spec draft based off of that. IIRC that draft has some small changes compared to the email above, but they are largely the same.

Anne, do you have a link to that draft?
Blocks: 907468
Marcos asked me to give an update on the streams spec's progress. For those not aware, we have moved development to See also (temporary URL) which contains more of the explanatory text and examples, and will eventually contain the algorithms currently prototyped at the former URL.

Since this now part of my full-time job, we have made great progress on nailing down the spec and the accompanying polyfill and test suite. The remaining outstanding pieces are:

- Finishing the work on transform streams. This is being prototyped in a branch at [1], see e.g. [2] and [3].
- Building a quick-and-dirty performance-testing suite, and using the gathered data to decide if we can simplify some APIs which are currently potentially-overengineered in the name of performance. This is being done in a branch at [4].

The second point is the most interesting, as it could change the API surface in various ways: some subtle, some more complicated. That is why I am rushing to get it done as soon as possible---probably within the next week or two. The first is close to done; I started it to see how hard it would be, and have found that it's not very hard, so I am putting it on the backburner while I finish the perf test and related potential simplifications.

I know that tentative steps have started toward an implementation in Chrome, see [5] and especially the recently-filed [6], plus related discussions at [7].

In terms of spec-readiness timeline, my goal is for it to be completely finished, including a prototype and test suite, by end of Q3. If there are things that you guys would like sooner let me know and I can do my best to accommodate.

Superseded by bug 1128959 and the WHATWG Streams spec.
Closed: 9 years ago
Resolution: --- → DUPLICATE
Blocks: 1247634
No longer blocks: 1247634
Depends on: 1367375
No longer depends on: 1367375
Component: DOM → DOM: Core & HTML
You need to log in before you can comment on or make changes to this bug.