Last Comment Bug 891286 - Implement Streams API
: Implement Streams API
Status: RESOLVED DUPLICATE of bug 1128959
:
Product: Core
Classification: Components
Component: DOM (show other bugs)
: Trunk
: All All
P5 normal with 2 votes (vote)
: ---
Assigned To: Nobody; OK to take it and work on it
:
: Andrew Overholt [:overholt]
Mentors:
Depends on:
Blocks: 907468 1056444
  Show dependency treegraph
 
Reported: 2013-07-09 02:37 PDT by Romain Gauthier [:tOkeshu]
Modified: 2016-02-11 08:22 PST (History)
27 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments

Description User image Romain Gauthier [:tOkeshu] 2013-07-09 02:37:27 PDT
The W3C has a Working Draft for the Streams API.

http://www.w3.org/TR/streams-api/

It would be really useful to see that implemented in Firefox for applications that require the processing of large files with JavaScript.

Example use cases:

- End-to-end client side encryption
- Large file processing
- Piping files download/upload between 2 different sources
- Large file sharing in the context of WebRTC

My main use case is the last one. For now, to download a file sent via WebRTC DataChannels we have to load the whole file in memory to create a blob url. This reduce our ability to share large files (see https://github.com/Peer5/ShareFest/issues/21 for this kind of issue).

The Streams API should solve this problem via it's URL extensions.

Here is a few applications that could take advantage of these API:

https://github.com/mozilla/talkilla
https://github.com/Peer5/ShareFest
https://github.com/ShareIt-project/ShareIt
Comment 1 User image Olli Pettay [:smaug] 2013-07-09 06:11:15 PDT
Stream API as such isn't quite what we want. See the discussion on the webapps wg mailing list.

Gecko-only XHR responseTypes moz-chunked-text and moz-chunked-arraybuffer are closer to what is needed.
Comment 2 User image Anne (:annevk) 2013-07-09 07:13:04 PDT
Yes, Streams API is going to change. We should not implement that draft.
Comment 3 User image Jonas Sicking (:sicking) No longer reading bugmail consistently 2013-07-20 16:52:47 PDT
I think we could do a prototype implementation of the API drafted here though:

http://lists.w3.org/Archives/Public/public-webapps/2013AprJun/0727.html

This also has support for the "chunked" mode.

I believe google is working on a spec draft based off of that. IIRC that draft has some small changes compared to the email above, but they are largely the same.

Anne, do you have a link to that draft?
Comment 5 User image Domenic Denicola 2014-07-15 13:42:17 PDT
Marcos asked me to give an update on the streams spec's progress. For those not aware, we have moved development to https://github.com/whatwg/streams. See also https://whatwg.github.io/streams/ (temporary URL) which contains more of the explanatory text and examples, and will eventually contain the algorithms currently prototyped at the former URL.

Since this now part of my full-time job, we have made great progress on nailing down the spec and the accompanying polyfill and test suite. The remaining outstanding pieces are:

- Finishing the work on transform streams. This is being prototyped in a branch at [1], see e.g. [2] and [3].
- Building a quick-and-dirty performance-testing suite, and using the gathered data to decide if we can simplify some APIs which are currently potentially-overengineered in the name of performance. This is being done in a branch at [4].

The second point is the most interesting, as it could change the API surface in various ways: some subtle, some more complicated. That is why I am rushing to get it done as soon as possible---probably within the next week or two. The first is close to done; I started it to see how hard it would be, and have found that it's not very hard, so I am putting it on the backburner while I finish the perf test and related potential simplifications.

I know that tentative steps have started toward an implementation in Chrome, see [5] and especially the recently-filed [6], plus related discussions at [7].

In terms of spec-readiness timeline, my goal is for it to be completely finished, including a prototype and test suite, by end of Q3. If there are things that you guys would like sooner let me know and I can do my best to accommodate.

[1]: https://github.com/whatwg/streams/tree/transform
[2]: https://github.com/whatwg/streams/blob/394bb5e7e19c3455ce0803e227d2ee05e2f5831f/reference-implementation/lib/transform-stream.js
[3]: https://github.com/whatwg/streams/blob/394bb5e7e19c3455ce0803e227d2ee05e2f5831f/reference-implementation/test/transform-stream.js
[4]: https://github.com/whatwg/streams/tree/perf-test-suite/reference-implementation
[5]: https://code.google.com/p/chromium/issues/detail?id=240603
[6]: https://code.google.com/p/chromium/issues/detail?id=393911
[7]: https://groups.google.com/a/chromium.org/forum/#!topic/blink-dev/GoFbe0yLO50
Comment 6 User image :Harald Kirschner :digitarald 2015-10-30 14:04:46 PDT
Superseded by bug 1128959 and the WHATWG Streams spec.

*** This bug has been marked as a duplicate of bug 1128959 ***

Note You need to log in before you can comment on or make changes to this bug.