Last Comment Bug 783190 - Sync message API for workers
: Sync message API for workers
Status: RESOLVED WONTFIX
Latest draft in comment 33 and/or 36 ...
: dev-doc-complete
Product: Core
Classification: Components
Component: DOM (show other bugs)
: unspecified
: All All
: -- normal with 3 votes (vote)
: ---
Assigned To: Nobody; OK to take it and work on it
:
Mentors:
Depends on: transferables 720949 911972
Blocks:
  Show dependency treegraph
 
Reported: 2012-08-16 00:39 PDT by Andrea Marchesini [:baku]
Modified: 2016-03-30 00:28 PDT (History)
45 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments
Bug 783190 - Sync API for workers (34.64 KB, patch)
2012-08-23 09:07 PDT, Andrea Marchesini [:baku]
no flags Details | Diff | Splinter Review
Bug 783190 - Sync API for workers (35.57 KB, patch)
2012-08-23 12:24 PDT, Andrea Marchesini [:baku]
no flags Details | Diff | Splinter Review
Bug 783190 - Sync API for workers (35.57 KB, patch)
2012-08-23 12:26 PDT, Andrea Marchesini [:baku]
no flags Details | Diff | Splinter Review
patch version 2 (36.58 KB, patch)
2012-08-24 12:35 PDT, Andrea Marchesini [:baku]
no flags Details | Diff | Splinter Review
patch 3 (37.40 KB, patch)
2012-08-24 13:25 PDT, Andrea Marchesini [:baku]
no flags Details | Diff | Splinter Review
sync2.patch (67.76 KB, patch)
2014-09-18 07:40 PDT, Andrea Marchesini [:baku]
no flags Details | Diff | Splinter Review

Description Andrea Marchesini [:baku] 2012-08-16 00:39:57 PDT
The API we had in mind for sync messages would be something like this:

worker.js:
var res = postSyncMessage(data, transferrable, timeout);
// all arguments are optional. We don't yet support trasferrables, more on that below
// if timeout expires, returns 'undefined'.

page.html:
worker = new Worker(...);
worker.onsyncmessage = function(event) {
  setTimeout(function() {
    event.reply(data, transferrable);
  }, 1000);
}

If the event object is GC'ed before someone calls .reply on it, we
should terminate the worker because we know it's indefinitely hung.
Comment 1 Jonas Sicking (:sicking) No longer reading bugmail consistently 2012-08-16 00:43:46 PDT
FWIW, we don't *have* to implement the "transferrable" argument before we implement the sync API. But we'd have to throw if anyone passes anything other than null/undefined in the transferrable argument.
Comment 2 David Bruant 2012-08-16 01:14:19 PDT
(In reply to Andrea Marchesini (:baku) from comment #0)
> The API we had in mind for sync messages would be something like this:
Sorry if my question is stupid, but who is "we"?
Is there a spec? A draft spec (even a wiki.mo page) at least? I see the component is "DOM", but I can't recall having seen a discussion on this in any standards mailing-list I follow.


> worker.js:
> var res = postSyncMessage(data, transferrable, timeout);
> // all arguments are optional. We don't yet support trasferrables, more on
> that below
Where so?

> // if timeout expires, returns 'undefined'.
> 
> page.html:
> worker = new Worker(...);
> worker.onsyncmessage = function(event) {
>   setTimeout(function() {
>     event.reply(data, transferrable);
>   }, 1000);
> }
Why is there suddenly a need for sync communication? Can't this be done using async communication?
The experience with blocking primitive is that people use them and it leads them into deadlocks. The JavaScript non-blocking model has preserved the web from that. Introducing sync primitives should be done with care in my opinion.


> If the event object is GC'ed before someone calls .reply on it, we
> should terminate the worker because we know it's indefinitely hung.
This sounds very backward. It implies that the ability to send async messages to a worker now also depends on replying to all sync messages and some related non-deterministic GC behavior.
I'm also unsure designing an API where a parallel unit is meant to hang indefinitely is a good idea.
Comment 3 :Ms2ger (⌚ UTC+1/+2) 2012-08-16 01:21:35 PDT
And the spec for this is where...
Comment 4 Jonas Sicking (:sicking) No longer reading bugmail consistently 2012-08-16 01:29:18 PDT
APIs with this feature set has been discussed on the webapps list a couple of times. Most recently at the thread starting at [1], which contains pointers to an even older thread starting at [2]

[1] http://lists.w3.org/Archives/Public/public-webapps/2011OctDec/0965.html
[2] http://lists.w3.org/Archives/Public/public-webapps/2010OctDec/1075.html
Comment 5 Jonas Sicking (:sicking) No longer reading bugmail consistently 2012-08-16 01:40:21 PDT
> And the spec for this is where...

I'm not sure what you mean by this. Are you saying that we can't implement something until there is an official spec draft?

Obviously we should bring this to the webapps list as a proposal though. But filing a bug for it first should be ok I would think?
Comment 6 David Bruant 2012-08-16 02:01:19 PDT
(In reply to Jonas Sicking (:sicking) from comment #5)
> > And the spec for this is where...
> 
> I'm not sure what you mean by this. Are you saying that we can't implement
> something until there is an official spec draft?
I personally don't care about "official", but a spec draft more detailed than comment O would be nice in my opinion.

> Obviously we should bring this to the webapps list as a proposal though.
> But filing a bug for it first should be ok I would think?
If consensus among vendors is to not have a sync worker API in the web platform, I don't see why filing the bug is beneficial.

I have started to read the thread and to say the least, it seems that a the idea of a sync API meets resistance.

Also, some arguments fail to meet some reality [1]
"there are lots of synchronous APIs in Workers: File API, Filesystem API, IndexedDB.  These
are all new APIs, not historical accidents (like sync XHR in the UI
thread).  Synchronous (blocking) APIs in workers is entirely by design, in
order to allow writing clean, linear code instead of code that has to yield
to the calling environment all the time."
=> Node.js has file and database APIs that are async and people using them are not complaining. Quite the opposite I would say.
Synchronous APIs (especially for disk access!) are a performance bottleneck. I don't think "writing linear code" is worth the performance cost.
Hmm... I should bring that up on public webapps rather.

[1] http://lists.w3.org/Archives/Public/public-webapps/2011OctDec/0973.html
Comment 7 Jonas Sicking (:sicking) No longer reading bugmail consistently 2012-08-16 03:32:37 PDT
The bug I was talking about was this one.

And indeed. The proper place to discuss the API isn't here, but on the w3c webapps list.
Comment 8 Andrea Marchesini [:baku] 2012-08-23 09:07:51 PDT
Created attachment 654649 [details] [diff] [review]
Bug 783190 - Sync API for workers

This patch implements the sync API. It's just a proposal, so keep it as it is.
The current implementation does:

. postSyncMessage(data, transferable, timeout) for workers. Transferable is ignored. If the timeout is not null and > 0, postSyncMessage returns 'undefined' when it expires.

. onsyncmessage(event) is a event for the parent. The event has a reply(data, transferable) method that allow the callee to return data to the worker. Transferable is ignored here too.

The behaviours of posySyncMessage is:

1. if the reply(data) is called, postSyncMessage returns data.
2. if the timeout is set and it expires, postSyncMessage returns 'undefined'.
3. if the event is garbage collected, the worker is terminated if 1 or 2 do not happened yet.
Comment 9 Olli Pettay [:smaug] (vacation Aug 25-28) 2012-08-23 09:50:47 PDT
Multiple syncmessage listeners cause problems. Which reply should win?
It would be odd to restrict reply usage to the first or last event listener - whichever listener it is.

I think a simple callback object on (main thread) worker object would be better.

worker.syncMessageHandler = function(message) {
  setTimeout(function() {
    message.reply(data, transferrable);
  }, 1000);
}

message could be a simple object

interface SyncMessage
{
  readonly attribute Worker worker;
  void reply(any data, any tranferrable); // throws if called more than once.
};

This would force just one handler to take care of messaging.
Comment 10 Andrea Marchesini [:baku] 2012-08-23 12:24:34 PDT
Created attachment 654735 [details] [diff] [review]
Bug 783190 - Sync API for workers
Comment 11 Andrea Marchesini [:baku] 2012-08-23 12:26:26 PDT
Created attachment 654737 [details] [diff] [review]
Bug 783190 - Sync API for workers

I like the Olli's proposal but I would read what Ben and Jonas think about it.
This patch implements a test case for multiple reply() calls.
Comment 12 Jonas Sicking (:sicking) No longer reading bugmail consistently 2012-08-23 17:39:12 PDT
Actually, to allow libraries I think we should do something like this:

worker:
x = postSyncMessage("indexedDBData", data);

window:
worker.setSyncHandler("indexedDBData", function(message) {
  ...
  message.reply("hello");
});
Comment 13 Andrea Marchesini [:baku] 2012-08-24 12:35:05 PDT
Created attachment 655097 [details] [diff] [review]
patch version 2

A better management of the status of the response. Mochi test updated.
Comment 14 Andrea Marchesini [:baku] 2012-08-24 13:25:51 PDT
Created attachment 655126 [details] [diff] [review]
patch 3

Exception for multiple reply() calls.
Comment 15 Mounir Lamouri (:mounir) 2012-08-30 14:22:27 PDT
Andrea, could you go to the WHATWG mailing-list with the different proposals and see if we can find a consensus there? Given that will need to be standardized anyway, better to get some feedback sooner than later.
Comment 16 Mounir Lamouri (:mounir) 2012-08-30 14:29:08 PDT
Comment on attachment 655126 [details] [diff] [review]
patch 3

Review of attachment 655126 [details] [diff] [review]:
-----------------------------------------------------------------

Andrea, you need a peer to review that. Ben could do that. I will gladly super-review the API after the discussion from the WHATWG mailing-list.
Comment 17 Olli Pettay [:smaug] (vacation Aug 25-28) 2012-08-30 15:57:06 PDT
Worker stuff is discussed in WebApps WG mailing list.
Comment 18 David Bruant 2012-09-01 09:52:35 PDT
(In reply to Olli Pettay [:smaug] from comment #17)
> Worker stuff is discussed in WebApps WG mailing list.
I started a thread at http://lists.w3.org/Archives/Public/public-webapps/2012JulSep/0629.html
I summarize previous threads and raise concerns about the idea of a Sync API.
Comment 19 Ben Turner (not reading bugmail, use the needinfo flag!) 2013-05-21 15:13:02 PDT
Comment on attachment 655126 [details] [diff] [review]
patch 3

Clearing review until the spec stuff is worked out.
Comment 20 Jonas Sicking (:sicking) No longer reading bugmail consistently 2013-07-12 17:52:26 PDT
This was the proposal on the mailing list that seemed to work best:

[Constructor]
interface SyncMessageChannel {
  readonly attribute MessagePortSyncSide syncPort;
  readonly attribute MessagePortAsyncSide asyncPort;
};

interface MessagePortSyncSide {
  void postMessage(any message, optional sequence<Transferable> transfer);
  any waitForMessage();
  void close();
};
MessagePortSyncSide implements Transferable;

interface MessagePortAsyncSide : EventTarget {
  void postMessage(any message, optional sequence<Transferable> transfer);
  void start();
  void close();

  attribute EventHandler onmessage;
};
MessagePortAsyncSide implements Transferable;


There's the additional limitation that MessagePortSyncSide can
only be transferred though MessagePortAsyncSide.postMessage() or
Worker.postMessage(), and MessagePortAsyncSide can only be transferred
though MessagePortSyncSide.postMessage() or
DedicatedWorkerGlobalScope.postMessage().

Note that even MessagePortAsyncSide is fully API compatible with the
normal MessagePort, but code can still differentiate between the two
by using the instanceof operator or Object.toString().
Comment 21 Vladimir Vukicevic [:vlad] [:vladv] 2013-07-25 14:44:17 PDT
Adding azakai for feedback.
Comment 22 Alon Zakai (:azakai) 2013-07-25 16:27:40 PDT
API looks fine to me (my use case doesn't need anything that sophisticated, but this would work for it).
Comment 23 Jonas Sicking (:sicking) No longer reading bugmail consistently 2013-07-30 15:53:04 PDT
Unassigning from Andrea since I don't believe he's actively working on this.

Andrea, feel free to take this up again if you want to work on it.
Comment 24 Jonas Sicking (:sicking) No longer reading bugmail consistently 2014-01-31 12:51:05 PST
Sounds like Andrea had the cycles to take this back up again! Woot!
Comment 25 K. Gadd (:kael) 2014-08-07 15:50:48 PDT
(In reply to Jonas Sicking (:sicking) Vacation until July 24 from comment #20)
> This was the proposal on the mailing list that seemed to work best:
> 
> [Constructor]
> interface SyncMessageChannel {
>   readonly attribute MessagePortSyncSide syncPort;
>   readonly attribute MessagePortAsyncSide asyncPort;
> };
> 
> interface MessagePortSyncSide {
>   void postMessage(any message, optional sequence<Transferable> transfer);
>   any waitForMessage();
>   void close();
> };
> MessagePortSyncSide implements Transferable;
> 
> interface MessagePortAsyncSide : EventTarget {
>   void postMessage(any message, optional sequence<Transferable> transfer);
>   void start();
>   void close();
> 
>   attribute EventHandler onmessage;
> };
> MessagePortAsyncSide implements Transferable;
> 
> 
> There's the additional limitation that MessagePortSyncSide can
> only be transferred though MessagePortAsyncSide.postMessage() or
> Worker.postMessage(), and MessagePortAsyncSide can only be transferred
> though MessagePortSyncSide.postMessage() or
> DedicatedWorkerGlobalScope.postMessage().
> 
> Note that even MessagePortAsyncSide is fully API compatible with the
> normal MessagePort, but code can still differentiate between the two
> by using the instanceof operator or Object.toString().

No timeouts in the proposed API, now? That seems like a step backward. I guess you could implement timeouts using a coordination worker that automatically sends you a message after some amount of time has expired, but I think that would be much harder to implement without races in user JS than it would be in the browser.
Comment 26 Jonas Sicking (:sicking) No longer reading bugmail consistently 2014-08-07 16:04:03 PDT
There's actually still uncertainty about if this API is really needed.

What's the main use-case that you have? Is it compiled C++ codebases which use sync disk-IO APIs?
Comment 27 K. Gadd (:kael) 2014-08-07 16:31:31 PDT
I need blocking Thread.Sleep and blocking mutexes. Need. Without them .NET code using sync primitives will basically never run. Doing compile time callback-passing transforms on an entire application to implement *specific* sync methods, and having to propagate that out through every method that calls them, is basically impossible if I want to preserve other aspects of runtime behavior. (I will note that C# *does* have affordances for non-blocking async stuff, so the present approach is basically 'your sync code doesn't work', and that's not the *worst* thing in the world...)

I suspect emscripten is in the same boat: Doing safe, reliable callback-passing transforms on arbitrary C to implement synchronous primitives is not particularly easy. I'm not sure if you can even do it at all? I have no idea how the stack would work at that point. JSIL at least gets to cheat there since I use JS locals.

I'm okay with using a coordinator worker to implement the features I need. So just sync 'waitForMessage' is enough for me to do everything else myself, I think. Sync postMessage with a return value would certainly be welcome because it's more specific and implementing that yourself without races/bugs is relatively non-trivial (I guess maybe a library could do it for you?)

Other things in C# are synchronous as well and not easily adaptable to async. One major example is assembly loading (for loading plugins, etc), another is key parts of the file IO API. Right now I have to explicitly preload all the assemblies I might ever need at page load time, and I have to preload any files I might ever need and store them in memory forever. With sync messages I'd be able to make the assembly load and file open operations sync and have them work correctly without preloading everything (though in practice, yes, this is not ideal). Once a file is opened .NET has proper callback-oriented async read/write primitives, so it would be possible to stream the actual file contents instead of having to block on the whole file during the open operation.
Comment 28 Alon Zakai (:azakai) 2014-08-07 16:56:16 PDT
Emscripten has similar issues. A summer intern project has experimented with transforming code to be asynchronous,

https://github.com/kripken/emscripten/wiki/Asyncify

This works on many things, but there is substantial overhead in terms of code size and performance, this is not free in any sense. Basically it requires carefully keeping the stack alive and being able to return to the same place. One way to do it might be generators, but lack of support in browsers and in asm.js led us to try it at the LLVM level. We break up and duplicate functions so that we can resume execution in them later, after manually recreating the stack.

A sync message API would make this much easier and more practical. As it is, the experiment shows it is possible, but only useful where performance does not matter that much.
Comment 29 David Bruant 2014-08-08 02:24:21 PDT
(In reply to Alon Zakai (:azakai) from comment #28)
> Basically it
> requires carefully keeping the stack alive and being able to return to the
> same place. One way to do it might be generators, but lack of support in
> browsers and in asm.js led us to try it at the LLVM level.
Interesting. Have you experimented with generators? What is the perf situation if you use them?
At the pace of things, generators are more likely to reach broad browser adoption (already in FF and Chrome) than a Sync message API.
(I'm happy to continue this conversation elsewhere, because this bug may not be the appropriate place)
Comment 30 Alon Zakai (:azakai) 2014-08-08 12:37:00 PDT
We did some tests with generators, but the current implementations in firefox and chrome are not fully optimized yet, and there is a big speed downside because of that. I don't remember exactly how much, but it was enough to get us to not use them. If they are optimized, though, then they might be a possibility.
Comment 31 Andrea Marchesini [:baku] 2014-08-30 00:03:14 PDT
(In reply to Jonas Sicking (:sicking) from comment #26)
> There's actually still uncertainty about if this API is really needed.

sicking, do we want to proceed with this API? With the current patches for MessagePort/MessageChannel I proposed for bug 911972, this API is very easy to implement.
Comment 32 Andrea Marchesini [:baku] 2014-08-30 06:32:39 PDT
I think the correct WebIDL spec should be:

[Constructor]
interface SyncMessageChannel {
  readonly attribute SyncMessagePort syncPort;
  readonly attribute MessagePort asyncPort;
};

interface SyncMessagePort : MessagePort {
  [Throws]
  any waitForMessage();
};

In this way we can transfer the asyncPort as any other MessagePort.
The SyncMessagePort can be still used as a normal MessagePort, but it offers "waitForMessage" method.
SyncMessagePort cannot be transferred.

WaitForMessage throws if the entangled port (the asyncPort) has not been shipped.
Comment 33 Jonas Sicking (:sicking) No longer reading bugmail consistently 2014-09-05 19:16:30 PDT
If the async side can be passed around arbitrarily that means that it's very easy to end up with a deadlock. See the last two paragraphs of comment 20.

Apart from that, if it's easy to implement this, I think we should implement it and enable it on m-c and aurora. There's definitely a risk that it won't go anywhere, but we've been stuck here for a while and it'd be good to get feedback.

That said, there are some known limitations in the API, which we could attempt to fix (again, we should stick with what's easy).

* We should enable doing a non-blocking waitForMessage() which simply returns 'undefined' if there's no
  pending message. So something like getMessageIfPending().
* We should enable getting notified when there's a pending message.
* We should enable a "select" operation. I.e. blocking until there's a message from one of several
  available ports.
* Adding a timeout to waitForMessage() is probably not a bad idea.


So something like:



[Constructor]
interface SyncMessageChannel {
  readonly attribute MessagePortSyncSide syncPort;
  readonly attribute MessagePortAsyncSide asyncPort;
};

dictionary MessagePortSyncSideSelectResult {
  MessagePortSyncSide port;
  any message;
}

interface MessagePortSyncSide : EventTarget {
  void postMessage(any message, optional sequence<Transferable> transfer);
  any waitForMessage(optional long timeout = 0);
  any getMessageIfPending();
  void close();

  static MessagePortSyncSideSelectResult waitForMessageFrom(sequence<MessagePortSyncSide> ports,
                                                            optional long timeout = 0);

  /*
   * This event does not expose the actual event. You have to call waitForMessage()
   * or getMessageIfPending() in order to retrieve the message.
   *
   * This event fires asynchronously when a message is available. If a message is received,
   * but the list of pending messages is exhausted before the event fires, that will prevent
   * the event from firing.
   *
   * If waitForMessage/getMessageIfPending is called when there are multiple pending messages
   * the event will be scheduled to asynchronously fire. However, again, if the list of
   * pending messages is exhausted before the event fires, that cancels the event.
   */

  attribute EventHandler messagepending;
};
MessagePortSyncSide implements Transferable;

interface MessagePortAsyncSide : EventTarget {
  void postMessage(any message, optional sequence<Transferable> transfer);
  void start();
  void close();

  attribute EventHandler onmessage;
};
MessagePortAsyncSide implements Transferable;


Potentially we could combine waitForMessage() and getMessageIfPending() by treating a 0 timeout as a non-blocking get.
Comment 34 K. Gadd (:kael) 2014-09-05 21:46:46 PDT
waitForMessage w/timeout and a select primitive would both be extremely welcome. They're nontrivial to implement yourself in userspace. Non-blocking 'getMessage' of some sort sounds handy too, though a waitForMessage w/timeout=0 seems equivalent?
Comment 35 Ben Turner (not reading bugmail, use the needinfo flag!) 2014-09-05 22:00:31 PDT
(In reply to Jonas Sicking (:sicking) from comment #33)
> Potentially we could combine waitForMessage() and getMessageIfPending() by
> treating a 0 timeout as a non-blocking get.

I like this. And the rest of it.
Comment 36 Andrea Marchesini [:baku] 2014-09-07 04:06:43 PDT
> * We should enable doing a non-blocking waitForMessage() which simply
> returns 'undefined' if there's no

waitForMessage MUST return a MessageEvent (or an object with similar attributes) otherwise you cannot receive the transferred object and you cannot have the target.
Think about this:

var a = new MessageChannel();
asyncPort.postMessage(42, [ a.port1 ]);

The syncPort must have something like:

var me = syncPort.waitForMessage();
me.ports[0].onmessage =... // this is the port1
me.target.postMessage("okidoki");

> dictionary MessagePortSyncSideSelectResult {
>   MessagePortSyncSide port;
>   any message;
> }

If we return MessageEvent, we don't need this. the 'port' is the target of the event.

About MessagePortAsyncSide, why we need to have an interface equal to MessagePort? Maybe we can have:

// This port can only be transferred though MessagePortSyncSide.postMessage()
// or DedicatedWorkerGlobalScope.postMessage().
interface MessagePortAsyncSide : MessagePort {}

// A nice comment here about when this port can be transferred.
interface MessagePortSyncSide : MessagePort {
  MessageEvent waitForMessage(optional long timeout = 0);
  MessageEvent getMessageIfPending();
  
  static MessageEvent waitForMessageFrom(sequence<MessagePortSyncSide> ports,
                                         optional long timeout = 0);
};
Comment 37 Olli Pettay [:smaug] (vacation Aug 25-28) 2014-09-07 11:23:40 PDT
(if we need getMessageIfPending, I'd call it takePendingMessage or some such.)
Comment 38 Jonas Sicking (:sicking) No longer reading bugmail consistently 2014-09-07 22:16:31 PDT
(In reply to Andrea Marchesini (:baku) from comment #36)
> > * We should enable doing a non-blocking waitForMessage() which simply
> > returns 'undefined' if there's no
> 
> waitForMessage MUST return a MessageEvent (or an object with similar
> attributes) otherwise you cannot receive the transferred object and you
> cannot have the target.
> Think about this:
> 
> var a = new MessageChannel();
> asyncPort.postMessage(42, [ a.port1 ]);
> 
> The syncPort must have something like:
> 
> var me = syncPort.waitForMessage();
> me.ports[0].onmessage =... // this is the port1

.ports was added before we had the structured-clone algorithm. Since structured-clones were invented there's really no reason for .ports to exist any more. If you want to send the message above you can just do:

asyncPort.postMessage([42, a.port1], [a.port1]);

var me = syncPort.waitForMessage();
me[1].onmessage = ...;


I tried to get .ports dropped, but since it had already shipped it didn't seem worth it to put up a big fight about it. But I really believe it's unnecessary and just makes for a clumsier API.

> me.target.postMessage("okidoki");

this would just be syncPort.

Event.target is mainly useful if you have a single event handler which handles events from multiple event targets. This is fairly common in the Node-tree when you have tree of event targets, but seems much less useful here.

Any time you get a message, you *just* called waitForMessage on the port. So returning the port doesn't seem terribly useful. If you want to pass the message to some other party that may or may not have access to the port, you can just send the port along as well.

> > dictionary MessagePortSyncSideSelectResult {
> >   MessagePortSyncSide port;
> >   any message;
> > }
> 
> If we return MessageEvent, we don't need this. the 'port' is the target of
> the event.

That's true, but the MessageEvent object is even more cumbersome than MessagePortSyncSideSelectResult is (other than the name :) ). I'm somewhat reluctant to return an Event object when there's no Event dispatch happening here.

> About MessagePortAsyncSide, why we need to have an interface equal to
> MessagePort?

Mainly because MessagePortAsyncSide behaves different from MessagePort in that there are rules about how you can transfer a MessagePortAsyncSide. So it seems important to enable code to test the type before passing it anywhere.

> Maybe we can have:
> 
> // This port can only be transferred though MessagePortSyncSide.postMessage()
> // or DedicatedWorkerGlobalScope.postMessage().
> interface MessagePortAsyncSide : MessagePort {}

This seems fine with me yeah. The only real downside is that it means that |myAsyncSidePort instanceof MessagePort| tests true even though myAsyncSidePort does not behave like a real MessagePort in that the transfer rules are different.

But it does have the advantage that if you don't care about the transferring aspect, then it's good that |myAsyncSidePort instanceof MessagePort| tests true since in all other aspects myAsyncSidePort does behave like a MessagePort.
Comment 39 Jonas Sicking (:sicking) No longer reading bugmail consistently 2014-09-08 11:46:55 PDT
Ok, after chatting with Andrea, we'll aim to implement the following as an initial pass:

[Constructor]
interface SyncMessageChannel {
  readonly attribute MessagePortSyncSide syncPort;
  readonly attribute MessagePortAsyncSide asyncPort;
};

// Note that this is not transferrable, you'll have to create the SyncMessageChannel in
// the worker where you want to receive messages.
interface MessagePortSyncSide : EventTarget {
  void postMessage(any message, optional sequence<Transferable> transfer);

  // Not passing a timeout means "no timeout", i.e. block forever. Passing 0 as timeout
  // means attempt a non-blocking read. If getMessage(0) is called and there are no pending messages
  // the function immediately returns null.
  MessageEvent? getMessage(optional long timeout);

  void close();

  /*
   * This event is scheduled to fire any time the queue of incoming messages goes from being
   * empty to being non-empty. Unless already scheduled, the event is also scheduled to fire any time
   * getMessage() is called but the message queue contains more than one message. I.e. any time
   * getMessage() is called but that this does not exhaust the queue of incoming messages.
   *
   * The event is cancelled and removed from the event queue any time the queue of incoming messages
   * is exhausted and becomes empty.
   *
   * This event does not expose the actual event. You have to call getMessage()
   * in order to retrieve the message.
   */

  attribute EventHandler messagepending;
};

interface MessagePortAsyncSide : MessagePort {};
MessagePortAsyncSide implements Transferable;


So for now we're leaving out the select function as well as the ability to transfer the synchronous side. If this is a problem, please do provide use cases. The reason we're leaving this out for now is that this API is still quite controversial and might not stick as a cross-browser API, so we're doing the simplest parts first.

However the whole goal of implementing a prototype is to get developer feedback, so please do provide such feedback, including feedback about missing capabilities.
Comment 40 Jonas Sicking (:sicking) No longer reading bugmail consistently 2014-09-08 11:48:36 PDT
Err, the last part of the comment for the messagepending event should say

   * This event does not expose the actual message. You have to call getMessage()
   * in order to retrieve the message.
Comment 41 Olli Pettay [:smaug] (vacation Aug 25-28) 2014-09-08 13:03:30 PDT
attribute EventHandler messagepending; ->
attribute EventHandler onmessagepending;
Comment 42 K. Gadd (:kael) 2014-09-08 15:47:57 PDT
The lack of select() is probably less important given the presence of the messagepending event. I think you can implement a reasonable select() using the event handlers + timeoutless getMessage.

The new API changes all look great.
Comment 43 Andrea Marchesini [:baku] 2014-09-18 07:40:01 PDT
Created attachment 8491526 [details] [diff] [review]
sync2.patch

The patch is written. I'll ask for a review when MessagePort is landed because this patch is built on top of bug 911972.
Comment 44 Kyle Huey [:khuey] (Exited; not receiving bugmail, email if necessary) 2014-10-04 16:37:25 PDT
Please stop randomly flipping flags on bugzilla.  If you want to experiment use landfill.bugzilla.org.
Comment 45 Andrea Marchesini [:baku] 2015-06-19 08:41:34 PDT
Sicking, now that we have MessagePort landed, do we still care about this API?
Comment 46 Jonas Sicking (:sicking) No longer reading bugmail consistently 2015-09-10 16:57:13 PDT
Martin Best said he'd help pull together the right people so that we can make a decision here.
Comment 47 Andrea Marchesini [:baku] 2015-09-11 00:03:27 PDT
Thanks Jonas. Would be nice if we can write a spec for this new API and propose it as HTML5 standard.
From an implementation point of view, I think it's quite easy to write, and actually, with current patches, it should not take too much (if we stay close to what we discussed so far).
Comment 48 Martin Best (:mbest) 2015-09-14 11:48:45 PDT
Luke Wagner seems like the best person to get this started.  I'm also pulling in Alon Zakai.
Comment 49 Martin Best (:mbest) 2015-09-14 11:49:22 PDT
Same as above.
Comment 50 Alon Zakai (:azakai) 2015-09-14 12:31:25 PDT
I believe this would still be useful, but it does get less useful as more APIs are present in workers, like WebGL, and as more sync versions of APIs appear in workers, like FileReaderSync. However, assuming the main thread can do async operations while the worker waits synchronously, this would still allow solving a bunch of situations that have no sync version anywhere.
Comment 51 Jonas Sicking (:sicking) No longer reading bugmail consistently 2015-09-14 13:02:26 PDT
It's unlikely that we're going to get more Sync APIs available on workers beyond FildReaderSync. Alex Russel from Google has been pushing pretty hard against them, which means that they are unlikely to get traction.

I thought the big question here was if SharedArrayBuffer would address these use cases, and if so, on what timeline. Apparently the current SharedArrayBuffer API proposal exposes the full sync primitives to the main thread, but it wasn't clear to me that browsers are going to be ok with having the main thread block on worker threads.

Alternatively SAB could expose async variants of the SAB API on the main thread, but no such proposal is in the works.

Hence the unclarity about if/when SAB would provide a solution here.
Comment 52 Alon Zakai (:azakai) 2015-09-14 13:08:29 PDT
I think Lars could help out regarding those questions.
Comment 53 Luke Wagner [:luke] 2015-09-14 13:23:32 PDT
Agreed with the above on this feature being "nice to have" but not necessary.  Whether or not we get main-thread SAB access, we could still simulate a sync worker APIs using SAB in workers (albeit with the overhead of proxying through a helper worker).  If we can polyfill sync worker APIs in Chrome (using SAB), then that might help entice Chrome to implement sync worker APIs.
Comment 54 Jonas Sicking (:sicking) No longer reading bugmail consistently 2015-09-14 13:42:13 PDT
We're going over old discussions here :(

I had hoped that we would pull together the right people and make a decision here.

To be clear: I do not feel like I have enough information to make a decision about if we should implement this API and push it through the standards process or not.
Comment 55 Benjamin Smedberg [:bsmedberg] 2015-09-14 14:58:52 PDT
I don't have an opinion about standardization, but in terms of existing asynchronous Firefox UI code which we'd like to make less crazy, a sync API back to the main thread would be very helpful.
Comment 56 Lars T Hansen [:lth] 2015-09-15 02:00:37 PDT
The SAB work is being presented to ECMA TC39 next week.  As I've said in other fora, I'm not sure TC39 has either the inclination nor the jurisdiction to address whether the SAB APIs can be blocking on the main thread.  ECMAScript currently has no notion of concurrent agents at all and the SAB work introduces only the barest minimum; anyway ECMAScript is now used also outside the browser where a "main thread" is a less useful notion perhaps.  Still, no doubt several committee members will have opinions on the matter and can make their opposition heard in other standards bodies.  I think we won't know how that discussion will be going for a few months yet.

(IMO the SAB work - if left unrestricted - can be used to experiment with and even implement a good sync API, but I suspect that if we already know what a sync API should look like it would be better to just go ahead and implement it.)
Comment 57 Andrea Marchesini [:baku] 2015-09-24 09:26:38 PDT
ready to go? :)
Comment 58 Jonas Sicking (:sicking) No longer reading bugmail consistently 2015-09-24 14:56:32 PDT
I think it's Martin's decision. But based on Luke's and Alon's responses, it doesn't sound like this is important enough to do. But I might be misreading their responses.
Comment 59 Andre Vrignaud [:andre] [Seattle - PST] 2015-09-25 12:40:48 PDT
Having synced with Luke (who has discussed with Alon) it is fair to say this is not a priority from the games program perspective, especially with SAB appearing to be on track. However, others may have their own needs for this.

You can consider this the games program response.
Comment 60 K. Gadd (:kael) 2016-03-26 18:41:55 PDT
Is there any possibility of this ever happening? It's still needed and nothing else in the web platform appears to be mitigating its importance until SAB ships to end users. This is a (comparatively) small feature with multiple years of thought behind it that would solve real use cases well before then.
Comment 61 Jonas Sicking (:sicking) No longer reading bugmail consistently 2016-03-26 19:04:30 PDT
It seems like we've decided to use SAB to solve this use case. So it seems unlikely that we'll fix this bug as it is currently filed.
Comment 62 Lars T Hansen [:lth] 2016-03-27 07:35:12 PDT
(In reply to Jonas Sicking (:sicking) from comment #61)
> It seems like we've decided to use SAB to solve this use case. So it seems
> unlikely that we'll fix this bug as it is currently filed.

I haven't looked in details at the proposals backing this bug, but for the record be aware that SAB will have some restrictions that may affect its utility as polyfill here: It will (probably) only be possible to share memory between agents that are closely coupled already (eg a document and its dedicated workers), not arbitrarily (eg between a document and a SharedWorker that it can postMessage to).  The purpose of the restriction is to reduce the risk of deadlocks when different parts of the system are suspended / killed without the others.  I would say the specific restrictions are in flux but discussion is at least ongoing.

The ongoing discussion is not in one place, precisely (and unfortunately).  But this may be a good place to start: https://github.com/tc39/ecmascript_sharedmem/issues?q=is%3Aissue+is%3Aopen+label%3AAgents.  Issues 39 and 55 of that repo are probably particularly relevant.

Also see bug 1232973.

Note You need to log in before you can comment on or make changes to this bug.