582808 - Add an asymmetric_rpc message semantic type to IPDL

Status: RESOLVED WONTFIX

(Core :: IPC, defect, P3)

Description

[Atul Varma [:atul]]

In Jetpack child processes, introduced in bug 556846, there is a global function called callMessage() which blocks the child process until a message is sent to the parent (chrome) process and an answer is received. The underlying IPDL implementation defined by js/jetpack/PJetpack.ipdl appears to use 'rpc' message semantics here, which are defined here:

  https://developer.mozilla.org/en/IPDL/Getting_Started#RPC_semantics

This means that the parent process is limited to responding to the request in its message handler, i.e. synchronously. However, since the Jetpack child process isn't actually coupled to the UI, we'd like to offer addon developers the kinds of developer-ergonomic ease of use available from DOM Workers on the Web platform.

What this means is that it would be great if synchronous calls could be made from Jetpack child processes that block the child process until a response is received from the parent--but the parent has the freedom to do something *asynchronous* on its end before sending its response. This would allow us to provide really simple, developer-ergonomic, blocking APIs to Jetpack developers without being tied down by what the parent process can do synchronously.

A simple example of this would be the case of the simple storage API: Jetpack code could use the API synchronously--similar to window.localStorage--and each access would block the child process, send a message to the parent process, which would asynchronously perform the access, and then return the result back to the child.

Presumably, in the implementation this could be done by having the message handler in the parent return a promise that it eventually fulfills; once it's fulfilled, the response would be sent as a message to the child process, which would immediately unblock it.

The reason I'm calling this "asymmetric_rpc" is because it appears to be synchronous on the child-side while being asynchronous on the parent-side, but I'm not particularly tied to the name.

Anyways, would love to know if this idea sounds insane or reasonable, or if it can actually already be done with the tools we have. Thanks!

Comment 1

[Benjamin Smedberg]

Yeah, we've talked about this before. It's possible, but the people who could implement it are kinda busy right now. How urgent is it, and what set of jetpack APIs really need it? Is it something we can postpone?

Comment 2

[Atul Varma [:atul]]

Cool, I think we can postpone it, I mainly filed this bug just to start the conversation and get it on your radar. I'll post to this bug again if/when it becomes a real priority.

Comment 3

[Atul Varma [:atul]]

I've implemented a terrible proof-of-concept hack we can use to get around this for now--it's not actually required in the situation in which it's being used here, but it should convey the general idea:

  http://github.com/toolness/oop-jetpack-sdk-poc/commit/373e7d24f10e9a482dec4125b92de9e8df087fe0

Basically, the child process blocks by constantly polling the parent process via callMessage() to see if it's done yet. We can abstract things such that once this bug is fixed, we just change a few lines of code and everything works as expected.

If this bug will take a really long time to fix, I can submit a patch to add a sleep() global function to the child process, so that at least the CPU isn't churning too hard while the polling occurs.

(Another idea that occurred to me is to have the parent process call nsIThreadManager.currentThread.processNextEvent() while waiting for the async event to occur, but that worries me a bit more; it also seems like it'd be harder to replace with the "real thing" once this bug is fixed.)

Let me know if you think this hack is too horrible for temporary use; otherwise we'll use it as a fallback if we need to.

Comment 4

[Benjamin Smedberg]

I think that hack is too horrible for even temporary use. Which jetpack API requires this behavior?

Comment 5

[Atul Varma [:atul]]

At the very least, we'd really like to use it for the simple storage API:

  https://jetpack.mozillalabs.com/sdk/0.6/docs/#module/jetpack-core/simple-storage

We got the API working in such a way that it *appears* to be synchronous, but it's actually writing to a database after-the-fact and asynchronously notifying the caller of quota failures rather than throwing an exception from the offending database write, which would be preferable.

Not what other cases there are, but it'd be nice to at least have the freedom to create more developer-ergonomic APIs in this way.

Comment 6

[Atul Varma [:atul]]

Hey Ben, is there any way this can be implemented anytime soon? Myk, Drew and I did a review of APIs last week and ran into a number of situations where having the functionality described in this bug would make Jetpack APIs significantly easier to use.

Comment 7

[Benjamin Smedberg]

I can talk to cjones, but I don't think you should expect this to be available for Firefox 4.

Comment 8

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

This is implementable, but I have some reservations about adding this interface, and additionally I don't understand all the requirements, especially concerning re-entry of parent messages while the child is blocked.  This might be best to discuss at the work week.

For now, you can send an async request to the parent and then spin a nested event loop in the child waiting for the async response message (in the spirit of http://mxr.mozilla.org/mozilla-central/source/services/sync/modules/ext/Sync.js#106).  This allows the jetpack code to decide for itself which messages can re-enter the "blocked" child.  I suspect this is probably the implementation we'd want in the long term, instead of new messaging semantics.

Comment 9

[Benjamin Smedberg]

Note that we would need to provide a primitive wait() function in the child side in order for that to be implementable. But I'm more than a little worried about unintended side effects of re-entry really confusing jetpack authors if you use this method (asymmetric_rpc or the async+yield): because the chrome process continues to send all kinds of notifications, it won't be uncommon for incoming messages to nest in unexpected ways.

Comment 10

[Atul Varma [:atul]]

Yeah, that is true. We thought it might be cool to queue all the notifications until the current sync call unblocks to prevent re-entry and maintain the simplicity of flow control for Jetpack authors, but I suspect that carries its own challenges and edge cases with it as well.

Let's talk during the all-hands!

Comment 11

[Andrew McCreight [:mccr8]]

Jetpack is no more, and I think we don't want more sync messages.