Closed Bug 925128 Opened 11 years ago Closed 11 years ago

allow sync calls to target the non-main thread

Categories

(Core :: IPC, defect)

Product:
Core
Component:
IPC
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED WONTFIX

People

(Reporter: luke, Unassigned)

Details

I have a use case where the child process wants to sync call into the parent, but the parent impl needs to run off the main thread.  (Specifically, the parent impl needs to run QuotaManager::EnsureOriginIsInitialized on the quota manager's IO thread.)

After talking about this a bit with dvander, I think, ideally, ipidl would allow us to say that a given method would be dispatched to some non-main thread by specifying a symbolic name.  A hacky workaround would be to have the child send an async message to the parent and then block on a reply from the parent, thereby allowing the parent to dispatch a runnable to a non-main-thread and reply when it's done.  (This would still be slightly suboptimal since it adds another thread hop.)  It seems nice to solve this problem once in the IPC layer, though, since I can imagine other uses for this functionality.
I think this will be hard... How do you manage the lifetime of these special target threads? Are they created/destroyed entirely by IPDL?
By my thinking, the target threads would be pre-existing (in my case, it'd be QuotaManager::IOThread), long-lived, and with the choices being statically enumerated by IPIDL.
The easiest/most general way to solve this would be with a new message protocol, that has the same semantics as "sync", but gives you a dispatch signature like:

  struct ReplyTo {
    int32_t seqno;
    

  bool Listener::ReceiveStuff(<inputs>, const ipc::ReplyTo &to);

Then you could post a message to another thread with the ReplyTo object, and asynchronously reply when needed:

  actor->ReplyToStuff(to, <params>);

This would be pretty straightforward to implement since replies are already asynchronous in the IPC implementation. Also, we'd only really need to provide a replacement for "sync" semantics; async wouldn't need this, and rpc/urgent/interrupt should only ever be handled on the main thread anyway.

The threading idea is sort of appealing since there are also use cases for that (like, the compositor has at least one message that just posts to the compositor thread). It's quite a bit more complex though, since MessageChannel isn't threadsafe, you'd end up having cross-thread messages anyway.
Another threading solution would be to actually instantiate a MessageChannel on the QuotaManager thread, but MessageChannel has statics so that wouldn't be easy.
A new channel is the only sane way to do this. And we are already using multiple channels for OMTC, so I'm not sure what statics you mean, but if there are statics that would prevent multiple channels, they are probably already broken.
There are additional complications that make IPDL somehow referencing the QuotaManager thread really difficult. I'd rather not venture down that path unless we have no other options.
So if we make a new message channel for the quota manager thread, how do my messages get there?  I was under the impression that messages start on the ipc thread and then always get dispatched to the main thread.
MessageChannel messages are dispatched to its mWorkerLoop, which is whatever thread calls MessageChannel::Open.
Ok, cool, it looks like these message channels are basically what I need.

So then is it the parent actor that gets to specify the MessageChannel which will receive messages?  (Sorry if these are basic questions.)
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → WONTFIX
Yes. PTopLevelActor::Open forwards to its internal channel, so you create the toplevel actor and then .Open() it on the QuotaManager thread.

I'm not exactly sure how we deal with connecting that to a child, but I know that we have a bridge system of some sort because that's how cross-process OMTC works.
So cross-process OMTC is another case where a child message gets routed directly to a non-main thread (well, after going through the parent's ipc thread)?  That'd be great if I could have another existing impl to look at.
Yes, I am almost certain of that. But I don't actually know how it works; benwa or somebody would have to guide you there.
This won't work for the QuotaManager IO thread since it uses LazyIdleThread. Or, at least, you'll have to do some extra work to try and make it behave since it expects all runnables to be dispatched to it from a single thread and it can time out and shutdown automatically.
(In reply to Benjamin Smedberg  [:bsmedberg] from comment #5)
> A new channel is the only sane way to do this. And we are already using
> multiple channels for OMTC, so I'm not sure what statics you mean, but if
> there are statics that would prevent multiple channels, they are probably
> already broken.

There are statics in MessageChannel for dealing with the Windows nested-event-loop craziness.
(In reply to ben turner [:bent] (needinfo? encouraged) from comment #13)
Is it LazyIdleThread or QuotaManager or both that are assuming that runnables are only being dispatched from a single thread?  Do you know what requires this assumption?
LazyIdleThread does for sure, I don't know if QuotaManager also assumes that. You could make QuotaManager hold a regular thread but then we'd always have a mostly useless thread hanging around...
Do you know how hard it would be to make LazyIdleThread allow multiple dispatching threads?
It might be tricky but to be honest it was just never a needed use case before now. I wrote it a long long time ago so I've forgotten most of the details. It basically just tries to keep track of the time since events were dispatched and run in order to time the thread out.
Ok, that's hopeful.

Another thought: how heavyweight is a MessageChannel?  I was wondering if we should/can easily create the MessageChannel lazily.
I don't know the use-case here, but typically I'd expect that IPDL would be in charge of creating the MessageChannel when needed. e.g. you'd start this whole process off by one side sending a message with a socket pair, which is used to set up the channel. Is that something you can do lazily?
In my use case, the child initiates the child-parent communication, so that is when we'd need to set up the channel if we did this lazily.  I'm happy to do that; do you see any potential problems with the child creating a new off-thread MessageChannel in the parent?

Just to check my rough understanding: after creating the socketpair, I assume the child would need to send the parent's descriptor over to the parent (presumably through PContent), and then, from the parent, we'd use this descriptor to construct the new MessageChannel/top-level actor on the QuotaManager IO thread.
That seems accurate, yes. I'm pretty hazy on some of the important details such as how you create the socket. I *think* IPDL has some plumbing to make this really easy, but I can't find it right now.
Anyhow, all this MessageChannel stuff sounds like a ton of work, especially if I wanted to make it all lazy (b/c this stuff should be rarely used), so at Jan's recommendation (which makes sense) I'm just going to have the child send an async message and sync-wait for a reply.

If I'm the only one doing this, that seems fine, but I think we should be on the lookout for other similar use cases because it'd be silly to have everyone doing the same dance and this use case doesn't seem particularly unique.
When you say "sync-wait for a reply" do you mean by spinning an event loop, or some IPC mechanism to syncwait?
(In reply to Benjamin Smedberg  [:bsmedberg] from comment #24)
I mean a condvar.Wait(), which is occurring on a non-main (compilation) thread.  It will be Notify()ed when the reply message from the parent is received (presumably on the main thread).
You need to log in before you can comment on or make changes to this bug.