601567 - Workers: concurrency issues and main event queue objects

Status: RESOLVED INCOMPLETE

(Core :: DOM: Core & HTML, defect)

Description

[passfree]

User-Agent:       Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-us) AppleWebKit/533.18.1 (KHTML, like Gecko) Version/5.0.2 Safari/533.18.5
Build Identifier: firefox3.6/xulrunner1.9.2.10

I've been experimenting with concurrency in the last couple of months and found one major problem.
 
1. Workers and XMLHttpRequest objects: 
Workers are a great way to do concurrency as long as you stick to simple for/while loops and math. The first major obstacle is the XMLHttpRequest object which if used extensively from a worker will likely to stall the browser, i.e. the main event queue. I believe the reason for this is because web requests in general execute from the main thread. This limitation is not however very clear to firefox/mozilla developers. Moreover the specs do not suggest that such limitation should exist and therefore if workers were supposed to be used for some serious background process crunching, this should be fixed or mitigated somehow. 

2. Workers and postMessage 
Similarly, if the worker produces too many massages with the postMessage function, it will likely cause the main thread to stall, which again, should not happen because it is unclear to the developer/user the underlaying reason. The Worker specs suggest that worker can be used for background intensive computation and it feels natural to post thousands of messages back when necessary: think of progress events of an intensive task. 

Reproducible: Always

Steps to Reproduce:
This is a snippet from a code which is used as part of a simple spider to demonstrate the issue:

web.sendAsynchronously('GET', 'http://test', 'HTTP/1.1', '', '', function (responseParts) {
	var f = arguments.callee;
	
	crawl.extractDestinationRequests(responseParts.data, responseParts.url).forEach(function (requestParts) {
		web.sendAsynchronously(requestParts.method, requestParts.url, requestParts.protocol, requestParts.headers, requestParts.data, f);
		
		dump(httpparse.buildRequestString({method:requestParts.method, url:requestParts.url, protocol:requestParts.protocol, headers:requestParts.headers, data:requestParts.data}));
	});
});

this will likely freeze the browser or crash it depending if it is executed from a worker or xpcom js.
Actual Results:  
The browser/app becomes unusable.

Expected Results:  
This shouldn't happen. As far as I am aware a lot of the networking tasks happen on the main thread but this should change. Workers should have their own event queues and all the networking tasks associated with the worker should happen there.

this could potentially become a major source for instabilities and security bugs

Comment 1

[Olli Pettay [:smaug][[e-mail]]]

Note, workers and xpcom js are very different things.
If you get any crashes, please file separate bugs for those.

About XHR in workers; I believe we use XHR in main thread mainly 
because of cycle collector and Necko API limitations, but bent should know more.

Would be great to have a testcase, hopefully attached using "Add an attachment".

Comment 2

[Olli Pettay [:smaug][[e-mail]]]

And btw, most of the networking code runs in background threads.

Comment 3

[Boris Zbarsky [:bzbarsky]]

The postMessage part of this looks invalid, and the XHR part I'm pretty sure is a dup...

Comment 4

[Ben Turner (not reading bugmail, use the needinfo flag!)]

XHR for workers uses a main-thread XHR object but all network activity happens on different threads. If you create thousands of XHR objects on a worker I would expect it to behave exactly as if you created thousands of XHR objects on the main thread. Perf may suck there, I guess, but surely you can get by without creating so many XHR objects? I'd be interested if you can prove a difference between main thread XHR and worker XHR here. The snippet posted in comment 0 tells us nothing, really. A reduced test case of the XHR slowdown you describe would be most helpful.

Posting tons of messages to the main thread can slow the browser, yes, but there are already tons of ways that web page script can do that. If you want people to use your page it would be best to minimize the number of messages that you post back. We may take steps to throttle worker messages in the future.

Comment 5

[Ben Turner (not reading bugmail, use the needinfo flag!)]

Not enough to go on here, please reopen if you have more information.