308158 - UI locks when scripts are running for a long time

Status: RESOLVED WORKSFORME

(Core :: DOM: Core & HTML, defect, P5)

Description

[Matthew Gertner]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.10) Gecko/20050716 Firefox/1.0.6
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.10) Gecko/20050716 Firefox/1.0.6

When running a long operation in Javascript, the UI is not repainted until the
entire operation has completed and the script returns control. As I understand
it, this is because scripts run in the UI thread and do not relinquish control
until completion. The practical result of this is that it is useless to have,
for example, a progressmeter to display the status of a processor-intensive
script, since no update of the script's progress is visible while the script is
running.

I did manage to work around this problem by running a separate thread in a JS
XPCOM component. However, I would consider the level of sophistication required
to get this to work to be far beyond the abilities of the average scripting
developer. If there is a reasonably straightforward way of performing periodic
UI updates automatically while a script is running, this would be highly desirable.

Reproducible: Always

Comment 1

[Brendan Eich [:brendan]]

Why is this a JS engine bug?  Cc'ers, please fix in the future.

Since Netscape 2, when JS was introduced, the execution model has been
"apparently single-threaded, run to completion".  That may require splitting
lengthy scripts up and chaining them via setTimeout, e.g.  The alternatives that
do not require any script changes are:

1) Run JS in a separate thread from the DOM and rendering code, _a la_ Netscape
4.  Having done that, it's my judgment that the costs outweight the benefit by
miles.  Any pseudo-threading of the JS engine in the main thread, using
continuation passing, a la Opera, is formally equivalent to, but just as hard or
harder than, using a thread.

2) Use the existing branch callback API, into which the DOM hooks, to do
something to help repaint.  This requires reentrancy in a big way.

If this bug has any value, it's about (2).  Reclassifying based on that.

/be

Comment 2

[Boris Zbarsky [:bzbarsky]]

Doing anything like that means that the page JS can be reentered while it's
executing, which seems highly undesirable from the page's point of view.

Put another way, if the UI can respond while the page JS is running, that means
the user can call random page JS functions (event handlers, onunload handlers,
etc) while said JS is running.  Most scripts out there can't deal with that. 

Comment 3

[Matthew Gertner]

Wouldn't it be possible to somehow lock the script while it is running so it
can't be reentered? I can't believe that making the UI completely unresponsive
is the best way to deal with reentrancy issues.

Comment 4

[Boris Zbarsky [:bzbarsky]]

> Wouldn't it be possible to somehow lock the script while it is running so it
> can't be reentered?

And do what, exactly, when relevant DOM events fire?  Just not run the event
handlers?

Comment 5

[Matthew Gertner]

At a functional level, I think the ideal solution would be:

1) to allow script reentrancy with some synchronization mechanism so that
threading issues can be avoided and
2) to disable all reentrancy by default as you suggest to prevent problems with
legacy scripts. When the DOM event handlers fire, a warning would be output to
the console.

The point of 1) would be to allow some kind of cancel functionality for long
operations (i.e. the script can receive a message and stop the long operation).
I know that script reentrancy and synchronization sounds very ambitious. This is
probably the Right Way to do it but perhaps someone more knowledgeable could
suggest a simpler, acceptable way.

My main point is still that the UI should be updated during long operations.
Ignoring DOM events while the script is running seems like a small price to pay.
Cancel functionality would be gravy, albeit very tasty gravy.

Comment 6

[Boris Zbarsky [:bzbarsky]]

> My main point is still that the UI should be updated during long operations.

There is no substantive difference between "the UI" and "the web page" from this
point of view, since they interact with each other via events and direct DOM access.

> Ignoring DOM events while the script is running seems like a small price to
> pay.

Doing that will effectively break lots of existing content.  That's by no means
a small price.

Comment 7

[Matthew Gertner]

I'm struggling to understand why this would break existing scripts. Currently,
event handlers can't be triggered while a script is running since the UI thread
is blocked. Under my proposal, the UI thread would no longer be blocked but the
event handlers still wouldn't run (by design in order to avoid reentrancy
issues). So, as far as I can see, the script behavior would remain the same,
apart from the fact that the UI (i.e. DOM) would be updated during script
execution to give the user feedback.

What am I missing?

Comment 8

[Caleb]

*** Bug 308376 has been marked as a duplicate of this bug. ***

Comment 9

[Caleb]

The main problem here IMO is that one tab can lock up the whole browser in some
cases, like loading this page:
http://archive.dojotoolkit.org/nightly/tests/animation/test_animation.html

Comment 10

[Boris Zbarsky [:bzbarsky]]

Say the UI changes the currently loaded URI.  Normally, that would fire an
unload handler.  With your proposal it would not.  Quite a number of sites
actually depend on such handlers running when the page unloads for proper
functioning...

So the problem is that the state of things could change without the right
notifications being sent if we don't fire events, thus causing a mismatch
between the actual state and the state the script expects.

Comment 11

[Hixie (not reading bugmail)]

Blue sky (Gecko 3.0):

There are various things that can be done. For example, disabling any UI in any
tab or window that can access a DOM that is accessible from an executing script.
Such disabling would only be shown in the UI if the script was taking long
enough that the user might have tried to use the UI two or three times; i.e. to
prevent flicker, in the common case nothing would be changed when the UI is
disabled in this way, it would only actually show in the UI after a second or two.

Comment 12

[Boris Zbarsky [:bzbarsky]]

Ian, any chrome window can access any DOM (via the window enumerator).

Comment 13

[Hixie (not reading bugmail)]

Yes; you only need to disable the UI that might actually do so though. Viwe
Source could remain enabled, for example.

I'll grant you that most things would end up disabled. But at least the UI
wouldn't be locked up.

Comment 14

[Boris Zbarsky [:bzbarsky]]

> Viwe Source could remain enabled, for example.

Why?  There's no reason an extension couldn't overlay code into the view-source
window that touches the original page...

Comment 15

[Eyal Rozenberg]

(In reply to comment #14)
> There's no reason an extension couldn't overlay code into the view-source
> window that touches the original page...

But is Hixie wrong in principle or can something be hacked up?

Comment 16

[Boris Zbarsky [:bzbarsky]]

He's wrong in principle as far as I can tell from the code we have.

We could ditch XUL and hack something up then, of course...  ;)

Comment 17

[Eyal Rozenberg]

(In reply to comment #16)
So what can be done about this? 

- Making more code reentrant?
- Using multiple processes sharing read-only data and only maintaining state for a number of tabbrowsers?
- "ditch XUL" :-( ?

Comment 18

[Matthew Gertner]

If we are talking about a blue sky solution, I definitely think that making the relevant code (DOM, XUL, etc.) more reentrant is the solution. Based on Boris's comments, this sounds like a very significant effort, but surely worth it in the long term.

Comment 19

[Boris Zbarsky [:bzbarsky]]

There is no way to make "DOM" more reentrant.  We're talking about scripts in web pages here -- we have no control of them.

In addition to ditching XUL we'd also have to disable window.open and all cross-frame script access, probably.

Comment 20

[Eyal Rozenberg]

(In reply to comment #19)
So what do you see as the solution to this issue?

Comment 21

[Boris Zbarsky [:bzbarsky]]

If I saw a solution I would have implemented it by now.  Take two and call me in the morning?  ;)

Comment 22

[Brendan Eich [:brendan]]

First, we could have concurrency among scripts and UI without violating the JS execution model, so long as scripts in windows reachable from one another run to completion.  I'm talking about content scripts here.  This kind of concurrency would be provided by any complete fix to bug 40848.  But that's both overkill and not enough: things could still lock up for a tab or set of tabs or other kinds of windows.

Second, "Ditch XUL" is too strong.  We could ditch the run-to-completion execution model shared globally, provided we did tons of other work to support concurrency, and let chrome scripts race with content scripts.  Then a content script could run a long time, and not starve the UI.  Still a lot of work.

Third, and most likely to happen soon: JS could grow support for concurrency of a limited kind, namely constrained call-with-current-continuation syntax that looks like Python generators.  I have some plans here, and I'll note the bug for this in due course.  But I want to ask here, would this really help?  Matthew, would you be willing to yield and resume your long operation by manually adding preemption points to it?

/be

Comment 23

[Brendan Eich [:brendan]]

(In reply to comment #22)
> Third, and most likely to happen soon: JS could grow support for concurrency of
> a limited kind, namely constrained call-with-current-continuation syntax that
> looks like Python generators.

The advantage here is that the script author chooses to break, or really relax, the default JS run-to-completion execution model.  But again, I'm not convinced this is something authors would do, and do well.  I'm encouraged because savvy authors have to break things up using event handlers and timeouts, and that sucks by comparison.  MochiKit has twisted-inspired Deferred objects, e.g.  All really nice, and something that should be built into JS.

/be

Comment 24

[Matthew Gertner]

(In reply to comment #22)
> Third, and most likely to happen soon: JS could grow support for concurrency of
> a limited kind, namely constrained call-with-current-continuation syntax that
> looks like Python generators.  I have some plans here, and I'll note the bug
> for this in due course.  But I want to ask here, would this really help? 
> Matthew, would you be willing to yield and resume your long operation by
> manually adding preemption points to it?

Definitely and this would be a big improvement over the status quo.

In a broader sense, I do feel strongly that Mozilla needs to be more concurrency friendly. I understand that things like scripts not blocking UI updates and isolating tabs so that one of them can't lock up the whole browser are big long-term efforts, but they are very much in line with the browser-as-a-platform philosophy that Mozilla seems to be (rightly) pursuing.

I would be in favor of setting up an umbrella bug to keep track of concurrency-related issues like 32482, 314635, making the MIME service threadsafe, etc., similar to what has been done for 1.8 memory issues. This way, we improve the chance of addressing these issues in future version of Gecko.

Comment 26

[Andrew McCreight [:mccr8]]

I think e10s fixed the complaint here, because content JS runs in a separate process than the UI. We also do interrupt content JS when it runs for too long.