610077 - Nested setTimeout should clamp to 4ms instead of 10ms (HTML5 spec)

Status: RESOLVED FIXED

(Core :: DOM: Core & HTML, defect, P2)

Description

[Trev]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.12) Gecko/20101026 Firefox/3.6.12
Build Identifier: Mozilla/5.0 (Windows NT 5.1; rv:2.0b8pre) Gecko/20101105 Firefox/4.0b8pre

According to the HTML5 spec, the setTimeout should clamp to 4ms if it is a nested setTimeout.  Look at #5 under "The setTimeout() method must run the following steps: " on http://www.w3.org/TR/html5/timers.html#timers.

I've made a test page that can determine what the setTimeout clamp is and I'll be attaching it.

The bug that set the clamp to 10ms is bug #123273.
The bug that disabled the clamp for <5 nested timeouts is bug #512645.

Reproducible: Always

Steps to Reproduce:
1. Run timeout.html in FF
2. Click the "Run Timeout with No Nesting"
3. You should see something around 2ms
4. Click the "Run Timeout with 5 Nesting"
5. You should see something around 10ms
Actual Results:  
With nesting, it is 10ms

Expected Results:  
With nesting, it should be 4ms

Comment 1

[Trev]

Attachment: timeout.html

Comment 2

[Boris Zbarsky [:bzbarsky]]

I don't believe that nsITimer gives us reasonable 4ms resolution (heck, on WinXP it doesn't even manage a reasonable 10ms!), so claiming to fire every 4ms would just be a lie.

Given further that Chrome is the only browser that does the 4ms thing, it's not clear to me that the spec text will stay as it is: it would require two interoperable implementations to do so...

Comment 3

[Trev]

It looks like we do get 2 ms resolution on WinXP based on my test case.  On Mac, I got 1 ms resolution.

And Opera does a 2ms clamp.

Comment 4

[Boris Zbarsky [:bzbarsky]]

> It looks like we do get 2 ms resolution on WinXP based on my test case.

No.  We just don't.  We're faking it with some 0-delay timeouts and some larger-delay ones.  See http://groups.google.com/group/mozilla.dev.tree-management/browse_thread/thread/c78af2ac978a80c/cf54b61444897195?lnk=gst&q=zbarsky+scroll#cf54b61444897195

Comment 5

[Trev]

Attachment: timeout.html

Comment 6

[Trev]

(In reply to comment #4)
> > It looks like we do get 2 ms resolution on WinXP based on my test case.
> 
> No.  We just don't.  We're faking it with some 0-delay timeouts and some
> larger-delay ones.  See
> http://groups.google.com/group/mozilla.dev.tree-management/browse_thread/thread/c78af2ac978a80c/cf54b61444897195?lnk=gst&q=zbarsky+scroll#cf54b61444897195

That test in google groups is dealing with setInterval.  setInterval is supposed to have a clamp of 10ms.  This bug is dealing with setTimeout.

I changed the test case to print out all the times that it recorded.  It does 100 iterations and looking through them for the "no nesting", almost all of them are 0, 1, or 2 ms.  Every now and then (1 out 100), I see a larger time.  But I see the same thing in Chrome and Opera.  And this is on WindowsXP.

In the "5 nesting", Firefox jumps to an average of 10ms while the other two stay lower.  I've made changes to the code and it is only when it hits a nesting of 5 that Firefox clamps at 10ms (which is because of bug #512645).

Comment 7

[Boris Zbarsky [:bzbarsky]]

> That test in google groups is dealing with setInterval.

It doesn't matter.  We use exactly the same code for both internally; intervals just automatically reset themselves.

> And this is on WindowsXP.

Interesting.  That hasn't been our experience on at least some XP systems...

Comment 8

[Boris Zbarsky [:bzbarsky]]

To be clear, I'd love to lower this clamping limit.  It's just that the data we have so far suggests we can't actually promise reliable timing at that frequency.  I'd love to be proved wrong!

Comment 9

[Trev]

(In reply to comment #7)
> It doesn't matter.  We use exactly the same code for both internally; intervals
> just automatically reset themselves.

I know it uses the same code.  What I meant is that if you are only looking at setInterval, you won't see it go faster than 10ms on average because of the clamp.  But if you use setTimeout, you'll be able to see it go faster than 10ms as long as you don't nest it 5 times.

Also, in Firefox 3.6 on Windows XP, it stays around 10-11 ms.  I'm guessing that is because bug #512645 is not fixed in 3.6.

Comment 10

[Trev]

Attachment: timeout.html

Updated test case.  This one shows a graph on how fast settimeout(,1).  It runs it 100 times with 1, 2, 4, 8, 16, 32, 64, and 128 concurrent timeouts.

Comment 11

[Trev]

Attachment: timer.png

This is the graph from my machine.  It is Windows XP SP3.  It pretty much hovers around 2ms.

Comment 12

[Trev]

Attachment: timeout.html

My bad.  I had posted one that still had some debugging stuff in it.

Comment 13

[Trev]

I created a thread over on forums.mozillazine.org asking for what others are getting with my test case.  The thread is at http://forums.mozillazine.org/viewtopic.php?f=23&t=2028011.

The responses so far show that <4ms resolution is possible with setTimeout in almost all cases.  The cases that went over 4ms still stayed under 10ms and looked to be when 8 or more concurrent timers were running.

There were:

12 - WinXP
1 - Vista
15 - Win7
1 - OSX
3 - Linux

Comment 14

[David Jackson]

Attachment: comparative settimeout test results for Mac OS X

I ran the latest Minefield, Webkit (+Safari 5.0.2), Chrome 9.0 and Opera 11 beta/alphas against the settimeout test on OS X, 10.6.4.  Minefield (but not Webkit, Chrome or Opera) was very spiky at higher timer numbers.

Comment 15

[David Jackson]

Attachment: comparative settimeout test results for Windows XP SP3

I ran the latest Minefield, Webkit (+Safari 5.0.2), Chrome 9.0 and Opera 11 beta/alphas against the settimeout test on Windows XP, SP3.  Graphs are not spiky with any browser.

Comment 16

[David Jackson]

Attachment: comparative settimeout test results for Windows 7

I ran the latest Minefield, Webkit (+Safari 5.0.2), Chrome 9.0, Opera 11 and IE9 beta/alphas against the settimeout test on 64-bit Windows 7.  Graphs were not spiky for any browser on this platform.

Comment 17

[Boris Zbarsky [:bzbarsky]]

OK, I don't see an existing bug on this, so confirming.

The XP data here is confusing me, since Microsoft's documentation is pretty clear: unless an app that uses high-resolution multimedia timers is running, the synchronization primitives XPCOM timers use (WaitForSingleObject) have 15ms resolution.  At least on XP; it's not clear to me what the Win7 situation is.  Note that this is the _resolution_.  When the timer actually fires depends on when the setTimeout call happens relative to the current tick.  Sometimes it'll actually fire before the time it's supposed to; sometimes it'll be 15ms later.

Chrome apparently uses a multimedia timer plus polling to get higher resolution on Windows; this means it's got an entire thread waking up every 1ms all the time just to handle this case.  Sucks for battery life and whatnot, but maybe that's the best we can do on Windows.

We need to investigate the behavior on other platforms too; the relevant thing there is the time resolution of pthread_cond_timedwait, I think.  On my fairly new mac, Chrome seems to not actually hit 4ms, by the way; it's closer to 4.5-4.7ms in a simple test that just sets a 0ms interval and waits until it fires 100 times.

That's the other thing we need here: tests.  The attached test is a good start, but we also need tests that do repeating timers, etc, that we can use to measure proposed implementations.  And we need to look at distributions, not just averages (because for a 1ms timer it's easy to have an average of 1ms with the times actually spread out between 0 and 15ms).

Comment 18

[Manoj]

(In reply to comment #17)
> OK, I don't see an existing bug on this, so confirming.
> 
> The XP data here is confusing me, since Microsoft's documentation is pretty
> clear: unless an app that uses high-resolution multimedia timers is running,
> the synchronization primitives XPCOM timers use (WaitForSingleObject) have 15ms
> resolution.

Just a slight update - the timer fires every 10ms on a single-processor machine and every 15ms on a multi-processor machine. The situation is the same for Windows 7.

Comment 19

[Boris Zbarsky [:bzbarsky]]

For those who want to experiment, I just checked in a new preference: "dom.min_timeout_value".  This is the clamp value, in ms.  It still defaults to 10, but you can lower it to 4 if you want to gather data on how timers behave as a result (looking for distributions here, not just averages).  If people do some measurements on that, it would be much appreciated!

The preference should be available in about:config starting with tomorrow's nightlies.

Comment 20

[Trev]

Attachment: Adds setTimeout, nested setTimeout, and standard deviation calculations

This new test file can test either setTimeout, a nested setTimeout, or setInterval.  It calculates the standard deviation as well as the average.  You can specify how many times you want the timer to fire (default 100) and the maximum number of concurrent timers to run (default 128).  It always starts at 1 timer and doubles until it reaches the maximum.

Comment 21

[Trev]

I've changed my timeout to 4ms with the new preference (thanks for that).  Running my new test file, I get (for 100 iterations with 128 maximum timers):

setTimeout:
Overall, Mean: 2.2743933216851095ms, Std Deviation: 1.4901225665962885

Nested setTimeout:
Overall, Mean: 4.770529994175888ms, Std Deviation: 0.8585579189237926

setInterval:
Overall, Mean: 4.002912055911474ms, Std Deviation: 0.8241044566458109

On Chrome, I get:
setTimeout:
Overall, Mean: 4.777363618714813ms, Std Deviation: 0.6123614340937168

Nested setTimeout:
Overall, Mean: 4.887866433702194ms, Std Deviation: 0.33833977093348533

setInterval:
Overall, Mean: 4.880100951271598ms, Std Deviation: 0.3417422287820634

This on Windows XP - Mozilla/5.0 (Windows NT 5.1; rv:2.0b10pre) Gecko/20110115 Firefox/4.0b10pre.

So, somehow, Firefox 4 on my Windows XP box handles 4 ms intervals fine.

Comment 22

[Emanuel Hoogeveen [:ehoogeveen]]

Things also look good here on Windows 7 x64, with the occasional spike (as high as 50ms). Since they heavily affect the standard deviation, it may be worth adjusting the calculation to cut off the top and bottom 1% of values. Either that, or a fancier iterative calculation where you cut off any values outside n standard deviations, then calculate the standard deviation again (probably overkill for this test).

Oddly, an earlier test I did was all over the place - not sure what caused that. I've rerun the test a few times at 2500 iterations and things look good.

I also ran the test with BOINC running Einstein@Home in the background. Everything looked good until the 128 timer test, which was all over the place (average 18.6ms, standard deviation of 9.7ms). Since the others were okay however, there's probably not much that can be done about that.

Comment 23

[Boris Zbarsky [:bzbarsky]]

The tests in comment 21 and 22 were done with no other apps running, right?  Some applications using multimedia timers can affect how the normal timer stuff works on Windows.

Comment 24

[Trev]

I restarted my machine and only opened up FF4 to run the tests.  The results are quite a bit different:

setTimeout:
Overall, Mean: 2.103164434090468ms, Std Deviation: 1.5039603964926858

Nested setTimeout:
Overall, Mean: 7.196777324791302ms, Std Deviation: 5.547820771342595

setInterval:
Overall, Mean: 8.629275868763347ms, Std Deviation: 6.666975651622262

So, the single setTimeout results don't change (which is weird), but the other two are all over the place (they look like they bounce from 15 ms to 0 ms over and over again).

Comment 25

[Emanuel Hoogeveen [:ehoogeveen]]

Wow, yeah. Before rebooting I was getting 4ms +/- 0.3ms or so, with the occasional spike. After rebooting, I seem to be getting either 0ms or 10ms - the ratio changes between the different tests, but it's clear from the graphs. So something must have triggered the 'good behavior'.

This is a guess, but user mode timing is system-wide - I'm guessing some application set it to 1ms using timeBeginPeriod() (down from the default 10ms) and never called timeEndPeriod() to reset it. Since it seems to affect whatever timers Firefox is using, maybe Firefox should call it during initialization? I'm not sure what the Linux or MacOS alternatives would be.

Comment 26

[Boris Zbarsky [:bzbarsky]]

Your hypothesis about timeBeginPeriod() is almost certainly right (and yes, it would cause the numbers to look like comment 24 and comment 25).  I'll guess that [e-mail] has a multicore machine and Emanuel has a single-core one?  ;)

The problem with calling this during Firefox init is this, from the documentation:

  However, it can also reduce overall system performance, because the thread
  scheduler switches tasks more often. High resolutions can also prevent the
  CPU power management system from entering power-saving modes.

I suppose we could only call it when we need a timer that's shorter than 10ms or something.... I'm not sure how quickly calling this function takes effect.  Ccing some folks who know more about winapi.

On Mac and Linux, I think we still need data on what the situation looks like.  At least on Linux, the scheduler timeslice is 1ms, not 10ms, so we may in fact get 1ms accuracy on the pthreads waits.

Comment 27

[Boris Zbarsky [:bzbarsky]]

One other note.  I've been thinking about clamping timeouts differently in foreground tabs and background tabs.  Thoughts on that?

Comment 28

[Emanuel Hoogeveen [:ehoogeveen]]

Actually my CPU is a quad-core. If you were referring to my comment about BOINC, it runs 4 applications at the same time ;)

Comment 29

[Boris Zbarsky [:bzbarsky]]

No, I was referring to the MS documentation about when the tick is 10ms vs 15ms...  But whatever; I'm not sure how much I trust their documentation, necessarily.

Comment 30

[Manoj]

(In reply to comment #27)
> One other note.  I've been thinking about clamping timeouts differently in
> foreground tabs and background tabs.  Thoughts on that?

My thoughts:
1. Introducing this dichotomy adds the headache of ensuring that nested timeouts are clamped to the new value when the current tab goes into the background. Same holds for a background tab that comes to the foreground. 
2. Would you be able to reliably and uniformly enforce the lowered timeout clamp for all nested timeouts in the new foreground tab?

IMHO, that the foreground tab has the value clamped to as close a value as the desired timeout duration is the main consideration here. The background tab stuff is goodness.

Comment 31

[Boris Zbarsky [:bzbarsky]]

> Introducing this dichotomy adds the headache

Well, sure.  That's an implementation detail; I was more interested in the authoring angle.

> Would you be able to reliably and uniformly enforce the lowered timeout

Well, that's the reason this bug isn't fixed yet.... ;)

Comment 32

[Boris Zbarsky [:bzbarsky]]

I filed bug 633421 on comment 27.

Comment 33

[Mark Funk]

IE9 changed to 4ms resolution timers when its RC was released.
See the chart on their blog: http://blogs.msdn.com/b/ie/archive/2011/02/10/acting-on-feedback-ie9-release-candidate-available-for-download.aspx

Comment 34

[Boris Zbarsky [:bzbarsky]]

Yes, you don't have to evangelize this.  We want to make the change.  The question is how to fix the timer code to make it possible.  ;)

Comment 35

[Boris Zbarsky [:bzbarsky]]

I filed bug 640796 to deal with Windows timers; once that's done we need to remeasure see how things look.

Comment 36

[Boris Zbarsky [:bzbarsky]]

Attachment: Just do it

Comment 37

[Dão Gottwald [:dao]]

From http://blogs.msdn.com/b/ie/archive/2011/03/28/browser-power-consumption-leading-the-industry-with-internet-explorer-9.aspx :

"[...] The exception is Opera 11 which is consuming about 5% more power than other browsers when idle. One reason for this is that Opera changes the system timer resolution from the default 15.6ms to 2.5ms which prevents the CPU from entering low power states."

Not sure if this is directly relevant for this bug.

Comment 38

[Boris Zbarsky [:bzbarsky]]

It's not.  It's relevant to bug 640796, though.  And we do know about the issue; see the discussion in that bug.

Comment 39

[Shane Bundy]

http://hg.mozilla.org/mozilla-central/rev/231d47984200

Comment 40

[Boris Zbarsky [:bzbarsky]]

Indeed.

Comment 41

[Eric Shepherd [:sheppy]]

Documentation updated:

https://developer.mozilla.org/en/DOM/window.setTimeout#Minimum_delay_and_timeout_nesting

Also mentioned on Firefox 5 for developers.